FAST Updates Index, and starts new spidering cycle

Forum Moderators: open

Message Too Old, No Replies

FAST Updates Index, and starts new spidering cycle

redzone

4:51 pm on Jan 19, 2001 (gmt 0)

We had several new domains update in both FAST and Lycos on 1/18, and FAST kicked their spiders into "overdrive" late last night.

If Google updates on 01/20, and kicks their spider's off on the 21st, everyone better have their servers "tuned" for some peak loads... :)

rzfree

8:58 pm on Jan 19, 2001 (gmt 0)

Hi Redzone, and thanks

I read elsewhere that Lycos is slow in picking Fast's database, like 2-3 months. Is this still true?

redzone

2:19 pm on Jan 20, 2001 (gmt 0)

The new domains that just indexed in FAST, are already receiving clicks in Lycos... :)

Brett_Tabke

3:41 pm on Jan 20, 2001 (gmt 0)

You can play along with us on the home version:

I just submitted this thread to Fast...

rzfree

7:59 pm on Jan 20, 2001 (gmt 0)

Assuming AV dislike new pages some, Fast/Lycos could be important part of the game to those like me trying to get maximum off free engines. Sure will be watching this thread appear in Lycos.
Thanks R

mrVW

7:35 am on Jan 23, 2001 (gmt 0)

Yep, they have been hitting my sites all week, I see the results on the main alltheweb.com site but not on lycos.com yet.

Brett_Tabke

10:31 am on Jan 23, 2001 (gmt 0)

And they are live [alltheweb.com].

Full crawl on Sunday morning.

Brett_Tabke

9:34 am on Jan 24, 2001 (gmt 0)

They recrawled here, search engine world, and dozens of other sites in the last 24. They pulled a huge number of pages here in a short time frame yesterday - they have really got that thing cranked up.

Brett_Tabke

4:01 pm on Jan 29, 2001 (gmt 0)

Followup:

I got to looking at logs from last week, and that update was big for us across the board. Sympatico continues to crank out the Fast referrals for us. Same terms, same rank, sympatico is beating lycos. We have the fast-sympatico/lycos algo very close right now - as close as I've come to an algo in a year). As far as I can deduce, they are both serving up identicle (sans paid 'feature links'). Cloaking is working well - in fact, me and several members here are ranked high and cloaked under several dozen identicle terms - smile rz ;)

Anyone willing to share? What is the major difference between Lycos/sympatico and Fast proper? All those where we are top 3 in Lycos/sympatico, are having near-death-experiences in Fast proper.

msgraph

4:16 pm on Jan 29, 2001 (gmt 0)

>All those where we are top 3 in Lycos/sympatico, are having near-death-experiences in Fast proper.

I've been knocked out of FAST since the beginning of Jan. for reasons unknown to me, but somehow my Lycos rankings are improving.

Do you think Lycos takes results from FAST and tweaks them around a bit?

Or could it be they are reshuffling old FAST results until they get a new batch?

luckynh

4:58 pm on Jan 29, 2001 (gmt 0)

I have a new site that has never be submitted to Fast/ATW

So I'll submit and run a new time line and see what happens.

This site is only in ODP.
Not in anyother SE.
Not linked from any site
NO linkpop.
<edit>
This site has been banned from AV
</edit>

Let's take a spin and see where we wind up....

Lucky

han solo

7:37 pm on Jan 29, 2001 (gmt 0)

That was fun, I just found my "user agent" moniker, what have you, "han solo" in a search at alltheweb.

Searching for webmasterworld, and then my name here, brings up a couple a pages...pretty neat.

Cheers,

Han Solo

BTW, care to share where the stuff is listed by both of you? Am I there too? Or does my stuff not compete well in the new fast? ;)

Brett_Tabke

9:00 pm on Jan 29, 2001 (gmt 0)

lol, Getting a bit personal there hans ;-) ...in some of the seo/se kw's.
btw: since you were searching, this here site sucks in fast - I believe it is the word "forum" in the urls. That's ok, I don't stress out about it. I'd rather stay below the radar.

startup

6:00 am on Jan 30, 2001 (gmt 0)

This is your standard Alltheweb search string for
the search term toys:
[alltheweb.com...] . Take a look and you will notice the number of documents is not listed.

Okay, now use this search string:
[alltheweb.com...] I have taken the "type=all&" out of the first string and the results match what the partners are displaying, also note the # of documents is displayed.

At this point, it seems that Fast has 2 databases or their partners only search part of the main database. I believe it is 2 dbs, because all the partners produce the same results. Some of them compete with each other, so if they could tweak the results,"ink", they would.

Now the interesting question. How does a site get listed in the partners db? I don't know, but at this point I believe it is a time factor.

NFFC

1:40 pm on Jan 30, 2001 (gmt 0)

Well spotted startup,

>Now the interesting question

Can I add click tracking to the list of possible reasons.

rzfree

11:51 pm on Feb 1, 2001 (gmt 0)

"I just submitted this thread to Fast... "

Well it's there [alltheweb.com], took 12 days. Hello Lycos ?

luckynh

6:53 pm on Feb 13, 2001 (gmt 0)

"I have a new site that has never be submitted to Fast/ATW"

Submited on 1-29 to fast
Found on 2-12 at lycos

Not bad turn around time

luckynh

6:54 pm on Feb 13, 2001 (gmt 0)

"I have a new site that has never be submitted to Fast/ATW"

Submited on 1-29 to fast
Found on 2-12 at lycos

Not bad turn around time
Listing every page