homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
Forum Library, Charter, Moderator: open

Deprecated - Altavista, Alltheweb.com Forum

This 55 message thread spans 2 pages: 55 ( [1] 2 > >     
Where's the update then?
I'm still waiting

 12:10 am on Jul 1, 2002 (gmt 0)

Well, FAST did a major crawl at the same time as Google, but it is showing no signs of updating to use that crawl information yet.

The current database is still showing pages that we removed 2 months ago, and none of the new pages have appeared yet that were crawled.

Has anyone been tracking the FAST update cycles? I suppose that it will become much more important if/when Yahoo swaps over.



 12:14 am on Jul 1, 2002 (gmt 0)

Well, I just had a big crawl by Fast. So that is a good question. I though they would release an index before they start crawling again.


 4:20 pm on Jul 2, 2002 (gmt 0)

I had a crawl about 4 days ago


 4:25 pm on Jul 2, 2002 (gmt 0)

It's quite hard to follow when FAST updates their index, they do it quite often and they don't always refresh the whole database, often they update 25% of the index every week, but usualy you will see a "BIG" update once a month.
Also FAST crawls up to 150 Million pages a day so it's a pretty big chance they will visit your page.


 5:12 pm on Jul 2, 2002 (gmt 0)

Still, none of the pages spidered in the last big crawl at the start of the month have appeared yet, and our dead pages are still appearing in the SERPs.


 5:32 pm on Jul 2, 2002 (gmt 0)

Abrexa, as Lazer suggests it's hard to see a clear pattern with FAST's update cycles.
Obviously they are not (yet?) updating the complete 2,1 Bill pages in one go.
Statements from FAST indicate there's a popular section of the db, which gets updated more frequently, ca every 9 days, than the rest.
See this thread:
FAST / ATW update cycles [webmasterworld.com]


 7:49 pm on Jul 2, 2002 (gmt 0)

Thanks, I did follow that thread at the time.

It just seems strange that there is such a large time span between the crawl and the update - longer than Google even. FAST took the entirety of our new UK directory...so I am just keen to see how much difference this makes.


 8:30 pm on Jul 2, 2002 (gmt 0)

Not even the paid has been updated and it's been well over a week.


 8:38 pm on Jul 2, 2002 (gmt 0)

Abrexa, sorry, I see you were posting in that thread. Had it flagged and didn't read again.

Marcia, paid pages not updating is not what you paid for. I would contact Lycos, whom you have a deal with...

One thing is for sure, the FAST PFI model as resold and handled by Lycos.com and Lycos.Europe needs some serious reevaluation.


 12:54 pm on Jul 3, 2002 (gmt 0)

Not even the paid has been updated and it's been well over a week.

Our PFI sites are always updated in 24-48 hours. We have a bulk submission deal with Lucos Europe.


 12:55 pm on Jul 3, 2002 (gmt 0)

Sorry, it`s Lycos not Lucos...


 12:27 pm on Jul 7, 2002 (gmt 0)

It has been a week since I first posted this issue - and FAST still hasn't updated. Our dead pages are still there, and none of the new ones have appeared.

Are we in the quiet period before a massive update? There are people complaining about the PFI pages not being spidered/updated as well. This lends me to believe that either:

a. There is some heavy work going on behind the scenes that will bring a major update to their database.


b. FAST is a shambles at the moment and they can't run a scheduled PFI service or update their index properly.

But seriously, the problem with long delays between a crawl and an update is that the content becomes staler every day.

Google suffers from the same problem to some extent - a 3/4 week delay means that many pages are out of date. If FAST's update period is going to be 5 weeks or more, then their content is never going to be as fresh as Google's.


 11:36 am on Jul 16, 2002 (gmt 0)

And another week. My faith in FAST is fading. If they don't hurry up, Google will have updated *twice* in the same amount of time it takes Alltheweb to do a single update.


 6:23 pm on Jul 16, 2002 (gmt 0)

I have a few pages indexed in fast that are no less than 6 months old. I basically ignore fast.


 5:20 pm on Jul 19, 2002 (gmt 0)

Any luck, Abrexa_UK?


 10:08 pm on Jul 21, 2002 (gmt 0)

Just checked again (11pm GMT, 21st July) and still nothing - same old dead pages and no new ones.

You can rest assured that Fast's update time isn't a patch on yours :)

I know that you probably don't want the competition, but I really want Fast to be a decent competitor to keep Google on its toes. As you may have read elsewhere, I do hope that they got the Yahoo deal. You don't need it any more, given your total domination of the search engine industry ;)


 2:52 am on Jul 31, 2002 (gmt 0)

31st july, 3.48 am GMT. No update yet - same dead pages still listed.

Google has updated twice now while we have been waiting for Fast to update. :(


 1:01 pm on Aug 1, 2002 (gmt 0)

AllTheWeb has updated, the new build is also showing over at Lycos.com.

brotherhood of LAN

 1:12 pm on Aug 1, 2002 (gmt 0)

hmmm Im number 1 for a single keyword with ATW now....anyone have a link to their "dance card" list on WebmasterWorld?


 1:21 pm on Aug 1, 2002 (gmt 0)

You sure about that rubik? I made a could of spotchecks and could see the update?

<added>BOL, looking for this [webmasterworld.com]? </added>


 2:36 pm on Aug 1, 2002 (gmt 0)

FAST Portal Partners 2002 [webmasterworld.com]


 2:40 pm on Aug 1, 2002 (gmt 0)

there seems to have been an update for my keywords....i did the lycos express submit domain for $39...it seems to be rubbish as the last few updates ive moved from 50th - 3rd - gone - 20th???? im 5th - 9th in google on every update...........i have plenty of links so would it be better to not do the submit when the express runs out? Do you get your whole site in eventually as only my domain name is in at the moment.


 2:49 pm on Aug 1, 2002 (gmt 0)

hmmm Im number 1 for a single keyword with ATW now....

congratulations BOL, that would not be hard to guess given your profile.

Any idea what made it happen?
I would be interested in what type of traffic such a one word No1 ranking gives within ATW, if you would be willing to share..

brotherhood of LAN

 6:19 pm on Aug 1, 2002 (gmt 0)

>>Any idea..

The only thing I have changed is use a little more CSS and some absolute DIV's, relatively same old cwap design........just sometimes re-arranged what order the content falls in.

I don't think the ATW number 1 will amount to much, but I get not bad traffic from Lycos.....so I'll check it out and try to measure what sort of traffic comes from it.......and post here

I'd be willing to share because I have nothing to lose ;)



 6:24 pm on Aug 1, 2002 (gmt 0)

Well, our listings are the same as before - it is still listing the pages that have been dead for months now, but none of the 15,000 pages crawled at the start of June :(

brotherhood of LAN

 10:02 pm on Aug 1, 2002 (gmt 0)

Evidently the SERP's have changed again.

Not sure what is going on then....seems like little algo tweaks before/after an update...im at a loss :)


 6:17 pm on Aug 11, 2002 (gmt 0)

Abrexa_UK, so you're saying that even after a FAST update you're still seeing old content from your site? Have you seen any update at all?


 7:02 pm on Aug 11, 2002 (gmt 0)

Well, frankly I find FAST a strange beast indeed. I have sites that are integrated well into the web... large sites, very well respected for their informative content... yet they are somehow missed.

Yes, they are in Yahoo, ODP, and a lot of other serious portals, including a number of academic ones (they are not overtly commercial). No, they are not banned or penalized in any way (nor should they be). Yet they don't appear. Much more flimsy efforts, from my old stock, are in there.

I've been waiting about 6-8 months and have all but given up. I find FAST very slow to index at the best of times. I live with it because I won't pay. Bt the missing site syndrome is a strange one.


 9:31 pm on Aug 11, 2002 (gmt 0)

Well to answer the title's question - where's the update then -: updates have happened.

Fast - as we have discussed in several threads - apparently does not update a complete db in one go but in portions.

Here's my preliminary explanation of Fast update behaviour from one of those threads [webmasterworld.com]:

It's clear that the db is devided in several segments of freshness.

Freshest is of course the segment with the sites crawled for the news search - those are crawled permanently.

Next is the db segment with the PFI pages - refreshed each 24-48 hours.

After those comes a chunk which supposedly gets updated each 7-10 days. This is what Fast refers to as popular content.
What's in this segment? How do you get in there? CTR? If so from ATW - or Lycos? Fresh content on the sites? Linkpop?

Finally - and that's where the lights get dimmer and the moaning gets louder - comes a base of urls which sometimes it takes months to get updated.

Abrexa's site obviously did not belong to the portion with the fast update cycle.

Napoleon, that's strange. I've not yet come across well linked sites, listed in ODP, that didn't make it into Fast sooner or later.
Any idea what might hinder Fast from indexing your sites?

GG - how about your sites - you too encounter problems with Fast?


 1:28 am on Aug 12, 2002 (gmt 0)

Alltheweb is still listing the same pages it had on 1st June when it began a complete crawl of our site/directory. It still has all the dead pages that were removed some time in April (maybe even before that!). It isn't just "old" content, it is dead, buried and decomposed content.

It is now something like 70 days since the full crawl and still no update for us.

What I don't understand is that Fast HAS crawled the site and HAS updated, probably a couple of times, since then, but it simply hasn't used any of the crawl information for our site.

This 55 message thread spans 2 pages: 55 ( [1] 2 > >
Global Options:
 top home search open messages active posts  

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved