Fast doesn't update my site

last month shows no pages indexed - why?

[latimer]

Have been tracking our number of pages in the alltheweb index closely in hopes that alltheweb will begin providing yahoo results in sept. What I have noticed is that when you go to advanced search and type in search only for www.yoururl.com and then use the option of searching for: any date, last month, last 3 months, last 6 months, last year, we have seen the number of pages that appear in the last month option drop each of the last two indexes. So now we still show lots of pages, but they are all older than at least a month. Why doesn't alltheweb crawl all the pages and update fresh to the index, rather than create this huge index of old pages? Sounds impressive when they say how many pages they have, but the quality of the index is lacking. Any thoughts on what the strategy is here? Can we expect a deep crawl and fresh update from alltheweb anymore?

[Rumbas]

We can for sure expect updates and respidering from Fast in the future. That's what they do; crawl and update.

I agree though that the updates/refreshs of the index has been somewhat unstable and lacking from time to time. In fact we have a thread [webmasterworld.com] running to keep a little track of them.

[heini]

Well - despite repeated announcements from Fast and resulting speculations here on the board I must admit I still don't know exactly what Fast updates when.

It's clear that the db is devided in several segments of freshness.

Freshest is of course the segment with the sites crawled for the news search - those are crawled permanently.

Next is the db segment with the PFI pages - refreshed each 24-48 hours.

After those comes a chunk which supposedly gets updated each 7-10 days. This is what Fast refers to as popular content.
What's in this segment? How do you get in there? CTR? If so from ATW - or Lycos? Fresh content on the sites? Linkpop?

Finally - and that's where the lights get dimmer and the moaning gets louder - comes a base of urls which sometimes it takes months to get updated.

That's where we're at. Any comments and further insights welcome.