Welcome to WebmasterWorld Guest from 107.20.75.63

Forum Moderators: open

Message Too Old, No Replies

Fast badly needs more updates

     
3:14 pm on Dec 12, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:July 5, 2002
posts:602
votes: 0


I have been watching the listings for some of our sites in Fast/Alltheweb for some some time, and IMHO, Fast is pretty poor at updating.

As an example, we closed down a site over 6 months ago, possibly even up to 9 months ago. Yet when I checked, Alltheweb still has 271 pages listed from the site.

Checking for one of our live sites which has been up for a year, with many thousands of pages, reveals just 70 pages. The same story applies to a number of our other sites. It almost looks like it is just the first layer of pages that have been indexed and the spider isn't following the links downward, despite a strong and clear linking structure on each site.

Surely it is time for Fast to up the pace of spidering?

9:50 am on Dec 13, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 23, 2002
posts:165
votes: 0


Following some changes, my site was indexed properly (tens of thousands of pages) by FAST about three weeks ago, and it has since spidered and reindexed - which I can tell from (a) the number of pages on particular searches and (b) the freshness of some of the content.

However I agree that it does not seem to be too good at removing outdated pages - we have a churn rate of around 2500 pages a month (new data arriving and old data expiring)and it looks to me as though the old data is still there.

9:56 am on Dec 13, 2002 (gmt 0)

Senior Member from ZA 

WebmasterWorld Senior Member 10+ Year Member

joined:July 15, 2002
posts:1720
votes: 1


I have a site that has only just been fully indexed by Fast in the last month or so, it has been online since last summer. The linking structure hasnt changed on the site since it was first went live.

It was fully indexed by Google, Altavista (which amazed me) and Inktomie all with in the first 3 months of the site going live... it just seemed that fast had a problem with the site.

Craig

[edited by: creative_craig at 10:01 am (utc) on Dec. 13, 2002]

10:00 am on Dec 13, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 27, 2002
posts:1685
votes: 0


Earlier today e-mail'd taskforce on this very issue. It would seem that the folks in Norway might just be preparing to do an update.

Pendanticist.

3:34 pm on Dec 13, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:July 5, 2002
posts:602
votes: 0


At least one of the sites has been fully crawled and then not updated, which is even stranger. Perhaps we are doing something to upset the Fast spider, but it seems strange that they would index some pages and not others.