homepage Welcome to WebmasterWorld Guest from 54.167.41.199
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
Forum Library, Charter, Moderator: open

Deprecated - Altavista, Alltheweb.com Forum

    
Fast badly needs more updates
Bobby_Davro




msg:222833
 3:14 pm on Dec 12, 2002 (gmt 0)

I have been watching the listings for some of our sites in Fast/Alltheweb for some some time, and IMHO, Fast is pretty poor at updating.

As an example, we closed down a site over 6 months ago, possibly even up to 9 months ago. Yet when I checked, Alltheweb still has 271 pages listed from the site.

Checking for one of our live sites which has been up for a year, with many thousands of pages, reveals just 70 pages. The same story applies to a number of our other sites. It almost looks like it is just the first layer of pages that have been indexed and the spider isn't following the links downward, despite a strong and clear linking structure on each site.

Surely it is time for Fast to up the pace of spidering?

 

HenryUK




msg:222834
 9:50 am on Dec 13, 2002 (gmt 0)

Following some changes, my site was indexed properly (tens of thousands of pages) by FAST about three weeks ago, and it has since spidered and reindexed - which I can tell from (a) the number of pages on particular searches and (b) the freshness of some of the content.

However I agree that it does not seem to be too good at removing outdated pages - we have a churn rate of around 2500 pages a month (new data arriving and old data expiring)and it looks to me as though the old data is still there.

creative craig




msg:222835
 9:56 am on Dec 13, 2002 (gmt 0)

I have a site that has only just been fully indexed by Fast in the last month or so, it has been online since last summer. The linking structure hasnt changed on the site since it was first went live.

It was fully indexed by Google, Altavista (which amazed me) and Inktomie all with in the first 3 months of the site going live... it just seemed that fast had a problem with the site.

Craig

[edited by: creative_craig at 10:01 am (utc) on Dec. 13, 2002]

pendanticist




msg:222836
 10:00 am on Dec 13, 2002 (gmt 0)

Earlier today e-mail'd taskforce on this very issue. It would seem that the folks in Norway might just be preparing to do an update.

Pendanticist.

Bobby_Davro




msg:222837
 3:34 pm on Dec 13, 2002 (gmt 0)

At least one of the sites has been fully crawled and then not updated, which is even stranger. Perhaps we are doing something to upset the Fast spider, but it seems strange that they would index some pages and not others.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved