Page is a not externally linkable
- Yahoo
-- Deprecated - Altavista, Alltheweb.com
---- FAST crawler - very strange behaviour


Vlas_L - 1:13 am on Feb 11, 2003 (gmt 0)


Has anyone seen FAST Crawler request the same page hundreds of times a day?

Here are the stats:

UA - FAST-WebCrawler/3.6 (atw-crawler at fast dot no; [fast.no...]

IP - 66.77.73.77

Number of requests for the same page
February 7, 2003 559
February 8, 2003 771
February 9, 2003 699
February 10, 2003 386
(there are still about 7 hrs left today - PST)

Has anyone seen this before?

How is FAST about feedback?

I've been a little hesitant about contacting them as they have been very good to me. Wouldn't want to change any of that - most of my content is targeted at Canadians.

Thanks for any insight.


Thread source:: http://www.webmasterworld.com/altavista/799.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com