Today's web log at the office reveals a pile of requests from "FAST-RealWebCrawler/3.8 (crawler@fast.no; [fast.no...] Note the insertion of "Real" before "WebCrawler". It's operating off of cr010r01-test.sac2.fastsearch.net. Close to 300 requests so far, in approx. 150 minuts, all apparently to areas of the website that are okay.
claus
6:41 pm on Aug 27, 2003 (gmt 0)
And here is yet another:
UA: FAST-WebCrawler/3.7/FirstPage (atw-crawler at fast dot no;http://fast.no/support/crawler.asp)
IP: 66.77.73.89
/claus
added:
Just found this thread with a few more fast firstpage UAs: [webmasterworld.com...]
