Welcome to WebmasterWorld Guest from 54.147.232.40

Forum Moderators: open

Message Too Old, No Replies

Fast and Google on the Crawl

Both out crawling today

     
3:43 pm on Mar 11, 2003 (gmt 0)

Junior Member from GB 

10+ Year Member Top Contributors Of The Month

joined:Oct 16, 2002
posts:182
votes: 3


Fast and Google are both on the prowl today:

reqs: pages: browser
58: 57: FAST-WebCrawler
58: 57: FAST-WebCrawler/3
53: 38: Googlebot
53: 38: Googlebot/2

cr004r01-test.sac2.fastsearch.net - - [11/Mar/2003:13:14:10 +0000] "GET /robots.txt HTTP/1.0" 200 71 "-" "FAST-WebCrawler/3.6 (atw-crawler at fast dot no; [fast.no...]

216.239.46.85 - - [11/Mar/2003:00:45:58 +0000] "GET /robots.txt HTTP/1.0" 200 10037 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

3:44 pm on Mar 11, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 13, 2002
posts:676
votes: 0


ZyBorg's been out in full force too.

G.