homepage Welcome to WebmasterWorld Guest from 54.196.194.204
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Fast and Google on the Crawl
Both out crawling today
Made In Sheffield




msg:193534
 3:43 pm on Mar 11, 2003 (gmt 0)

Fast and Google are both on the prowl today:

reqs: pages: browser
58: 57: FAST-WebCrawler
58: 57: FAST-WebCrawler/3
53: 38: Googlebot
53: 38: Googlebot/2

cr004r01-test.sac2.fastsearch.net - - [11/Mar/2003:13:14:10 +0000] "GET /robots.txt HTTP/1.0" 200 71 "-" "FAST-WebCrawler/3.6 (atw-crawler at fast dot no; [fast.no...]

216.239.46.85 - - [11/Mar/2003:00:45:58 +0000] "GET /robots.txt HTTP/1.0" 200 10037 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

 

Grumpus




msg:193535
 3:44 pm on Mar 11, 2003 (gmt 0)

ZyBorg's been out in full force too.

G.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved