| Welcome to WebmasterWorld Guest from 18.104.22.168 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|what kind of spider is it if at all?|
suproxy.google.com - - [26/Jan/2001:17:04:59 -0500] "GET /...html HTTP/1.0" 200 13087 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
is it a spider or google personel surfing!
That seems to be a spider.
Google personnel surfing does not show up as googlebot.
Also, would googlep personnel request a robots.txt file?
thanks, for the info i just never seen suproxy,
i always see crawler#.googlebot
thank you again
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved