homepage Welcome to WebmasterWorld Guest from 54.145.183.169
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
what kind of spider is it if at all?
nube

10+ Year Member



 
Msg#: 325 posted 12:33 am on Jan 27, 2001 (gmt 0)

suproxy.google.com - - [26/Jan/2001:17:04:59 -0500] "GET /...html HTTP/1.0" 200 13087 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

is it a spider or google personel surfing!
thanks g

 

skirril

10+ Year Member



 
Msg#: 325 posted 2:27 pm on Jan 27, 2001 (gmt 0)

That seems to be a spider.

Google personnel surfing does not show up as googlebot.

Also, would googlep personnel request a robots.txt file?

nube

10+ Year Member



 
Msg#: 325 posted 2:42 pm on Jan 27, 2001 (gmt 0)

thanks, for the info i just never seen suproxy,
i always see crawler#.googlebot

thank you again
g

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved