homepage Welcome to WebmasterWorld Guest from 54.204.79.235
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
polybot 1.0
Froggyman




msg:402143
 9:12 pm on May 22, 2001 (gmt 0)

Reporting a new (?) robot.

outland.poly.edu (128.238.32.77)
polybot 1.0

Ignored my non-existant robot.txt file. :)

 

lizzie




msg:402144
 4:19 am on May 23, 2001 (gmt 0)

I saw polybot visited several of my domains today.
What could it be?

Froggyman




msg:402145
 7:15 am on May 23, 2001 (gmt 0)

Maybe this:
[poly.edu...]

lizzie




msg:402146
 12:11 pm on May 23, 2001 (gmt 0)

wow

Son_House




msg:402147
 9:18 am on May 26, 2001 (gmt 0)

We have also been visited by polybot. Made 24 requests including one for robots.txt

Here is a little more info: [cis.poly.edu...]

Brett_Tabke




msg:402148
 3:03 am on May 27, 2001 (gmt 0)

Great another agressive rogue research crawler. Just what the web doesn't need.
Banned em.

Froggyman




msg:402149
 5:15 am on May 27, 2001 (gmt 0)

They must be watching if they're pulling robot.txt files now.

Son_House




msg:402150
 7:28 am on May 27, 2001 (gmt 0)

Here are three more ip's:

weasel.poly.edu (128.238.55.7)
gator.poly.edu (128.238.55.6)
ferret.poly.edu (128.238.55.8)

All have the same ua as outland.poly.edu
polybot 1.0

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved