homepage Welcome to WebmasterWorld Guest from 23.22.29.137
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
EasouSpider
keyplyr




msg:4514463
 1:38 am on Nov 1, 2012 (gmt 0)

Attempted site scrape. I block all known China ranges. Took about 50 403s very fast before it quit.

robots.txt: no
auto-referrer

Chinanet
183.0.0.0 - 183.63.255.255

 

keyplyr




msg:4514768
 7:13 pm on Nov 1, 2012 (gmt 0)

Day 2 - Still eating 403s, same IP: 183.60.212.160

dstiles




msg:4514798
 8:33 pm on Nov 1, 2012 (gmt 0)

I have a note of several bad hits within 183.60.0.0/16 so have blocked the full /16.

Bewenched




msg:4516693
 6:58 am on Nov 7, 2012 (gmt 0)

Had a visit the other day from them as well. Thanks guys... blocked and chasing their tails.

Some piss me off enough that I forward them on to another bot ip ... lol.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved