homepage Welcome to WebmasterWorld Guest from 54.204.58.87
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Squider crawling from many IPs, may be a botnet
major hitter over the past ten days
dstiles




msg:4665360
 8:42 pm on Apr 23, 2014 (gmt 0)

Over 400 hits since April 10th, all (I think) blocked. There may be some from server farms but those are not specifically logged here, just rejected. The appearance is of a botnet running with the UA below. It hits several times in a row with different IPs. Blocked on empty header fields.

UA:
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36 Squider/0.01

Almost all IPs were France or Brazil with a couple of US and Belgium. Very unusual pattern (I'm in UK).

 

incrediBILL




msg:4672102
 9:24 pm on May 18, 2014 (gmt 0)

Yup, when I looked after you mentioned it there's a huge mix of IPs. It made 5K attempted hits on one site, I didn't check the others.

That's the beauty of automated bot blocking, I never knew it existed until you mentioned it. ;)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved