homepage Welcome to WebmasterWorld Guest from 54.196.207.55
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Rampant of Redmond
Shivering my Timbers
Angonasec




msg:4635862
 1:51 am on Jan 8, 2014 (gmt 0)

Be warned, it's cold in WA:

It hit one popular page dozens of times, non-stop, peaking at 12 hits +per second+

No robots.txt, no favicon, no js, nor images, just this frenziedly packing today's access log;


65.52.246.161 - - [07/Jan/2014] "GET /example.htm HTTP/1.1" 200 10522 "-" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)"

Appears to be genuine MS:
Microsoft Hosting
65.52.0.0 - 65.55.255.255 65.52.0.0/14

We're currently directing it to the rubber-room.

 

keyplyr




msg:4636307
 3:45 am on Jan 10, 2014 (gmt 0)



That covert M$ UA has been active for years. I safely block it.

Angonasec




msg:4636317
 5:06 am on Jan 10, 2014 (gmt 0)

KeyP: Thanks, first time it's had a fit on our site. Do you block it on UA or IP range? The UA posted above appears to be legit except for the 64bit WOW64; missing...

lucy24




msg:4636324
 6:15 am on Jan 10, 2014 (gmt 0)

Page only? No supporting files? css and js always, midi and piwik.php if it can get them; images only from 199.30.

Not the ordinary plainclothes bingbot, then.

Angonasec




msg:4636332
 7:02 am on Jan 10, 2014 (gmt 0)

Hence the new thread :)

keyplyr




msg:4636347
 8:16 am on Jan 10, 2014 (gmt 0)

I do it with a couple filters: if it has that UA *and* comes from that IP *and* asks for... You get the picture.

Angonasec




msg:4636440
 12:52 pm on Jan 10, 2014 (gmt 0)

Yes, there's a few legitimate bing/msn bots using that IP range/CIDR, but "usually" with a kosher UA, so I'm just directing the specific convulsing IP to a dark-green room.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved