Welcome to WebmasterWorld Guest from 54.166.122.69

Forum Moderators: mack

Is this an MSN Search bot?

Strange behaviour

   
5:38 pm on Nov 16, 2004 (gmt 0)

10+ Year Member



One page on my site was requested by an IP belonging to Microsoft Corporation. The User Agent is very ordinary looking.

On the next row in my log an IP of XO Communications requests the same page. (The time stamp is actually 3 sec. earlier, but I don't know if the time in the log is the time of full delivery or of request - it's a big 250K file.) This time the UA is special: "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ...../1.0 )".

Then 245 more files (html, jpeg, all different) are requested in 30 minutes (when interrupted) with the very same UA, but the IP's are either one belonging to XO, Sprintlink/Microsoft or AT&T. Three different IP's are used and only one or two requests are made with the same IP before it's changed. Alternating between the three IP's goes on for the whole session. My robots.txt is not requested.

This doesn't look like a DoS. Neither Site Grabbing (no need to use three IP's for my tiny site), and if IP's are spoofed grabbing wouldn't work. But why wouldn't an MSN Search bot identify itself properly? The clues point at MS, but they can't have any professional interest in my site.

7:30 pm on Nov 17, 2004 (gmt 0)

10+ Year Member



[webmasterworld.com...]
[webmasterworld.com...]

Bad one. I denied the entire MS IP range for any User-agent that does not begin with msnbot.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month