Welcome to WebmasterWorld Guest from

Forum Moderators: mack

Message Too Old, No Replies

Is this an MSN Search bot?

Strange behaviour

5:38 pm on Nov 16, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 6, 2004
votes: 0

One page on my site was requested by an IP belonging to Microsoft Corporation. The User Agent is very ordinary looking.

On the next row in my log an IP of XO Communications requests the same page. (The time stamp is actually 3 sec. earlier, but I don't know if the time in the log is the time of full delivery or of request - it's a big 250K file.) This time the UA is special: "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ...../1.0 )".

Then 245 more files (html, jpeg, all different) are requested in 30 minutes (when interrupted) with the very same UA, but the IP's are either one belonging to XO, Sprintlink/Microsoft or AT&T. Three different IP's are used and only one or two requests are made with the same IP before it's changed. Alternating between the three IP's goes on for the whole session. My robots.txt is not requested.

This doesn't look like a DoS. Neither Site Grabbing (no need to use three IP's for my tiny site), and if IP's are spoofed grabbing wouldn't work. But why wouldn't an MSN Search bot identify itself properly? The clues point at MS, but they can't have any professional interest in my site.

7:30 pm on Nov 17, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:June 3, 2002
votes: 0


Bad one. I denied the entire MS IP range for any User-agent that does not begin with msnbot.


Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members