homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Microsoft / Bing Search Engine News
Forum Library, Charter, Moderators: mack

Bing Search Engine News Forum

Is this an MSN Search bot?
Strange behaviour

10+ Year Member

Msg#: 241 posted 5:38 pm on Nov 16, 2004 (gmt 0)

One page on my site was requested by an IP belonging to Microsoft Corporation. The User Agent is very ordinary looking.

On the next row in my log an IP of XO Communications requests the same page. (The time stamp is actually 3 sec. earlier, but I don't know if the time in the log is the time of full delivery or of request - it's a big 250K file.) This time the UA is special: "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ...../1.0 )".

Then 245 more files (html, jpeg, all different) are requested in 30 minutes (when interrupted) with the very same UA, but the IP's are either one belonging to XO, Sprintlink/Microsoft or AT&T. Three different IP's are used and only one or two requests are made with the same IP before it's changed. Alternating between the three IP's goes on for the whole session. My robots.txt is not requested.

This doesn't look like a DoS. Neither Site Grabbing (no need to use three IP's for my tiny site), and if IP's are spoofed grabbing wouldn't work. But why wouldn't an MSN Search bot identify itself properly? The clues point at MS, but they can't have any professional interest in my site.



10+ Year Member

Msg#: 241 posted 7:30 pm on Nov 17, 2004 (gmt 0)


Bad one. I denied the entire MS IP range for any User-agent that does not begin with msnbot.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Microsoft / Bing Search Engine News
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved