homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Microsoft / Bing Search Engine News
Forum Library, Charter, Moderators: mack

Bing Search Engine News Forum

Stop DOS attack from MSN bot

 3:52 am on Dec 8, 2012 (gmt 0)

Recently MSN bot keeps crawling my site almost all day long and sends the loading to the roof. It is really a DOS attack. I googled it and found many people have the same problem. How can I stop this from happening? Using robots.txt? I also heard the MSN bot does not follow robots.txt and heaves really bad.

BTW, whenever i see the word bing, I want to vomit.



 3:46 am on Dec 15, 2012 (gmt 0)

Did you determine it was MSNbot by the User Agent or IP?


 11:22 pm on Dec 15, 2012 (gmt 0)

Newbies - I suggest for a better over-view you read the Search Engine Spider and User Agent Identification forum - lots of recent stuff there about MSN/bing.

From my own viewpoint I see no more over-browsing than from googlebot.

If it's not a bot IP (check rDNS to see that) AND it's not a bot UA (there is only one I accept but there are two or three valid ones) then you should be rejecting access.


 1:49 am on Dec 16, 2012 (gmt 0)

msnbot (and bingbot) will obey the crawl-delay directive from the robots.txt file. This will slow it down and prevent the DOS. It has worked wonders for me.


 9:26 pm on Apr 20, 2013 (gmt 0)

Did you determine it was MSNbot by the User Agent or IP?

I've been seeing the same thing lately and it's most certainly MSNbot and Bingbot from both IP addresses (do they really need so many?) and the UA. The also send a no-cache header which really messes up my squid front end.


 4:36 pm on Sep 4, 2013 (gmt 0)

We observe 1000+ parallel connections by msnbot. It's insane and unacceptable.

And yes, it really is msnbot based on ip


 6:15 pm on Sep 4, 2013 (gmt 0)

Free load testing, aren't they nice!

Global Options:
 top home search open messages active posts  

Home / Forums Index / Microsoft / Bing Search Engine News
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved