Page is a not externally linkable
- Microsoft
-- Bing Search Engine News
---- MSNBot killing my server, not this again!


TheMadScientist - 11:14 pm on Feb 25, 2010 (gmt 0)


Could you elaborate on what you type of site you're running (EG News, blog, informational, static or dynamic, etc.) and what else you're doing to try to slow the bot down besides blocking it? EG Serving last modified headers, serving e-tag headers, expires, etc.

I ask because I work on a couple of decent sized sites which are both dynamic, but behave as if they are static (serve full headers, including different expiration times by file type, etc.) and haven't ever had an issue with MSNBot at all. It usually requests everything twice in a row, but the issue you are talking about is definitely not an 'everyone issue' so knowing the differences in situations would be good, IMO, and might help figure out what's causing the issue for you and not everyone...


Thread source:: http://www.webmasterworld.com/msn_microsoft_search/4087251.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com