Forum Moderators: mack

Message Too Old, No Replies

MSN Bot 65.54.188.29 Crashing Server

How can we stop it?

         

financialhost

9:53 am on Oct 28, 2005 (gmt 0)

10+ Year Member



Our site is getting hammered by MSN bot 65.54.188.29, it is passing big company id's instead of 10 digit id which is causing a lot of errors on our server & performance. Can anyone advise on a fix, perhaps we need to place something in the robots.txt?

Some of our logs are below,

09:54:16",,"ODBC Error Code = 22003 (Numeric value out of range)
[Microsoft][ODBC SQL Server Driver][SQL Server]The number '2704761012300045935026544063089455660250879685212 06553584895358851535885953588645358867535887453589 0653625205362523' is out of the range for numeric representation (maximum precision 38).

Error 1160 10/21/05 09:54:16",,"65.54.188.29, msnbot/1.0 (+http://search.msn.com/msnbot.htm), ODBC Error Code = 22003 (Numeric value out of range)
[Microsoft][ODBC SQL Server Driver][SQL Server]The number '2704761012300045935026544063089455660250879685212 06553584895358851535885953588645358867535887453589 0653625205362523' is out of the range for numeric representation (maximum precision 38).

Date/Time: 10/21/05 09:54:16
Browser: msnbot/1.0 (+http://search.msn.com/msnbot.htm)
Remote Address: 65.54.188.29
Query String: Companyids=270476101230004593502654406308945566025 08796852120655358489535885153588595358864535886753 58874535890653625205362523

Dijkgraaf

8:49 pm on Nov 9, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I take it is some sort of dynamic page that is using a query string?
A quick solution is to ban MSN from requesting that page altogether in robots.txt.

For a longer term soltion the thing to investigate is where is MSN getting the URL's with those query strings that are invalid?

Another way of fixing the issue is to have some validation in the page before it passes it on to the database, if it fails validation get it to issue a 400 code "Bad Request"

Lord Majestic

8:50 pm on Nov 9, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Check length of the id in the script itself and quickly issue error (404?) if id looks incorrect (ie over 10 digits). This way your database won't be touched and MSN will gain knowledge that page does not exist.