Welcome to WebmasterWorld Guest from 54.224.108.189

Forum Moderators: mack

Message Too Old, No Replies

MSNbot Hitting servers hard

Asking about why MSNbot is hitting my server very hard

     
2:24 pm on Nov 22, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Oct 6, 2003
posts:234
votes: 0


Have seen a massive increase in hits from MSNbot from about 30 to 100 a day to what is now 1000+ hits per day. Anyone esle seeing this?

If not anyone else know who I ask to calm it down?

2:52 pm on Nov 22, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


According to the MSN webmaster help page [search.msn.com], you should be able to handle this in robots.txt:

# Limit msnbot to 100 requests per day (86400 seconds)
User-agent: msnbot
Crawl-delay: 864

Jim
2:56 pm on Nov 22, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 8, 2004
posts:1679
votes: 0


I am just wondering here - do you get hammered by bot from the same IP, or is it multiple MSN bots coming from different IPs?
12:40 am on Nov 23, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Oct 6, 2003
posts:234
votes: 0


Thanks jdMorgan

Not sure about the IP addresses tbh LordM

9:33 am on Nov 24, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 31, 2003
posts:196
votes: 0


I have the same thing going on. Could you check the IP numbers and verify that the ones that do have rDNS specified create roughly the same traffic as before, and that the ones without rDNS are responsible for the extra traffic?

I'm suspecting a test-run of a new bot.