homepage Welcome to WebmasterWorld Guest from 54.161.214.221
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Microsoft / Bing Search Engine News
Forum Library, Charter, Moderators: mack

Bing Search Engine News Forum

    
MSNbot Hitting servers hard
Asking about why MSNbot is hitting my server very hard
stinkfoot




msg:1537673
 2:24 pm on Nov 22, 2005 (gmt 0)

Have seen a massive increase in hits from MSNbot from about 30 to 100 a day to what is now 1000+ hits per day. Anyone esle seeing this?

If not anyone else know who I ask to calm it down?

 

jdMorgan




msg:1537674
 2:52 pm on Nov 22, 2005 (gmt 0)

According to the MSN webmaster help page [search.msn.com], you should be able to handle this in robots.txt:

# Limit msnbot to 100 requests per day (86400 seconds)
User-agent: msnbot
Crawl-delay: 864

Jim

Lord Majestic




msg:1537675
 2:56 pm on Nov 22, 2005 (gmt 0)

I am just wondering here - do you get hammered by bot from the same IP, or is it multiple MSN bots coming from different IPs?

stinkfoot




msg:1537676
 12:40 am on Nov 23, 2005 (gmt 0)

Thanks jdMorgan

Not sure about the IP addresses tbh LordM

Zaphod Beeblebrox




msg:1537677
 9:33 am on Nov 24, 2005 (gmt 0)

I have the same thing going on. Could you check the IP numbers and verify that the ones that do have rDNS specified create roughly the same traffic as before, and that the ones without rDNS are responsible for the extra traffic?

I'm suspecting a test-run of a new bot.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Microsoft / Bing Search Engine News
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved