gethan
msg:4384691 | 4:07 am on Nov 8, 2011 (gmt 0) |
More details: around 10 requests a second... like a DOS attack. I emailed abuse@msn.com - but don't expect much if anything...
www.example.org:80 207.46.194.123 - - [08/Nov/2011:04:04:54 +0000] "GET /Example/3458472 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
www.example.org:80 207.46.194.82 - - [08/Nov/2011:04:04:54 +0000] "GET /Example/2010978 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
www.example.org:80 207.46.194.79 - - [08/Nov/2011:04:04:54 +0000] "GET /Example/704586 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
www.example.org:80 207.46.194.125 - - [08/Nov/2011:04:04:54 +0000] "GET /Example/5822984 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
www.example.org:80 207.46.204.128 - - [08/Nov/2011:04:04:55 +0000] "GET /Example/546760 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
www.example.org:80 207.46.194.48 - - [08/Nov/2011:04:04:55 +0000] "GET /Example/5453835 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
www.example.org:80 207.46.195.167 - - [08/Nov/2011:04:04:55 +0000] "GET /Example/1667704 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
www.example.org:80 207.46.194.102 - - [08/Nov/2011:04:04:55 +0000] "GET /Example/3574573 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
www.example.org:80 207.46.194.67 - - [08/Nov/2011:04:04:55 +0000] "GET /Example/4814833 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
www.example.org:80 207.46.194.69 - - [08/Nov/2011:04:04:55 +0000] "GET /Example/1161610 HTTP/1.1" 503 20 "-" "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)"
|
johnh123
msg:4386007 | 6:21 pm on Nov 11, 2011 (gmt 0) |
i've been getting this on and off for the last week - any ideas as to what is going on or what can we do to stop it?
|
bingdude
msg:4386056 | 9:04 pm on Nov 11, 2011 (gmt 0) |
Responded to both via IM. ;)
|
SEOPTI
msg:4386373 | 6:55 am on Nov 13, 2011 (gmt 0) |
gethan:
You are member since 2001, I believe you should have your own server and even under biggest crawler load you will stay at 1% CPU and 1% RAM usage. Or are you with 1000+ other people on a VPS machine which consts $ 3.99/month, if this is the case I believe website creation is your hobby.
I see no problem at all, Bing is hitting my sites with 100k+ bot requests/day.
You should feel lucky and thankful they try to crawl and index your site, send you traffic, send you conversions, people are crying because they don't see bingbot for months.
Sending mails to their abuse department because they crawl you hard is pointless and complete nonsense.
If you don't want their traffic just set your crawl rate in webmaster tools as low as possible or block them in robots.txt.
I love Bing traffic and their awesome traffic conversion levels so I go for the highest possible crawl rate day and night.
If you are completely unable to handle few bingbot requests how do you treat GBot and scraper requests, they are hitting your site 10 times harder.
Look at your code, probably your php/mysql code/queries were poorly written or you have not optimized your DB (indexes ...).
|
|
