Over the past three days bing's crawler is giving Googlebot a run for it's money on my forum in terms of crawl rate. I triple checked to make sure the IP and user agent was legit and it is.
Any idea if Bing just got caffeinated too?
7:26 pm on Jan 27, 2013 (gmt 0)
I've noticed this for a month or two now, although it might have instensified still more: haven't checked.
My guess is that bing is trying to improve its index - it was always behind the unmentionable SE as far as page count went. Now it has a chance of catching up a bit I wonder if it's not simply trying to grap more pages than it used to.
10:26 pm on Jan 27, 2013 (gmt 0)
I noticed a huge increase in one old site that is not updated often, bing had a party in there! I was checking the access log for those two weeks after New Year and the whole, plain .txt file is 8.9 Mb and of that, 8.1Mb is only the bingbot. Does not include BingPreview which is currently blocked, that was only 10kb. Does not include any msnbots. To put that in perspective, only 347kb represents gbot's share of the total. That is pretty abusive IMHO.
Shell Of A Man
1:29 am on Jan 31, 2013 (gmt 0)
The last I had noticed a rise from Bing was back in October 2012 but that was only a few thousand extra visits a month. Looking at it recently it looks like it has stayed pretty steady, but has not grown since.
5:14 am on Jan 31, 2013 (gmt 0)
Funny this should come up. I recently did some preliminary bot-checking in preparation for this year's installment of At Home With The Robots, and was staggered to find at least four times as many bings as googles. And that's, ahem, after filtering out the requests for robots.txt.