joined:Jan 8, 2008
In the past couple of months I've seen an exponential increase in crawling from Bing - so much so that it been impacting server performance and taking sites down from time to time, definitely impacting response times for users and bots, both Bing and Google.
They aren't identifying themselves properly all of the time, either via reverse dns or user agent - however, the IP is either owned by Microsoft or originates in Redmond, Washington. Given that earlier this week they crawled over 9.7 million pages from a single site in one day it definitely appears to be Bing.
My question is - if Bing's crawling has been taking my sites down, could this be a possible reason for traffic drops from Google? Starting yesterday I've seen a 10-20% decrease in tag page traffic across 20+ sites, all in different countries, all at the same time. A few potential reasons come to mind - Panda (20+ markets and only affecting a single page type...unlikely), Google downgrade due to bad response times (again, single page type and all at the same time), or general Google algorithm change targeting 'tag' type pages..... Any thoughts would be appreciated...