I'm scared boys and girls, my site was crawled by nearly triple the googlebot average on May 12 and historically big changes(mostly negative) have followed big googlebot crawls so... what's on the radar?
My sense of googlebot's history is different. I often see major crawling spikes (as well as dips) with no correlation to updates or ranking changes. I say "don't borrow trouble" - you can't guess what's coming anyway. However, with over 50 ranking changes in a month, it's safe to say something is definitely coming.
I got a message in my adsense dashboard telling me that I can increase my earnings by an estimated percentage if I opt in to images/rich media. What's new is that the message names the unit on my page using the tracking I provided for that specific unit. Could the big crawl by Googlebot have been for this adsense purpose?
a) Did you make any recent major changes to your site? Sometimes a major change can cause a major recrawl.
b) I saw the spikes twice also, Once was right before Panda 1.0 which affected my site. The second time was right before Penguin 1.0 which caused an increase in traffic. So guess, it could go either way.
a) no, this site is one I try never to make major changes with other than add new content 3-4 times a week.
b) or stay the same? My site seems to be doing this for now. Every other spike before the last one led to rank changes, usually negative but not always. So far no change, this would be the second in a row within 30 days that changed nothing noticeable (the site was affected by penguin/panda so if the crawls were for those reasons it WAS affected but this latest one?).
@Sgt_Kickaxe - I've also seen a huge ongoing spike in the number of kilobytes downloaded by Google beginning roughly one week into May. This is on a site that was negatively impacted by Penguin. You can see my chart here: [imgur.com...]
Something's coming down the track, but who knows what. My money would be on the first iteration of Penguin.
Now, when even my graph shows a dramatic spike around that time, jumping up to almost twice as many hits as the next-highest day on the graph...
:: detour here for closer investigation ::
First discovery: The caption "Pages Crawled" is a brazen lie. It isn't pages; it's all URLs counted together.
Second discovery: If I exclude the imagebot, the remaining spike comes from the vanilla Googlebot doing its pretending-to-be-human thing. The one where it picks up images, giving the page as referer. And since it chose to concentrate on pages that each include a fistful of small images, the total number of Googlebot hits can jump pretty dramatically.
Wouldn't it be nice if for once an algo change resulted in a positive move? From what I've personally experienced (since the first nasty update in May 2010) is that we get a huge negative result followed by a very, VERY slow positive recovery. Don't get me wrong, a positive recovery is always good, but 8 weeks of that destroys cash flow and any chance of (paying a designer for instance) improving the site.
I will admit that in my niche, the Penguin update did remove ONE obviously spammy, over SEO'd site. However, it replaced all the good sites with thin ad ridden e how type sites. It's like they went 2 steps forward and 12 backwards.