Googlebot Crawling rates way down in September

[1script]

Is this just my sites or somebody else is also seeing this?

Gbot crawling rate started faltering around Aug 28-29th and since Sept. 1st it only goes down on all my sites (as seen via WMT). Some sites are affected more than others but there isn't a single site in the account that has that graph pointing up, and some are down to 20% and less of what was the summer 2012 average.

Traffic hasn't been affected yet but I'm sure it will if the crawling rate keeps dropping like that. Sites are spread across different servers, so it's not a localized network access issue.

Anyone has an opinion on what's going on here?

[n00b1]

I have noticed this too. I have added some new content to my site and this would usually be picked up and cached within the day. It has been several days and nothing.

[1script]

It's getting slower still. Pretty much every site in the account is now at the lowest crawl per day since the visible part of the chart began (June 1st 2012)

What's unnerving about these swings is that there isn't a lot that ties the sites in the account together - different types of site, templates, link profiles, ages, niches etc. Just about the only thing that ties them together is me through this one WMT account. Do they dole out crawl budget per WMT account now?

[menntarra 34]

Same for me, crawl rate is way down...

[Sgt_Kickaxe]

Confirmed for me too, crawled pages per day is down 50% from pre Aug 30th levels. Something has changed.

[Dymero]

Mine went down for a few days, but then shot right back up to where it was at the end of August.

[serpsup]

Mine was down to all time lows for the last week - 10% of previous average. However, it just shot back up to normal today. Maybe it is a reporting error?

Glad to hear it is not just me. I also read that Crawl stats includes Google Image bots and coincidentally my Image traffic just tanked (-60%) in the last day so thought the two were related.

More info on Crawl stats from the Goog: [support.google.com...]

[phranque]

within the small sample set checked - one that was notably on the high side and the others on the "low end" of normal variation but too early to call it a trend imo.

[1script]

A complete a pretty dramatic reversal just happened last night: all sites have jumped up considerably and several are at the all times high (since the graph began, now it's around June 5th, 2012).

Could it be that Google's resources have been spent on something else these 5 days? Like a Panda or Penguin recalculations? I wonder if something big is coming ...

[johnmoose]

Mine went down the first couple of days but is back at the August level right now.

[phranque]

possibly related to these reports from late august?
Big Googlebot Crawling Spike - Update Soon? - Google SEO News and Discussion forum:
http://www.webmasterworld.com/google/4489654.htm [webmasterworld.com]

[1script]

Well, the rumors about crawling rate drop reversal have been greatly exaggerated. It's not up so much as it has become simply erratic. After several months of rather smooth crawl rate graph, September looks like a mountain range with peaks and valleys repeating every 3-4 days.