Welcome to WebmasterWorld Guest from 54.242.224.250

Forum Moderators: Robert Charlton & aakk9999 & andy langton & goodroi

Message Too Old, No Replies

Googlebot Crawling rates way down in September

     
3:38 am on Sep 5, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 17, 2006
posts: 838
votes: 0


Is this just my sites or somebody else is also seeing this?

Gbot crawling rate started faltering around Aug 28-29th and since Sept. 1st it only goes down on all my sites (as seen via WMT). Some sites are affected more than others but there isn't a single site in the account that has that graph pointing up, and some are down to 20% and less of what was the summer 2012 average.

Traffic hasn't been affected yet but I'm sure it will if the crawling rate keeps dropping like that. Sites are spread across different servers, so it's not a localized network access issue.

Anyone has an opinion on what's going on here?
7:56 am on Sept 5, 2012 (gmt 0)

Junior Member

5+ Year Member

joined:May 17, 2011
posts: 193
votes: 0


I have noticed this too. I have added some new content to my site and this would usually be picked up and cached within the day. It has been several days and nothing.
3:44 am on Sept 6, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 17, 2006
posts: 838
votes: 0


It's getting slower still. Pretty much every site in the account is now at the lowest crawl per day since the visible part of the chart began (June 1st 2012)

What's unnerving about these swings is that there isn't a lot that ties the sites in the account together - different types of site, templates, link profiles, ages, niches etc. Just about the only thing that ties them together is me through this one WMT account. Do they dole out crawl budget per WMT account now?
11:38 pm on Sept 6, 2012 (gmt 0)

Junior Member

joined:Oct 16, 2011
posts:130
votes: 0


Same for me, crawl rate is way down...
2:35 pm on Sept 7, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3169
votes: 0


Confirmed for me too, crawled pages per day is down 50% from pre Aug 30th levels. Something has changed.
2:58 pm on Sept 7, 2012 (gmt 0)

Full Member

joined:Apr 26, 2012
posts:328
votes: 8


Mine went down for a few days, but then shot right back up to where it was at the end of August.
3:37 pm on Sept 7, 2012 (gmt 0)

New User

5+ Year Member

joined:Aug 18, 2011
posts:26
votes: 0


Mine was down to all time lows for the last week - 10% of previous average. However, it just shot back up to normal today. Maybe it is a reporting error?

Glad to hear it is not just me. I also read that Crawl stats includes Google Image bots and coincidentally my Image traffic just tanked (-60%) in the last day so thought the two were related.

More info on Crawl stats from the Goog: [support.google.com...]
3:37 pm on Sept 7, 2012 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:10563
votes: 15


within the small sample set checked - one that was notably on the high side and the others on the "low end" of normal variation but too early to call it a trend imo.
3:47 pm on Sept 7, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 17, 2006
posts: 838
votes: 0


A complete a pretty dramatic reversal just happened last night: all sites have jumped up considerably and several are at the all times high (since the graph began, now it's around June 5th, 2012).

Could it be that Google's resources have been spent on something else these 5 days? Like a Panda or Penguin recalculations? I wonder if something big is coming ...
3:49 pm on Sept 7, 2012 (gmt 0)

Full Member

5+ Year Member

joined:May 31, 2008
posts:264
votes: 0


Mine went down the first couple of days but is back at the August level right now.
4:18 pm on Sept 7, 2012 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:10563
votes: 15


possibly related to these reports from late august?
Big Googlebot Crawling Spike - Update Soon? - Google SEO News and Discussion forum:
http://www.webmasterworld.com/google/4489654.htm [webmasterworld.com]
2:57 pm on Sept 11, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 17, 2006
posts: 838
votes: 0


Well, the rumors about crawling rate drop reversal have been greatly exaggerated. It's not up so much as it has become simply erratic. After several months of rather smooth crawl rate graph, September looks like a mountain range with peaks and valleys repeating every 3-4 days.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members