homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Gold Sponsor 2015!
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

Huge Spike in Crawl Rate

Msg#: 4519022 posted 2:15 pm on Nov 14, 2012 (gmt 0)

In looking at the Googlebot pages crawled per day in Webmaster Tools I noticed a recent large spike in activity last week.

Just wondering what this means, good or bad?





WebmasterWorld Senior Member tedster us a WebmasterWorld Top Contributor of All Time 10+ Year Member

Msg#: 4519022 posted 5:34 am on Nov 15, 2012 (gmt 0)

Gieven what we see reported here, most crawl spikes have to do with Google's technical scheduling needs for large groups of sites, rather than being specific for just one. Because of this, I'd be on the lookout for comment from others that they also saw a crawling spike at the same time. The exact days of the crawl spike might help.

If you recently made some changes to your site structure, or to the content of many changes, that might be one reason Google needs a more widespread crawl of your particular domain. However, even a crawl like that might be scheduled along with deep crawls of other sites.


WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Msg#: 4519022 posted 6:26 pm on Nov 15, 2012 (gmt 0)

I see a big spike on most sites at least every 6 to 10 weeks. Very often much of it is Google revisiting pages that previously returned 404 or 301 and which they then scheduled as low crawl priority.

404 pages are checked a couple of times within 48 hours of being discovered as no longer returning 200, then Google doesn't come back to that URL for weeks or months. This means that you need to be VERY quick to fix unexpected 404 errors on your site otherwise the corrections are not seen by Google for a very long time.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved