homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

Massive Crawl by Googlebot

Msg#: 4658103 posted 12:18 am on Mar 28, 2014 (gmt 0)

Has anyone else notice a big crawl by google going on, 3 large sites being fully crawled over the last few days

Good job im on a good fast server



WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month

Msg#: 4658103 posted 3:07 am on Mar 28, 2014 (gmt 0)

They have been hitting hard on 3 of mine. One got so bad I put a 1.25 minute crawl delay. It is like they got off the leash. I'm on a good server, but others are trying to see things too, and they are overdoing it.


Msg#: 4658103 posted 4:51 am on Mar 28, 2014 (gmt 0)

Yep. For me it went from the largest crawl I've seen over the past 2 years, down to pretty much the lowest I've seen since my site started!


Further analysis showed me that Googlebot has only crawled 1 (One) page over the past 24 hours..


Msg#: 4658103 posted 7:46 am on Mar 28, 2014 (gmt 0)

I have a feeling their about to run a massive update, see the same last year just before penguin refresh

was just wondering if others were seeing the same


Msg#: 4658103 posted 9:41 am on Mar 28, 2014 (gmt 0)

Just posted in another thread, Looking at WMT, massive crawl spike on 3/22 - 3/26 (last day of reporting), from 650 pages a day to 13,000 pages a day.


WebmasterWorld Senior Member 10+ Year Member

Msg#: 4658103 posted 1:25 pm on Mar 28, 2014 (gmt 0)

If your site is being hit with massive crawling, does that mean that it looks like your site will be affected by any new update they run?

So if your site is not seeing any massive uptick in crawling, does that mean your site is not likely to get affected by this soon-to-come update?

I guess my question is: when you are seeing massive uptick in crawling, is that a good or a bad thing?

[edited by: alika at 1:30 pm (utc) on Mar 28, 2014]


Msg#: 4658103 posted 1:35 pm on Mar 28, 2014 (gmt 0)

Think they need to crawl to find out, I saw this kind of crawl just before penguin refresh in October


Msg#: 4658103 posted 2:12 pm on Mar 28, 2014 (gmt 0)

Based on the history of penguin we are due for an update soon, could be what we are seeing, the precursor.


WebmasterWorld Senior Member Top Contributors Of The Month

Msg#: 4658103 posted 3:23 pm on Mar 28, 2014 (gmt 0)

Googlebot has been in maintenance mode on my sites (nothing even remotely deep since early March).


WebmasterWorld Senior Member 5+ Year Member

Msg#: 4658103 posted 10:36 pm on Mar 28, 2014 (gmt 0)

My largest site has shown a small decrease in crawl rate over the last few days. Two of my smaller sites have shown a very definite large increase in crawl rate over that time. Not the highest but very near.

I have not the slightest clue what use that information is to me or what use I can make of it. Seems like a case of whatever will be, will be.


Msg#: 4658103 posted 11:02 pm on Mar 28, 2014 (gmt 0)

Well after the massive crawl GoogleBot has only crawled 108 pages of my site within the last 24 hours. (BingBot was up around 13,000).

Weirdly though my impressions have gone through the roof, while clicks have stayed the same. Must be a bug -> [i.imgur.com...]

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved