homepage Welcome to WebmasterWorld Guest from 54.196.168.78
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Massive Crawl by Googlebot
kellyman




msg:4658105
 12:18 am on Mar 28, 2014 (gmt 0)

Has anyone else notice a big crawl by google going on, 3 large sites being fully crawled over the last few days

Good job im on a good fast server

 

not2easy




msg:4658122
 3:07 am on Mar 28, 2014 (gmt 0)

They have been hitting hard on 3 of mine. One got so bad I put a 1.25 minute crawl delay. It is like they got off the leash. I'm on a good server, but others are trying to see things too, and they are overdoing it.

ibeau




msg:4658135
 4:51 am on Mar 28, 2014 (gmt 0)

Yep. For me it went from the largest crawl I've seen over the past 2 years, down to pretty much the lowest I've seen since my site started!

[i.imgur.com...]

Further analysis showed me that Googlebot has only crawled 1 (One) page over the past 24 hours..

kellyman




msg:4658174
 7:46 am on Mar 28, 2014 (gmt 0)

I have a feeling their about to run a massive update, see the same last year just before penguin refresh

was just wondering if others were seeing the same

Shepherd




msg:4658192
 9:41 am on Mar 28, 2014 (gmt 0)

Just posted in another thread, Looking at WMT, massive crawl spike on 3/22 - 3/26 (last day of reporting), from 650 pages a day to 13,000 pages a day.

alika




msg:4658252
 1:25 pm on Mar 28, 2014 (gmt 0)

If your site is being hit with massive crawling, does that mean that it looks like your site will be affected by any new update they run?

So if your site is not seeing any massive uptick in crawling, does that mean your site is not likely to get affected by this soon-to-come update?

I guess my question is: when you are seeing massive uptick in crawling, is that a good or a bad thing?

[edited by: alika at 1:30 pm (utc) on Mar 28, 2014]

kellyman




msg:4658255
 1:35 pm on Mar 28, 2014 (gmt 0)

Think they need to crawl to find out, I saw this kind of crawl just before penguin refresh in October

Shepherd




msg:4658271
 2:12 pm on Mar 28, 2014 (gmt 0)

Based on the history of penguin we are due for an update soon, could be what we are seeing, the precursor.

EditorialGuy




msg:4658276
 3:23 pm on Mar 28, 2014 (gmt 0)

Googlebot has been in maintenance mode on my sites (nothing even remotely deep since early March).

nomis5




msg:4658339
 10:36 pm on Mar 28, 2014 (gmt 0)

My largest site has shown a small decrease in crawl rate over the last few days. Two of my smaller sites have shown a very definite large increase in crawl rate over that time. Not the highest but very near.

I have not the slightest clue what use that information is to me or what use I can make of it. Seems like a case of whatever will be, will be.

ibeau




msg:4658341
 11:02 pm on Mar 28, 2014 (gmt 0)

Well after the massive crawl GoogleBot has only crawled 108 pages of my site within the last 24 hours. (BingBot was up around 13,000).

Weirdly though my impressions have gone through the roof, while clicks have stayed the same. Must be a bug -> [i.imgur.com...]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved