Has anyone else notice a big crawl by google going on, 3 large sites being fully crawled over the last few days
Good job im on a good fast server
not2easy
3:07 am on Mar 28, 2014 (gmt 0)
They have been hitting hard on 3 of mine. One got so bad I put a 1.25 minute crawl delay. It is like they got off the leash. I'm on a good server, but others are trying to see things too, and they are overdoing it.
ibeau
4:51 am on Mar 28, 2014 (gmt 0)
Yep. For me it went from the largest crawl I've seen over the past 2 years, down to pretty much the lowest I've seen since my site started!
Further analysis showed me that Googlebot has only crawled 1 (One) page over the past 24 hours..
kellyman
7:46 am on Mar 28, 2014 (gmt 0)
I have a feeling their about to run a massive update, see the same last year just before penguin refresh
was just wondering if others were seeing the same
Shepherd
9:41 am on Mar 28, 2014 (gmt 0)
Just posted in another thread, Looking at WMT, massive crawl spike on 3/22 - 3/26 (last day of reporting), from 650 pages a day to 13,000 pages a day.
alika
1:25 pm on Mar 28, 2014 (gmt 0)
If your site is being hit with massive crawling, does that mean that it looks like your site will be affected by any new update they run?
So if your site is not seeing any massive uptick in crawling, does that mean your site is not likely to get affected by this soon-to-come update?
I guess my question is: when you are seeing massive uptick in crawling, is that a good or a bad thing?
[edited by: alika at 1:30 pm (utc) on Mar 28, 2014]
kellyman
1:35 pm on Mar 28, 2014 (gmt 0)
Think they need to crawl to find out, I saw this kind of crawl just before penguin refresh in October
Shepherd
2:12 pm on Mar 28, 2014 (gmt 0)
Based on the history of penguin we are due for an update soon, could be what we are seeing, the precursor.
EditorialGuy
3:23 pm on Mar 28, 2014 (gmt 0)
Googlebot has been in maintenance mode on my sites (nothing even remotely deep since early March).
nomis5
10:36 pm on Mar 28, 2014 (gmt 0)
My largest site has shown a small decrease in crawl rate over the last few days. Two of my smaller sites have shown a very definite large increase in crawl rate over that time. Not the highest but very near.
I have not the slightest clue what use that information is to me or what use I can make of it. Seems like a case of whatever will be, will be.
ibeau
11:02 pm on Mar 28, 2014 (gmt 0)
Well after the massive crawl GoogleBot has only crawled 108 pages of my site within the last 24 hours. (BingBot was up around 13,000).
Weirdly though my impressions have gone through the roof, while clicks have stayed the same. Must be a bug -> [i.imgur.com...]
