Huge crawl rate spike over the July 4 weekend

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Huge crawl rate spike over the July 4 weekend

Joshmc

4:41 pm on Jul 5, 2011 (gmt 0)

Over the weekend I have seen my crawl rate jump to double what it normally is. The site is a large website so double is whole lot of urls. It is back to what it was around the time panda 2.1 came out. This is true for both sites that I monitor. I was wondering if anyone else was seeing these changes and if this is something that should be good or worrying?

Thanks!

indyank

5:01 pm on Jul 5, 2011 (gmt 0)

yep, there had been a spike but will we get the independence? :)

Joshmc

5:23 pm on Jul 5, 2011 (gmt 0)

haha @indyrank that would really be great...

dickbaker

5:53 pm on Jul 5, 2011 (gmt 0)

I saw crawl numbers that were three to six times what's been "normal" for my site since Panda.

Frankly, after seeing the crawl spike before Panda, this spike scares me.

Joshmc

6:10 pm on Jul 5, 2011 (gmt 0)

@dickbaker, yea that is what was interesting to me about this as well, I am unsure whether to be happy or scared about a spike like this, I have seen spikes that lead to increases and more recently the pre panda spike that lead to a large decrease. Glad to know others are seeing it as well.

indyank

4:09 am on Jul 6, 2011 (gmt 0)

so far this is proving to be a negative crawl for me as the results seem to be out today.

topr8

6:18 am on Jul 6, 2011 (gmt 0)

yes huge increase in crawl from google but also bing for me, i'm assuming they thought it would be a quiet weekend so went mad

Shatner

8:12 am on Jul 6, 2011 (gmt 0)

Saw quite an increase in crawl rate as well. Maybe google figured that was a good time since surf rates would be down for the holiday?

andy_boyd

1:33 pm on Jul 6, 2011 (gmt 0)

Googlebot has been crawling deeply for the past couple of days here too. Have yet to see the changes get cached as yet, but will be interesting to see if anything happens when they do.

Joshmc

4:08 pm on Jul 15, 2011 (gmt 0)

My crawl rate has officially dropped way back down again to around post panda 2 crawl numbers

mhansen

4:13 pm on Jul 15, 2011 (gmt 0)

Same here on the crawl rate... We also noticed a significant loss of local search generated traffic at the same time.

indyank

4:40 pm on Jul 15, 2011 (gmt 0)

July 4 was very significant as not only the crawl rate peaked, all the user metrics looked good for that day.The very next day it was all back to panda nonsense.

And I just have no words to explain the SERPS I see today...