Welcome to WebmasterWorld Guest from 54.147.63.124

Message Too Old, No Replies

Huge crawl rate spike over the July 4 weekend

     
4:41 pm on Jul 5, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Oct 25, 2010
posts:81
votes: 0


Over the weekend I have seen my crawl rate jump to double what it normally is. The site is a large website so double is whole lot of urls. It is back to what it was around the time panda 2.1 came out. This is true for both sites that I monitor. I was wondering if anyone else was seeing these changes and if this is something that should be good or worrying?

Thanks!
5:01 pm on July 5, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Mar 9, 2010
posts:1806
votes: 9


yep, there had been a spike but will we get the independence? :)
5:23 pm on July 5, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Oct 25, 2010
posts:81
votes: 0


haha @indyrank that would really be great...
5:53 pm on July 5, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 14, 2003
posts:1782
votes: 0


I saw crawl numbers that were three to six times what's been "normal" for my site since Panda.

Frankly, after seeing the crawl spike before Panda, this spike scares me.
6:10 pm on July 5, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Oct 25, 2010
posts:81
votes: 0


@dickbaker, yea that is what was interesting to me about this as well, I am unsure whether to be happy or scared about a spike like this, I have seen spikes that lead to increases and more recently the pre panda spike that lead to a large decrease. Glad to know others are seeing it as well.
4:09 am on July 6, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Mar 9, 2010
posts:1806
votes: 9


so far this is proving to be a negative crawl for me as the results seem to be out today.
6:18 am on July 6, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 19, 2002
posts:3171
votes: 8


yes huge increase in crawl from google but also bing for me, i'm assuming they thought it would be a quiet weekend so went mad
8:12 am on July 6, 2011 (gmt 0)

Preferred Member

joined:Mar 20, 2011
posts:544
votes: 0


Saw quite an increase in crawl rate as well. Maybe google figured that was a good time since surf rates would be down for the holiday?
1:33 pm on July 6, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 1, 2003
posts:490
votes: 0


Googlebot has been crawling deeply for the past couple of days here too. Have yet to see the changes get cached as yet, but will be interesting to see if anything happens when they do.
4:08 pm on July 15, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Oct 25, 2010
posts:81
votes: 0


My crawl rate has officially dropped way back down again to around post panda 2 crawl numbers
4:13 pm on July 15, 2011 (gmt 0)

Preferred Member

5+ Year Member

joined:June 14, 2010
posts: 585
votes: 0


Same here on the crawl rate... We also noticed a significant loss of local search generated traffic at the same time.
4:40 pm on July 15, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Mar 9, 2010
posts:1806
votes: 9


July 4 was very significant as not only the crawl rate peaked, all the user metrics looked good for that day.The very next day it was all back to panda nonsense.

And I just have no words to explain the SERPS I see today...