Welcome to WebmasterWorld Guest from 3.93.75.30

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Big spike in pages crawled May 12th, impending doom?

     
1:09 pm on May 14, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3173
votes: 0


I'm scared boys and girls, my site was crawled by nearly triple the googlebot average on May 12 and historically big changes(mostly negative) have followed big googlebot crawls so... what's on the radar?
5:35 pm on May 14, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


My sense of googlebot's history is different. I often see major crawling spikes (as well as dips) with no correlation to updates or ranking changes. I say "don't borrow trouble" - you can't guess what's coming anyway. However, with over 50 ranking changes in a month, it's safe to say something is definitely coming.
11:31 am on May 15, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3173
votes: 0


A new surprise arrived from Adsense.

I got a message in my adsense dashboard telling me that I can increase my earnings by an estimated percentage if I opt in to images/rich media. What's new is that the message names the unit on my page using the tracking I provided for that specific unit. Could the big crawl by Googlebot have been for this adsense purpose?
1:51 pm on May 15, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member planet13 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:June 16, 2010
posts: 3828
votes: 31


FWIW: Recently I was over-crawled by google, and my rankings tanked.
3:02 pm on May 15, 2012 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 7, 2003
posts:358
votes: 0


I've seen two scenarios..

a) Did you make any recent major changes to your site? Sometimes a major change can cause a major recrawl.

b) I saw the spikes twice also, Once was right before Panda 1.0 which affected my site. The second time was right before Penguin 1.0 which caused an increase in traffic. So guess, it could go either way.
6:39 pm on May 15, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3173
votes: 0


a) no, this site is one I try never to make major changes with other than add new content 3-4 times a week.

b) or stay the same? My site seems to be doing this for now. Every other spike before the last one led to rank changes, usually negative but not always. So far no change, this would be the second in a row within 30 days that changed nothing noticeable (the site was affected by penguin/panda so if the crawls were for those reasons it WAS affected but this latest one?).
9:01 am on May 20, 2012 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 1, 2003
posts:490
votes: 0


@Sgt_Kickaxe - I've also seen a huge ongoing spike in the number of kilobytes downloaded by Google beginning roughly one week into May. This is on a site that was negatively impacted by Penguin. You can see my chart here: [imgur.com...]

Something's coming down the track, but who knows what. My money would be on the first iteration of Penguin.
4:52 pm on May 20, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


On a site with little or no change in content in the last year, Google usually pulls about 20 to 40 pages per day.

In mid-April Google pulled more than 3500 pages in one day, just that one day, then went back to the previous pattern.

No change resulted from that.
10:16 pm on May 20, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15810
votes: 848


Warming up for the Real Hits to come?

Now, when even my graph shows a dramatic spike around that time, jumping up to almost twice as many hits as the next-highest day on the graph...

:: detour here for closer investigation ::

First discovery: The caption "Pages Crawled" is a brazen lie. It isn't pages; it's all URLs counted together.

Second discovery: If I exclude the imagebot, the remaining spike comes from the vanilla Googlebot doing its pretending-to-be-human thing. The one where it picks up images, giving the page as referer. And since it chose to concentrate on pages that each include a fistful of small images, the total number of Googlebot hits can jump pretty dramatically.
11:57 pm on May 20, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3173
votes: 0


No change so far.
6:34 am on May 21, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Mar 4, 2010
posts:1302
votes: 0


Wouldn't it be nice if for once an algo change resulted in a positive move? From what I've personally experienced (since the first nasty update in May 2010) is that we get a huge negative result followed by a very, VERY slow positive recovery. Don't get me wrong, a positive recovery is always good, but 8 weeks of that destroys cash flow and any chance of (paying a designer for instance) improving the site.

I will admit that in my niche, the Penguin update did remove ONE obviously spammy, over SEO'd site. However, it replaced all the good sites with thin ad ridden e how type sites. It's like they went 2 steps forward and 12 backwards.
7:18 am on May 21, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15810
votes: 848


Wouldn't it be nice if for once an algo change resulted in a positive move?

Well, it's got to be positive for somebody. Basic mathematical principle. They must all be on a different forum.

Hmmm.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members