Google finally showing active spidering

[Bewenched]

After weeks of very little spidering it appears that Google finally got hungry and is eating up pages like mad today. Showing over 6000 pages hit today.

I'm hoping that this is going to be a good thing, since traffic from Google has been off for some time now.

[mercedesP]

Yeap, I see that too!

We'll see if it has any effect on the trafic.

[webdude]

Same here...

[mhansen]

Same here - I also noticed some newer cache dates in the serps.

[arronsky]

same here, basically a 100% increase in crawling in the last few days. However, the SERPs are actually showing outdated info, for example I changed a page title on Monday, by Tuesday the new title was in the SERPs and then Wednesday night it's back to the old title. Doesn't seem to matter which datacenter. Odd, odd odd.

[tedster]

Welcome to the forums, Arronsky. Odd, odd, odd indeed. It looks like Google may have Caffeine but the new engine needs a serious tune-up before it's purring like a kitten (we can only hope).

I thought - in fact I think everyone thought - that Caffeine meant fresher crawling that could be tagged with Google's own meta information much more quickly and get into the SERPs really fast. It does seem to be happening in some areas, but others seem to be suffering, as you've noted.

Maybe this new up tick in crawling will mark the beginning of a more widespread improvement.

[arronsky]

Tedster, my guess as a CS guy for this very odd behavior, in context of what they publicly have stated is their goal, is that they are running into data import problems and have to roll back certain crawl information to 'known-good' sets. This may be accidental, or it may be purposeful. If I was them, I'd use long tail and mid-tail to test the heck out of the system, while ensuring that head of tail results were as pristine as they could be, and flexed Caffeine's muscle. When I search for Obama, I expect crystal clear results that do all of what Caffeine promise. When I search for blue widget x in location y, my pain tolerance and expectations is much higher and lower, respectively, making it a great place to 'play' with real world data and loads.

Anyways, we're all guessing.

[tedster]

Your educated guess sounds right to me. There are some other reports here that also sound like data import problems - such as missing domain roots for obviously trusted sites.