Welcome to WebmasterWorld Guest from 54.163.142.67

Forum Moderators: Robert Charlton & aakk9999 & andy langton & goodroi

Message Too Old, No Replies

Google finally showing active spidering

     
5:35 pm on Jun 24, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 26, 2006
posts:1619
votes: 0


After weeks of very little spidering it appears that Google finally got hungry and is eating up pages like mad today. Showing over 6000 pages hit today.

I'm hoping that this is going to be a good thing, since traffic from Google has been off for some time now.
6:28 pm on June 24, 2010 (gmt 0)

New User

5+ Year Member

joined:May 16, 2009
posts:30
votes: 0


Yeap, I see that too!

We'll see if it has any effect on the trafic.
6:39 pm on June 24, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 3, 2002
posts:894
votes: 0


Same here...
6:43 pm on June 24, 2010 (gmt 0)

Preferred Member

5+ Year Member

joined:June 14, 2010
posts:585
votes: 0


Same here - I also noticed some newer cache dates in the serps.
12:01 am on June 25, 2010 (gmt 0)

New User

5+ Year Member

joined:June 22, 2010
posts:29
votes: 0


same here, basically a 100% increase in crawling in the last few days. However, the SERPs are actually showing outdated info, for example I changed a page title on Monday, by Tuesday the new title was in the SERPs and then Wednesday night it's back to the old title. Doesn't seem to matter which datacenter. Odd, odd odd.
12:13 am on June 25, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


Welcome to the forums, Arronsky. Odd, odd, odd indeed. It looks like Google may have Caffeine but the new engine needs a serious tune-up before it's purring like a kitten (we can only hope).

I thought - in fact I think everyone thought - that Caffeine meant fresher crawling that could be tagged with Google's own meta information much more quickly and get into the SERPs really fast. It does seem to be happening in some areas, but others seem to be suffering, as you've noted.

Maybe this new up tick in crawling will mark the beginning of a more widespread improvement.
12:44 am on June 25, 2010 (gmt 0)

New User

5+ Year Member

joined:June 22, 2010
posts:29
votes: 0


Tedster, my guess as a CS guy for this very odd behavior, in context of what they publicly have stated is their goal, is that they are running into data import problems and have to roll back certain crawl information to 'known-good' sets. This may be accidental, or it may be purposeful. If I was them, I'd use long tail and mid-tail to test the heck out of the system, while ensuring that head of tail results were as pristine as they could be, and flexed Caffeine's muscle. When I search for Obama, I expect crystal clear results that do all of what Caffeine promise. When I search for blue widget x in location y, my pain tolerance and expectations is much higher and lower, respectively, making it a great place to 'play' with real world data and loads.

Anyways, we're all guessing.
2:59 pm on June 25, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


Your educated guess sounds right to me. There are some other reports here that also sound like data import problems - such as missing domain roots for obviously trusted sites.