Google Updates and SERP Changes - February 2010

[mikeclover]

< continued from [webmasterworld.com...] >

I have noticed a big increase in my Blog Traffic, but my keyword traffic is down. I am going to assume that Caffeine has not hit all data servers yet ? I would assume this because my contacts in Chicago, Iowa all say my keyword rankings are -30. My assoicates on the west coast say I am on the first page. Here in Texas I am on the first page. The wierd thing is my traffic is not really down. hmmmmmm

[edited by: tedster at 5:47 pm (utc) on Feb. 1, 2010]

[Shaddows]

He could have said "Personally I can't imagine the Quality team needing yet more time to pull its socks up, so 'after the holidays' seems reasonable to me"

MC, even if you think he does not deliberately mislead, is neither omniscient nor omnipotent. Having an opinion about a reasonable launch date does not mean it cannot change. Clearly, it is not ready for release. Clearly this was not an expected outcome. Equally clearly, the date has now been revised.

I'm not sure about you lot, but I personally have had projects overrun. If something is "due" in a month, then its not complete now. And if testing throws up bugs within that month, the date moves or you release it buggy.

Google normally releases buggy, so there must be major problems with Caffeine. Maybe it puts too many 'money' terms at the top of Organics?

[whitenight]

clearly, it is not ready for release. Clearly this was not an expected outcome. Equally clearly, the date has now been revised....

Google normally releases buggy, so there must be major problems with Caffeine.

Yes, and as Shaddows has already explained the general gist of the problem,
i would suggest MANY people go back and read his earlier post.


Too lazy? I'll repeat in my less "nice" manner

Gorg has TRIED to roll out Caff (both datasets and infrastructure) SEVERAL times. I've documented this in the past for those who care.

It did NOT have compatibility with the latest Big Daddy coding. In fact, it continues to be a massive failure.

Right now, Gorg has some older version, hybrid NONSENSE (ie. Florida/Big Daddy/pre-caff stop gap) in place for SERPs AND infrastruture ... while i imagine, coders are busily writing hundreds of thousands lines of new code in back rooms under punishment by whipping....

As many here used to say, when SERPs changes didn't go their way:
"Google is broken!"*...
except THIS time, Gorg actually is.

This isn't some "radical" theory, either.
It's quite logical, even if one does NOT pay attention to SERPs and infrastructure changes most of the day.

_{*broken - in the sense that the current SERPs and infrastructure is neither what Gorg wants or what users want}

[edited by: whitenight at 10:14 am (utc) on Feb 26, 2010]

[RishiRich]

Shaddows you are completely right. But the frustrating part is "Why let the Crawling suffer ?" The recent SEO efforts have all gone in vain because there has been very little crawl activity.

What if someone copies the content before G crawls it? Will G have an idea about "Who is is the original author of the document?"

Google has been fast to crawl new pages but any changes made to the OLD pages does not seem to be drawing the attention of the Bots.

I would have never put the new content online if Matt would not have said "After The Holidays" and if i would have been given some idea about "Crawling taking the Hit"

[Markoi]

I'm seeing a bug back into the serps which i reported 3/4 year back. This bug was fixed then, and description related. My guess were back a couple of months.

[RishiRich]

I'm seeing a bug back into the serps which i reported 3/4 year back.

Please can you elaborate on the bug....

[Markoi]

It was a IE bug. When a description was made like this: keyword1,keyword2,keyword3 instead of keyword1, keyword2, keyword3 the description in the SERPS goes behind the sponsored links on the right side. Its a authority website which uses description and keyword tag both the same on a subdomain.

Offcourse it not algo related but remarkable its back again.

[edited by: Markoi at 11:08 am (utc) on Feb 26, 2010]

[RishiRich]

I don't have a website address to check the same.... but that's something really strange....

[RishiRich]

Well Which Version Of IE are you using..?

[drall]

whitenight sounds bout right to me, the deep crawl has been dead on most of my big sites since mid Jan. And judging from my comp the same goes for them.

Ive HAD IT with all this Google nonsense.

[BillyS]

Lol -

Back here [webmasterworld.com] someone said...

sorry, but in my playbook, Gorg employee saying _anything_ gives LESS credibility to the underlying argument, NOT more.

[Shaddows]

Yeah, when normally this forum is so free from Confirmation Bias [en.wikipedia.org]

Really I get why its needed, but the new URL quoting thing is UGLY

[crobb305]

I'm not surprised it's still in development. In Google's defense, they told us it was going live to one datacenter back in November so they can "collect data and improve the technology". I can't imagine that getting done in 3 months. Caffeine can't be implemented until those data are analyzed and improvements are made. Having said that, I was hoping it would go live in January based on what I saw in the sandbox last summer. I had a feeling that would be too good to be true :)

[tedster]

Anyone who was in the room at Pubcon when Matt talked about Caffeine had no question about what he was saying. It was essentially that Caffeine was all but ready and Google was intentionally holding back the launch so as not to disrupt the holidays for websmaters. So yes, clearly something was not "all but ready" as anticipated.

We don't know all the infrastructure changes involved, but one thing we do know is that Caffeine includes a next generation rewrite of the Google File System - the underlying structure used to actually store the data. The individual chunks of data are much bigger than in the current Big Daddy version. So no, the two are not compatible with each other at all. The data sets can't even be stored on the same drives.

Something quite mission critical about Caffeine must be a fail, and it's probably not the SERPs it produces. For example [a guess], Google may be having trouble bringing enough servers online that are ready for the new infrastructure. It's one thing to run a test data center, and another thing to be ready to deal with the full firehouse of Google searches made across the world.

[edited by: tedster at 7:34 pm (utc) on Feb 26, 2010]

[roadkillu]

Agreed tedster. I was at Pubcon and listening to Matt it was clear that he was expected it soon after the holidays as well. The implication was that it was "ready", but as a courtesy to webmasters, they would hold off on rolling it out.

It seemed like a nice gesture by Google at the time...

[BillyS]

For example [a guess], Google may be having trouble bringing enough servers online that are ready for the new infrastructure. It's one thing to run a test data center, and another thing to be ready to deal with the full firehouse of Google searches made across the world.

I think this is a great example of what can happen on a project fo this scale. I'm not saying this case (nor is Tedster) but it points out very real challenges with live systems.

For example, Google has never done this type of infrastructure cutover in the past. Before the days of Big Daddy, Google would slowly refresh data across all centers. You could see an update happening at the IP adress level. Updates like this were started and rolled out manually. So when Google cut over to Big Daddy all they had to do was make sure no update was in progress. The systems / SERPS before and after could be compared side by side since information in each system would have been the same or the change was predicted ahead of time.

With the constant update infrastructure that has occured since Big Daddy, it's likely to be much more challenging to bring a new infrastructure online without an outage. It's not easy to compare systems (and identify defects) because they're both in a constant state of flux.

[kd454]

As many here used to say, when SERPs changes didn't go their way:
"Google is broken!"*...
except THIS time, Gorg actually is.

Man isn't this the truth!

Could it be that the reason that Gorg is keeping so many pages in the supplemental because they don't have the resources to manage everything they are implementing right now? Putting twitter in the serps and other useless nonsense they are doing must eat there bandwidth?

Gorg is putting a hurt on webmasters income and it has be having a effect on Gorg revenue as well!

[brinked]

Matt Cutts did in fact say expect caffeine after the holidays. On his "expect caffeine after the holidays" blog post, he responded in the comment area to another commenter with the following:

the changes that users see aren’t that noticeable. We could have rolled out a bit sooner, but we figured it’s better to save webmasters from the stress of worrying about Caffeine around this time of year–plus the extra time lets us collect more data and check for any remaining issues.

This tells me that the only reason why they were even waiting until after the holidays was as a courtesy to webmasters. He pretty much all but said caffeine was 100% ready to go.

[Leosghost]

And the number of angels which could dance upon the head of a pin is ?

According to Mr Cutts ?

And the dance they would be doing is ?

Please quit paying attention to Mr FUD ..

and before anyone comments ..yes it is "which" and not "who" or "whom" ..because angels dont exist and imaginary things are "which"

[UK_Web_Guy]

Why did google come out and make such a big deal about caffeine back in july last year and then say hey come and test and let us know what you think?

Maybe they wanted all webmasters to go and hit that test server checking their serps and hey presto, google now know what serps are big for SEO's these days, so they can then target these and hit seo's - paranoid? conspiracy theory? utter rubbish - you decide.

all i can say is sparing webmaster stress is bull***t.

They whole thing was contradictory, caffeine isn't a serp changer, it's behind the scenes, you shouldn't notice a difference, then why make such a fuss and why would there be stress to webmasters if it went live then if there weren't serps changes.

Mr Cutts is very quiet....................

[tedster]

Why did google come out and make such a big deal about caffeine back in july last year...

Many speculate that it was to grab some press attention away from Bing - they were really on a roll at that time.

...and then say hey come and test and let us know what you think?

I think that part was quite genuine. Notice they were especially asking for help from power searchers - researcher and the like. They already felt the basic SERPs for top terms were doing fine - they could tell that from their statistical models. But they weren't sure how Caffeine was treating the deeper and more obscure results.

[ChrisHamilton]

Caffeine is showing again on [209.85.225.103.....]

[TheMadScientist]

Tedster's post from a couple of pages ago summarizes why all may not see it there...

We have tended here to use the words "data center" and "IP address" as if they are the same thing - but because of Google's custom load balancing, they are not. Any IP address can be mapped to any data center!

WW_Watcher has been posting about the differences in what people see on a single IP Address due to load balancing for years, so YMMV individually on the above IP address.

@ ChrisHamilton No offense intended... I just thought I'd point out why it may seem to be there for you now and not for others, or be there for you now and not for you later, because all it really means is you got load balanced to the DC containing Caffeine. It does not mean Caffeine will show for everyone on the IP address you saw it on any time they visit directly.

[whitenight]

WW_Watcher has been posting about the differences in what people see on a single IP Address due to load balancing for years, so YMMV individually on the above IP address.

It is very important that people remember what a few of us were seeing with Caff. was more than load balancing
_{(please see my several posts about targeted spyware)}

And while one might only hear and wonder about the real "behind the scenes" story
in some book from an ex-Gorg employee 20 years from now,
or
at some random bar as ex-Gorg employee tells a "funny little story" from the past,

some very nasty things were prevented,
for the time being (and perhaps, permanently)
with the delay of the current incarnation of Caff.

To those people (they know who they are) who worked behind the scenes, we say "thank you!" :)

----------------------

Be aware!
I seriously don't post these "rants" to "sound cool"...
It is to make people more aware of whats really going on.

Take it or leave it,
but i have NOT misled or misinformed since posting....

[TheMadScientist]

I was talking more about why people might hit Caffeine one time and not another than what they see (or get) with Caffeine...

On the note of 'spyware' since IMO most won't go back and read your other posts, I did find it, uh, odd? (IOW mind-boggling, stunning, shocking) earlier today when I searched for a specific term using Bing (I haven't used G too much lately and have cleared my cookies a couple of times since) and visited two sites, then refreshed my gmail account and the ad displayed was for the exact term I searched for (using Bing not Google) even though there's no e-mail in my inbox related to the term and it's not in my normal rotation of ads... Needless to say I sat there and stared for about a full minute and all I could think was, 'Are you completely kidding?' Maybe it was a fluke, but it sure blew my mind for a minute...

[kevsta]

Shaddows you are completely right. But the frustrating part is "Why let the Crawling suffer ?"

The recent SEO efforts have all gone in vain because there has been very little crawl activity.

I expect this is one side effect of the situation they aren't bothered about.

[RishiRich]

Hey! few Senior Members over here have been saying that "Caffeine seems to be incompatible with Big Daddy"

So if it is actually true and caffeine is not going to be rolled out for the next couple of months. Is it ideal to assume that the next deep crawl will happen in April.

[panicbutton]

"I expect this is one side effect of the situation they aren't bothered about."

LOLS

[caribguy]

So, we all agree to call this period "Chicory" or "Sanka" until Caffeine goes live?

[kd454]

Caffeine is showing again on [209.85.225.103.....]

Yeah! finally can see it again, when ever I see caff on this data center traffic jumps very high as well as rev.

[TheMadScientist]

So if it is actually true and caffeine is not going to be rolled out for the next couple of months. Is it ideal to assume that the next deep crawl will happen in April.

I would think it would be more ideal to assume the next deep crawl will happen when they have the necessary processor resources to perform one and performing one will not cause any additional issues with the transition they are making from one size / format of data storage (Big Daddy) to a different one (Caffeine), and when that will be IMO is anyone's guess.

From what I've read they have two file structures that are incompatible with each other: Big Daddy and Caffeine, which strongly implies two different storage methods, which would mean the crawling data either has to update both or one has to be left out... I can think of a few good reasons they might be waiting to crawl deeper, and a few reasons why they might start again at any time, so assuming they will begin again at a certain date is simply a guess, IMO.

The information here might be useful for people to understand some of the complexity and challenges and why I think it could be any time: [webmasterworld.com...]

Post 21 is a nice summary and provides some other good references.

[edited by: tedster at 8:44 pm (utc) on Feb 27, 2010]
[edit reason] fix link - paginated links are problematic ;( [/edit]