Google Updates and SERP Changes - April 2010

[wmxpert]

< continued from [webmasterworld.com...] >

We are in the same situation, since March 15, 2010 our traffic is continuously declining with the average of 6% every week. In current week it gets worst, approx. 15% decline!

Overall, 30% to 35% traffic has been reduced compare to peak week of January 2010.

"site:" is also showing huge drop in index pages, though it only provide the avg. not actual.

Don't know what's going on, Is it really the effect of Caffeine roll-out?

[edited by: tedster at 9:46 pm (utc) on Apr 1, 2010]

[Kelowna]

The serps I watch are newer since yesterday, they are the non-caffeine results as they seem to have turned off the caffeine stuff for now on any of the proxies I use.

[TheMadScientist]

The SERPs I see now, and have been seeing for 2-3 days, are desperately old.

Can you define desperately old? I changed every URL on a site 4 months ago and added ~3000 new pages and traffic is steady, which IMO would definitely not be the case if 'old' is standard across the board, because they didn't exist very long ago, so if the SERPs are truly 'old' everywhere they should not get traffic should they?

It's actually a really interesting question to me, because if the SERPs are truly old, then how are any changes made since then getting traffic at all?

Seriously, it always shocks me when people post about how 'old' the SERPs are when I changed absolutely every URL on a site 4 months ago, and added new ones, because the traffic has been steady to increasing through this whole change over, so what people are reporting does not really make sense to me...

[tedster]

I'm in the same camp as the Mad One - I don't really understand what people mean by comments like "old SERPs" and "roll back". Does this mean that the rankings on certain queries look like they used to at some past date?

[helpnow]

-sigh- Well, these certainly aren't the new SERPs we've all been expecting. The newest SERPS in the past 6 weeks were the ones trotted out during Easter. Since then, there has been a roll-back. How old are the current ones? Well, not sure exactly, I've lost track of all the starts and stops gorg has put on lately, but as I recall, these are around 4-6 weeks old, meaning, these are the same ones we were seeing back then, and they were old then too. These SERPs have many up-to-date changes from 2 minutes ago, that's not the point, rankings are not right, and many reported stats are incorrect. Easter was the most-up-to date SERPs lately, and these are not like Easter. These SERPs are old. They are a roll-back. Sorry, it is what it is - I was just posting an observation, I always duck when I do because I don't want to be drawn into a debate, I am just posting an observation. These SERPS are undeniably old. Even site: worked better, though not perfect, during Easter, and now it is showing old stats again. These SERPs are old.

[arizonadude]

I don't agree that the current serps are old.

I have a set of very specific terms that only showed in the Caff index and they have been holding solid in the main Google now for several days and the serps do not look old for what I'm looking at.

Love it or hate it, I think the beast has been unleashed and now we are just going to see the standard flux that happens normally with different data centers.

[arizonadude]

I don't agree that the current serps are old.

I have a set of very specific terms that only showed in the Caff index and they have been holding solid in the main Google now for several days and the serps do not look old for what I'm looking at.

Love it or hate it, I think the beast has been unleashed and now we are just going to see the standard flux that happens normally with different data centers.

[tedster]

No debate intended. It's just that I want to be clear what any observation is all about.

So what you are seeing now appears not to be using the same algorithm config as what was there quite recently - and instead it looks like an older algorithm set-up. And you don't mean that new crawl data has gone missing, just that the rankings seem to have an old-time feeling to them.

Is that it? And if it is, do you any sense of what newer factors (or weighting of factors) seems to have disappeared?

[TheMadScientist]

These SERPs have many up-to-date changes from 2 minutes ago
...
These SERPs are old

I didn't really want a debate either, but I think it's fairly easy to find the preceding confusing, so I wanted to know what the observation actually was. I had no idea you would refer to 'an apparent roll back in ranking methods' as 'old SERPs'... I thought you were referring to an old dataset or something and could not figure out what you meant. Thanks for clarifying the ranking method appears to be a rollback but the data is up-to-date.

I think one of the confusions is people are seeing algo changes and infrastructure changes at the same time, and drawing a distinction is sometimes difficult... I tried to earlier in this thread when there was talk of Caffeine also being an algo change, because we know speed has been a part of the algo for quite a bit of time (mid Jan if I remember correctly) and at the time the dataset used was almost exclusively Big Daddy, so it's fairly reasonable to draw a distinction between an algo change and an infrastructure change, and IMO it's incorrect not to, but I feel the same as you and don't really want to debate the issue either, because I know (basically) the algo is detached from the underlying data storage mechanism, but if others choose to not see it that way, then whatever.

/rant

Thanks for the clarification on what you are seeing.

[internetheaven]

You know ... we could all be seeing different results ...

I would say there has been a serious roll-back too, and I can't see Caffeine anywhere no matter what I log in to, out of or over. But I'm willing to accept that it might just be me seeing these results as traffic has not dropped as far as the results I'm seeing would indicate my traffic should drop on some sites.

[gouri]

The number of competitive sites that are showing up for a keyword search on Google.com seem to be similar to what I was seeing earlier this week, when there was a steady increase in the number of competitive sites showing up for a keyword search compared to a couple of weeks ago.

That is one aspect of the current SERP that I think we can say has changed.

In terms of competitive sites showing up for a keyword search on the Caffeine datacenter, that has also increased considerably this week and I am not seeing the old Caffeine dataset now that in general had fewer competitive sites for a keyword seach than Google.com.

[TheMadScientist]

Here, I'm frustrated with work for a minute, so let me see if I can explain a bit for those who may be interested...

Google is basically a database... The original storage method was referred to as 'Big Table' implicitly indicating the data collected is stored in a table, and from this inference some reasonable conclusions can be drawn when understanding the workings of a database. (The resource(s) for the references to 'Big Table' and underlying data storage changes are linked in the first post of the Oct. Updates Thread, you may have to read more articles than the one linked to find the reference(s).)

From the preceding knowledge reasonable understandings of the process and what an infrastructure change is compared to an algo change can be drawn...

Note: The algo is technically a heuristic, but algo will be used for ease of reading and typing.

1.) Google is a big database.
2.) The information they have is stored in a 'Big Table'
3.) The algo creates an index of the pages to be shown to visitors for speed of access.

So, when GoogleBot spiders it gets information and stores the information in a table. Originally the storage system was referred to as exactly what it is: Big Table (I'll call it the Original GFS), then Big Daddy (GFS 1), and now switching to Caffeine (GFS 2).

When or after the data is stored in the database table the information is retrieved and the algo is applied to the information.

To access data more efficiently in a database an index is used, which is basically a 'key' to say 'go to this portion of the hard disk to get the data' so rather than having to 'scan' the hard drive to find the location of the correct information it can be 'jumped' to quickly.

Reasonable conclusion: Google and other SEs refer to the dataset they display the results from as the 'index' because pages the algo has been applied to are stored in the index they search to find the results for a given query. (Hence the use of a noindex robots reference to remove a page from the results... It's not a 'noresults' reference or 'nodata' reference or 'nostore' reference, because the technical action happening is the page is not added to the index which is searched to generate the results. It's a direct note to the algo doing the processing and creating the index to not include a certain reference. This is also why a page with a 'noindex' reference on it can (and does) still pass PageRank. It's available in the table (data) used by the algo for calculations, but it's explicitly (at the direction of the site owner or operator) not included in the index published for searchers, so the page is not returned in the results.)

An infrastructure change can cause the results (shown from the index generated by the application of the algo) to change in a number of ways without the algo generating the index (results) changing. The results (index) generated can change from difference in the table data, including: the amount of data available, the speed of the access to the data stored in the table, the speed with which a new index that is publicly available and searchable can be generated, and a number of other ways.

A change to the algo will have an effect on the results (index created for people to search) regardless of the underlying dataset, but there may be different indexes generated from the same algo because of a change to the underlying data.

Hence, IMO, the need to draw a distinction between an algo change and an infrastructure change to have some clue of what is going on, because it's entirely possible since it appears (IMO) we are seeing an algo and infrastructure change at the same time what some of the things being reported are is 'adjustments to the algo' and not necessarily a 'roll back' to the Big Daddy dataset.

Helpnow said the results appeared old, but the pages included in the resultset were up-to-date, so what Helpnow may be observing is a reversion to a different algo (ranking and index creation mechanism) even though the underlying data could still be Caffeine... It's been reported by tedster (and others I think) the Big Daddy and Caffeine infrastructure are not compatible and cannot even be stored in the same place, so my guess is once they roll with the Caffeine infrastructure to a data center they do not revert to the Big Daddy dataset, but rather change, adjust, tune and if necessary, revert the algo generating the index to a previous more stable version...

[tedster]

For anyone who wants to dig deeper into what is going on technically with Google's Caffeine infrastructure, here's a discussion from August 2009 right when the Caffeine announcements first came out. In fact, the code name Caffeine had yet to be announced. Some third party sources were calling it "GFS II".

The interview takes place between Kirk McKusick [from ACM - Association of Computing Machinery] and Sean Quinlan [Google infrastructure engineer].

GFS: Evolution on Fast-forward [queue.acm.org]

From what I can glean, several key factors are involved - first off, to make Google's entire system more failure aware with quicker error recovery.

But even more, Google is moving to a system that uses not only distributed slaves (servers that hold the basic data) but also distributed masters (servers that store metadata for the basic records). The master servers had become a major pinch-point in the system.

The new slaves will also store much smaller files. The chunks will go from 64MB down to 1MB. That alone is a reason why Caffeine and the previous infrastructure are not compatible.

[gouri]

If possible, can anyone please say what dates the new SERP were on Google.com?

I have read some postings on here saying that it was during Easter but if possible can you provide the dates?

[gouri]

I am seeing that in general, people reported seeing it sometime on April 1st and 2nd so I think those would be the dates.

Thanks.

[steveb]

"these certainly aren't the new SERPs we've all been expecting."

I don't know who "we" is, but anybody expecting a particular group of serps to become dominant at some point is living in a fantasy land.

[SEOWebArt]

Hello guys,

I've been reading all your posts in the last month, so I decided to tell my story here as it may be helpful for someone to know about.

9 years old domain ranked on the 19th page of Google. - after SEO is applied to the website around 15th of December 2009 (301s, URL rewrites, new content) it lost rankings and went down on page 60 in Google SERPs.

When 2010 started (mid-January or so) the website got new rankings and was ranked on 35th page for the most competitive keyword.

In the beginning of March the website went to page 24, and after a while, March 15th - the website was ranked on the 16th page in Google SERPs. Which is 3 pages less since December 2009.

I thought the 16th page was the Caffeine's calculated ranking for my website... but after 5 days, it all went back to the SERPs 2,3 months ago.
Today I'm on the last page of Google.com .. waiting for Caffeine to stabilize.

However, Google is perfectly working with indexing of new pages. We did another URL rewrite 1 month ago and I can see that Google is updating its index once a week or so...

As I'm building links since December, I can tell you for sure that every link from mid-February till today is not taken into Google SERPs calculation. The links I got in December and January are showing up in my link profile.. but those from mid-February, March and April, no.

So, all I can say is that this waiting is really frustrating. I don't know what to expect and I don't know what's going on in Google. I'm ranked on the 4th page in Bing.com for the most competitive keyword and on the other side, I'm (currently) ranked nowhere on Google.com for that same keyword.

I'm sure that these SERPs are not the final answer from Google's Caffeine.. but whatever the final answer is, I hope it will come soon.

[BillyS]

What does it mean when someone says the SERPs are old?

I get new pages in the SERPs every time we post a new article. It's usually on the first page too. I don't see a stop to the pattern of Google constantly updating SERPs.

And like steveb, I don't expect a lot of changes with any new infrastructure. I do suspect that some webmasters here will see a big shift at the individual site level, but I wouldn't expect the masses to experience mass changes. If a lot of webmasters are waiting for Caffeine because they think it's going to be their big break, I think a lot of people are going to be disappointed.

[tedster]

Hello SEOWebArt, and welcome to the forums.

every link from mid-February till today is not taken into Google SERPs calculation. The links I got in December and January are showing up in my link profile.. but those from mid-February, March and April, no.

The links that are listed in either Webmaster Tools or the link: operator are not necessarily counted for ranking purposes. The reverse logic also is true. Just because a link doesn't show in the reports doesn't mean Google isn't counting that link equity in the ranking algorithm.

From what I can see, the reporting features appear to be separate parts of Google's infrastructure from the actual ranking algorithm. At times the disconnect between the two gets dramatic and at other times, not so much.

[walkman]

" What does it mean when someone says the SERPs are old?"

Old might be inaccurate, but in my case I am ranking for the same keywords I ranked a few weeks ago. This has been the case several times during this months long update

[annej]

I haven't checked by here for quite a while but wanted to report my pages are doing better overall. I think it's because I have so many links from edu and gov that have built up over the years. My topic is somewhat academic though so I'm not competing with spammers who might be using edu links to up their rankings.

[aristotle]

Ever since early January people have been posting false alarms here about Google starting to roll out Caffeine. Other posters keep saying that Google is playing peekaboo with Caffeine, flashing it on and off on various datacenters. Some say that Google keeps switching around among old, new, hybrid SERPs, etc. Meanwhile most websites continue to get their usual Google traffic.

Maybe people should spend more time working on their sites instead of trying to figure out what Google is doing behind the scenes.

[cangoou]

What does it mean when someone says the SERPs are old?

In my observations I see things like:
- Sites which gained new good links in the last 2 or 3 month are shown at approximately the position they had 3 month ago in the SERPs I call "old", and they rank very much better in the "new" SERPs. So my guess is the new links have not flown into calculating of rankings yet in the "old" SERPs.

- If you have a new domain and want to rank it, you might have to wait up to 12 month (depending on the keyword). Imagine a domain on which the 12 month where over in february 2010 and which climbed steadily the last 12 month. I see such domains on first page on the "new" SERPs and not on the first 8-10 pages on the "old" SERPs (which matches the rankings in december as well).

- I watch some domains which got a penalty which normaly lasts 12 month. The penalty should have been lifted in march 2010. So I see the penalty lifted in the "new" SERPs and still applied on the "old" SERPs.

So it seems to me that new pages can get into the "old" SERPs, but the rankings seem to be calculated on "old" data (at least links and penalties). Might of course be wrong and just a coincidence of the chaos I see ;-)

[tedster]

Thanks, cangou. That is very useful detail - quite clarifying.

I'm just not clear how "climbed steadily the last 12 month" combines with "not on the first 8-10 pages on the "old" SERPs (which matches the rankings in december as well)".

Are you saying that in December, after 8 months online, these new sites had not yet surfaced above page 8 but had climbed steadily to some lower page?

[walkman]

Ever since early January people have been posting false alarms here about Google starting to roll out Caffeine. Other posters keep saying that Google is playing peekaboo with Caffeine, flashing it on and off on various datacenters. Some say that Google keeps switching around among old, new, hybrid SERPs, etc. Meanwhile most websites continue to get their usual Google traffic.

Maybe people should spend more time working on their sites instead of trying to figure out what Google is doing behind the scenes.

Since Jan Google has played with Caffeine, off and on and that's the point of these 'updates and changes' threads.

The good thing is that Webmasterworld has many other sections and threads to fit everyone's goals or concerns, let us speculate here in this thread.

[cangoou]

Are you saying that in December, after 8 months online, these new sites had not yet surfaced above page 8 but had climbed steadily to some lower page?

Yes. It's a bit strange I know, because what I experienced with new domains which became "1 year old" in 2008 or early 2009 is that they at least would be around rank 30-40 after 8-9 month.

[brinked]

For the first time ever I am seeing caffeine on my home computer. However...I do not see caffeine if I am logged in to my gmail account. If I sign out though, I see caffeine. Excelllent

[brinked]

I'm gonna go ahead and say it. Caffeine is everywhere. I see caffeine on all my proxies right now. Anyone else seeing the same?

[Kelowna]

Ya... I see them on search.aol.com and comcast too... is that everywhere? Pretty close...

[maximus12]

Yep I see it everywhere as well, this is definatley caffeine

[imbckagn]

It's been live for about 3hrs everywhere.