2: Google MAYDAY Update - SERP Changes May 2010

cangoou,

If possible, can you please tell me if you are seeing the Caffeine dataset from 0am to 5am local time and the regular dataset for the rest of the day?

I Google "site search" tested a bunch of small sites published this year. When I clicked through to the cached page I got, "Your search - cache:LVC4Sap71YQJ:www.(domain name).com.au/frequently_asked_questions site:(domain name).com.au - did not match any documents."

A number of these sites had been stripped back so that only their Home page remained indexed. All of the sites I tested were published by different companies - so no commonality factors were at play.

Doesn't this suggest Google is having problems with its index?

Just a quick update, traffic has yet to return but the sitrep seems "stable" atm. Gbot is still going insane, yesterday we recorded a new record of 300,000 gbot hits for the day on our biggest site.

Dusky is spot on with his/her reports, our data/conclusions are the same. If trust has been somehow turned off that would explain the pure dog crap that is taking our positions.

Gulp
All quiet on the western front. Hope the second act isnt a death blow but a fix. Silence out of the plex.

We use a 3rd party company to conduct feedback tests on our top 200 three phrase terms in google. We do this once a year. Lets just say the results that have taken our place did not come back well received.

Im sure google uses the same type of thing but on a much larger scale. They HAVE to be seeing this to.

They HAVE to be seeing this to.

I agree, and their silence speaks volumes.

..................

Google lost search share in March, something like almost a percentage point. This is the first time ever that I can recall.

May the users will give google what it deserves, a trip to also-ran land.
I'm using Bing now.

I think Google's health benefits are about to expire. There is always a first time for everything in life.

The main reason why Google lost its share in the past few months is because of the lesser activity happening in its SERP and the stability is at its lowest because of all the changes they are doing. People panicked; ...they were not ranking good, their site disappeared in Google, they are losing money, they sought help from Mr Bing, Mr Yahoo and Mr Facebook. But once Google finishes all these up and people started to see the benefits of the engine overhaul expect the shares to come back up even stronger.

[edited by: louieramos at 4:57 am (utc) on May 13, 2010]

I have not seen SERPS like this. At times, for 30 minutes I get ZERO-10 referrals, then I get referrals with keywords that are amazingly good.

There are definitely two sets of data, if not more.

There are 2 sets of data indeed, which just means Google aint done yet..

The main reason why Google lost its share in the past few months...

Google gained search share in April. See [online.wsj.com...]

There is a lot going on with Google and we may need to approach the Google SERPs from an entirely different model than the "data set" concept that we've been using in recent times. I'm currently thinking of something more along the lines of waveforms and continuous statistical testing... with near-real-time adjustments and experimentation in a kind of feedback loop. Something like automated algorithm evolution.

tedster,
that can be a codeword for a trying to fine tune it before taking it live. I noticed that in Google Canada what I consider Caff is live, in US it isn't, at least not for a significant amount of time.

So my guess is, like yours, that Google is using the data from those exposures to see how to improve it. I think we can agree that something is not going as planned, caffeine was supposed to come after the 'holidays,' and Google would announce it if it was live.

I see constantly two sets of data during the day when I check on my keywords. One has all my sites in better positions and the other data set in lower positions.

Maybe something isn't going exactly as planned last year - but I really do wonder if this aims to be an automated, and self-improving algorithm. With all that talk about AI over the past two years and then the signs we're seeing now, I do suspect it.

Right now it is self-improving like Hal9000.

And we're all named Dave, right? LOL!

I'm currently thinking of something more along the lines of waveforms and continuous statistical testing... with near-real-time adjustments and experimentation in a kind of feedback loop. Something like automated algorithm evolution.

Interesting point. I am just trying to be a devil's advocate here and trying to see what if this is how the new algo works (ignoring the GOOG is broken discussion for now).

Maybe Google studies big and small websites differently. Small websites which have only a few dozen pages are majorly static - a possible indication that they contain evergreen content. On the contrary, the big websites must possibly be adding several dozen pages every single day - indicating that the old content on these websites gets stale after sometime.

With Caffeine focusing on real-time, could this be a reason why small static websites are getting a lot of Google-love and big websites are seeing drops?

Hello everyone. Joined this site after seeing traffic all but go and a quick search led me here(about the only search that seemed to work!). I really do not know what G is up to but it has hurt us badly, sites that used to take 4 figures are doing zero. Searching for popular search terms reveals very strange results which deff. change at different times of the day. We have noticed erratic traffic for a while but this week (well, started last week) has been terrible. I really do not know what to do, we haven't changed a thing. The silence form G is very annoying, do they not realise this could actually close a small business down?! All i can do is sit tight but if this isn't fixed & fixed fast then it will begame over for us.

Right now it is self-improving like Hal9000

"What are you doing Dave?"

I'm just searching for something Hal.

"I don't think you meant to type that Dave ..."

I want to search for wigdets Hal, not widgets.

"I can't let you do that Dave ..."

@gouri:

If possible, can you please tell me if you are seeing the Caffeine dataset from 0am to 5am local time and the regular dataset for the rest of the day?

To be honest I'm not quite sure. I would bet the dataset from 0am to 5am is more "caffeinated" then the other one, but both datasets seem to be new somehow, because I never saw this constellation before (at least on the SERPs I watch).

@internetheaven: Great ;-)

Reading ^that^ has just made my morning :)

A newish site (7 months old) I've been working on has taken a major hit by practically dropping out of the results. No matter how deep I search I cant find it but its still indexed. The same thing happened during march for approx 2 weeks then traffic shot through the roof with it being in better positions than previous.

Competitors have also disppeared to be replaced with review sites, directories and facebook groups although they seem to be slowly coming back into google.com, its as if someone is tinkering at the backend. The results in Google.co.uk are however a complete mess, again with review sites and empty pages shooting with specific keywords in the title.

My biggest concern right now is my main site, it started to tank on Tuesday 11th May and its more than just longtail thats dropping. One keyword I held 2nd position on for a couple of years has started to move further down the page. I havent done anything mad with the site, its mostly unquie content articles has been rock solid for the past 4 years.

Has anyone else seen a nose dive since Tuesday?

I want to search for wigdets Hal, not widgets.

"I can't let you do that Dave ..."

==

Would be hysterical if it weren't so true. I'm getting sick of typing [+wigdets -widgets]

Hi,
I follow this thread occasionaly and from my niche (a very competitive one) I see no such dramatic changes as presented by other members. In fact I see the usual competitors, the usual top 3 results and some daily changes after the top 3 but no dramatic. Traffic as usual, my 2 cents.

map1, curious, do you track your long tail kws? any changes there?

No changes, the same, usually my top queries are one or 2 keywords following by 3 words (10% about).

Maybe the users will give google what it deserves, a trip to also-ran land.
I'm using Bing now.

If Bing wanted to do some totally radical, they'd copy exactly the classic Google interface, and advertise themselves as "Your Old Friend Bing" ~ the sounds you'd hear would be millions of people moving in their direction. What a coup that would be, and the irony would be delicious.

...........................

@tedster: interesting approach with the AI theory. You mean like:

1st: basic algos applied
2nd: basic filters applied

then let speed, time on page, clickthrus, etc. influence the ranking in real time over time?

Sounds interesting, but I doubt this methodology is real AI, because of the massive amounts of data. It is an interesting aspect, though that the movements of the masses influence rankings for keywords.

On the other hand: The masses very often make very bad decisions - let me tell you that as a German!

Since 1st April (but looks like it wasn't a joke...) I'm loosing referrals from Google, these are my stats:

- 50% of referrals from Google were lost, from 1000+/day to about 500 (every day going worse)

- A "site:" search now shows about 18K pages, while in April it reported about 200K.

- GWT doesn't say a word about possible issues, but the "crawl stats" are constantly lowering, today at 5K/day ( 25K in April )

So Google doesn't care about my site anymore ...


I'm not asking for a cure, I know there isn't, I'm just giving my 2 cents.

[edited by: tedster at 4:49 pm (utc) on May 13, 2010]
[edit reason] fix a small formatting issue [/edit]

Welcome to the forums, Andydandy. Sorry it's a devilish problem that that brings you here. This community may yet find a way to understand in a bit more depth - and in an actionable way.

WARNING: This post is very long

Just wanted to share some good news here - one of my (many) sites hit on March 15th is seeing a nice 20% up-tick in Google referrals since yesterday. Googlebot's activity has also increased 10 times.

This is a far cry from full recovery (lost 90% of G* traffic in March) but the first upward movement in almost 2 months.

Basically, what I'm seeing is a return of the same keywords to approximately the same positions (although they now tend to fluctuate a lot, as many pointed out here). Just as if the data about the pages was lost for two months, then some of it found.

I have also done some analysis of my G* referrals before March 15 crash and after. What I was specifically looking for is whether the type of the keywords that continued working after March 15 differs in any way from those during "normal" times. Meaning, has the traffic become more "long tail" or less. In this study I sided with those people that measure the "length" of the "tail" not in the number of words but in the number of searches for those keywords. This is because some of my best performing KWs would be ridiculously long tail if you measure in words - 5 to 7, and that was the case both before the crash and after.

So, I took the total number of keywords I received G* traffic on and came up with percentages for the keywords that only occurred once (produced referral once) in a month, then 1-5, 5-10, 10-50, 50-100 and more than 100. For all intents and purposes those above 100 are the "shortest tail" or "trophy" and 1 - the "longest tail".

In addition, I looked at two sites - one that has been affected and the other one that has not.

I'm going to have to spare you a lot of details (that I don't feel comfortable sharing in the open anyways) but, after a couple days of grepping, awking and sedding the end result was the exact opposite of the hypothesis I started the study with:

1. The affected site was slightly less dependant on long tail than the non-affected one
   
2. All percentages for keyword frequency ranges for before the crash and after were very similar, within the last significant digit or so (i.e. 0.015% vs. 0.011%), in other words no significant change in "length of tail" was observed.
   
3. Non-affected site was no less different in the percentages for keyword frequency ranges than the affected one, in other words, any difference can be explained by randomness.
   

The only thing that was actually very different was the total number of keywords that sent traffic. So, I'm squarely back in the camp that believes in G* loosing ( retiring, invalidating, corrupting, whatever it is that happened) some part of its index during the move to Caff infrastructure. Sh.. happens, so to speak.

They do seem to require heavy re-indexing in order to restore that lost data and they are doing it top-down, meaning, the top dogs (who's amount of backlinks still makes me cry) get the crazy re-indexing first and we, bottom feeders, get it later. Could be that the spammers and the penalized folks may not even get the re-indexing thus falling out of the index in a "natural" way.

The last paragraph is completely my speculation - if you have another way of interpreting the results I got, I would be very interested in hearing your point of view.

Peace, everyone!

P.S. if anyone's interested, I ran the same compilations for Y! and Bing referrals for the same periods. Both show slow but steady increase in the total number of referrals and the type of keywords is remarkably steady across the three month period I looked at. So, it looks like Google's changes are working. For Microsoft.