Google MAYDAY Update - SERP Changes May 2010

[pontifex]

< continued from [webmasterworld.com...] >

Now, as the dust settles from the last days and my panic is under control :->, I thought I share some ideas here.

Basic infos:

1. Nature of site: eCommerce

2. Type of link building: across the board
specialties: our sellers get subdomains, minimal link buying, majorly press, partners, etc.

3. Pages on domain: somewhere around 3 Mio. pages,
majorly product details and listing pages of products


History: Until Tuesday we ramped in around 200k Uniques a day - very strong long tail traffic, 65% of the landing pages are the product details.

What happened: First the site lost one PR point from 6 to 5 in the last update. Traffic went up, so I did not care at all, but maybe it completes the picture.

Tuesday, April 27th: solid and good traffic, but slightly lower than last week.

Wednesday, April 28th: first noticeable drop of around 10%

Thursday: 25% loss of traffic to Tuesday

Yesterday: 27% loss of traffic to Tuesday

Think this is the current status: Lost around 25% of power/traffic somehow.

--------------------------------------------

What happened?

Hard to say really, but here are some cents for the collective thought pool:

a) Keywords I monitor are still where they were, but these (around 20) keywords are not really LONG TAIL

b) Some merchants selling strong in the past weeks dropped out of the sky and vanished from our top sellers list: that could be the reason.

I looked here in this senseless long thread :-) and found the following quotes spark a clue:

I think the loss of long tail is more to do with Google's incredible improvements in Adwords relevance matching. - internetheaven

Well, my loss in traffic is majorly long tail - so that area fits at least.

Long tail traffic in most cases relies on internal link juice, this is how it works. - SEOPTI

There is so much insight in that one sentence, I love it!

And I can relate to it in my case:

The few top sellers vanishing from the list are providing thousands of niche products for very specific purposes. They lost the traffic and that whole picture falls in to place:

If this is the Caffeine update, IMHO the following happens while we type:

Deep content is gathered from domains not crawled so deep before. The amount of pages with long tail keywords doubles, tripples or just magnifolds and changes the game as a whole.

If internal link juice is now weaker or stronger: Link building has now a different meaning.

Link juice must be handled under a differen perspective.

Even if we can build up 1,000 named backlinks to a page, it looses its power on 3 Mio. children quite quickly.

If there are major authority sites unlocked by Caffeine with "before 100,000 pages" and after "10,000,000 deep pages" we have now

9,900,000 new enemies in the long tail....

That would explain a lot to me and I wonder what you think?

P!

< the active thread for this discussion is here: [webmasterworld.com...] >

[edited by: tedster at 7:44 pm (utc) on May 22, 2010]

[pontifex]

@renoirm: thanks for sharing. The 5 point list is a good summary. I looked at a few top 500 from Alexa and saw no changes in their traffic flow. There do not seem to be obvious drops for that subset - the opposite seems true, but that data is so unreliable!

Quantcast and compete data is not up2date.

So lets wait and see how that turns out or if Matt is willing to share some background when he is back from holiday...

P!

[renoirm]

@pontifex IMHO Alexa is not relevant anymore. Quantcast is good if you are quantified.

[TheMadScientist]

I think whatever is going on it hasnt finished...

Yeah, the biggest problem with Google updates anymore is they think it's politically incorrect to have a fat lady sing, so they just keep going... and going... and going...

[Andylew]

A bit more info, last homepage cache was on the 30th april.

Since then googlebot has been busy, it has crawled ~150k pages in the last 3 days. All ips seem to be in the range 66.249.65.x (66.249.65.54 - 66.249.65.55 - 66.249.65.12 - 66.249.65.15 - 66.249.65.21 within the last ten minutes alone) Are these specific to a single datacentre?

[TheMadScientist]

Actually, the IPs at Google don't work that way any more... Any single IP can be routed to more than one data center, so you really have to watch the results more than anything else to know what dataset you're getting.

I may have misunderstood your question if you're talking about GBot, but it's still good to know IMO.

[renoirm]

This a pretty expensive vacation Matt Cutt is taking. Guess everyone needs some R + R

[Simsi]

I belive the reason the clean well seo'd sites are feeling the effects first is that these are the easiest ones for google to digest and update (onto caffine) but doing these first results in reduced positions until the final switch.

Expanding on Andylew's point a little, and noting that most people here at WebmasterWorld - SEO aware people - are reporting drops, is it not possible that Google's new infrastructure means that certain aspects of SEO are now less important, maybe even redundant, to Google?

Are the sites that are rising any different in terms of content to those that are dropping and is that content more "sticky"?

Ultimately, Google's aim must be to eradicate any ranking factor that can be manipulated. Sure that may take years, but... baby steps...?

[renoirm]

Everyone...take a look at you google webmaster tools spider crawl rate setting. You can view it by changing it to custom rate. Do you see it higher? Lower? Just curious since google get's to pick if you have it set to Let Google determine my crawl rate (recommended).

[sean22]

I have 2 niche sites (little under a yr old) that have also been affected. One lost 50% traffic on April 27th and the other on April 28th. Traffic remains low, although it has improved a little since 6PM EST May 1st ( as noted by another member here as well).

I'm hoping this is similar to what happened to another site of mine on May 15. That particular site lost most of it's traffic and rebounded with 3x more traffic about 10 days after that. SERP wise my two affected sites remain the same if not better but traffic is much lower (probably very long tail phrases ). I am confident things will change for the better, if not then I might feel compelled to break something. These sites are clean, were slowly progressing and are unique, helpful and informative.

[1script]

Everyone...take a look at you google webmaster tools spider crawl rate setting. You can view it by changing it to custom rate. Do you see it higher? Lower? Just curious since google get's to pick if you have it set to Let Google determine my crawl rate (recommended).

Interesting, never noticed this feature because it has never worked (was disabled for my sites, anyways) when I opened the page. It does now. My server can handle something like 100 times what they recommend easily and will probably withstand some peaks way above even that just fine. So, in the spirit of testing I just cranked that setting up +600% - we'll see what happens :)

[BillyS]

We've experienced around a 15% drop in traffic, but based on the hourly pattern it seems like there is a lot of switching between data sets. In other words we're down 20% for two hours then up 10% in the next hour,then back down 20% for a couple more hours.

I'm comparing week-over-week hourly traffic patterns, which generally don't move more than around 2% each week.

[trakkerguy]

Are the sites that are rising any different in terms of content to those that are dropping and is that content more "sticky"?

Not in my case. My big site with good, sticky content has been dropping for various keyphrases. My little support sites have risen for same or other keyphrases. The ones that benefited all have "less sticky" content.

I do think it is intended to counter rank manipulation and present what they believe to be niche "info" sites, but isn't effective in my niches yet. For now, I'm starting to focus more on feeder sites.

[Andylew]

I have a new theory!

Im still convinced by my earlier post where i pointed out the drop in sitemap reported listed urls to 0 then the build up again is an indication of things moving to caffine.

My thinking after that as was pointed out was fundermentaly flawed. However, as stated previously google is taking down datacentres to update them to caffine. So what if the datacentre that has been taken down is the authority for a particular site? Im obviously well aware of the redundancy in google but fundermentaly somewhere must be an authority and Im guessing some sites, amazon etc would have a higher redundancy priority than others and be distributed and duplicated across multiple datacentres? For other long tail sites perhaps it is actually faster to have all their data stored in a single location with redundancy in that datacentre being viewed as enough, they may not have enough authority to demand multi location redundancy? Take that datacentre down and there is little redundancy for that particular site in other datacentres so it slips in the listings until other datacentres can pick up the slack?

[tedster]

One of the interesting factors people are reporting - if I understand correctly - is that we are not just seeing a one time drop in ranking. Instead, it's a process of dropping a bit, then a bit more, then a bit more - and sometimes the pattern breaks a bit with a return to good rankings for a period.

Is that an accurate reading?

[TheMadScientist]

I think so tedster and the reason why I wonder if it's a big site thing (I'll know more in a week or so) is mine're doing the opposite... They're climbing then taking a 'big step back' then returning to the 'climbing steadily' pattern, which seems to be what some of the really big sites are doing, in reverse. I'm only watching a couple of the intermediate sites right now, but they're definitely behaving almost exactly the opposite of what the people reporting drops here are reporting right now.

I say I'll know more in a week or so, because today is comparable to the beginning of March levels... Way down from the average trend for a Monday, but they've done this before occasionally and returned to the overall trend within a few days. It's actually been going on this way for months...

I should probably also note: Last week and weekend were right on the trend, and today's an anomaly day again. Watching traffic on a graph over the last few months has actually been quite a bit like watching stocks. I've seriously thought about drawing trend lines to see if I could sell at the top, LOL.

[renoirm]

@tedster @themadscientist I can't attest for other people but yes saw a small drop on subdomain. Then it slowly came back then drop on main site (our blog and single sub domains also effected). Our initial thought was spammer, algorithm change or some technical glitch. 2 days after site drop saw a return to levels (we didn't do anything so it was google). We were fine for 4-6 hours. Midnite hits and down again.

[BillyS]

Is that an accurate reading?

That's what I'm seeing. The site is not nearly as big as the others mentioned here. We've got around 1,400 pages and get around 700K visitors each month.

[Simsi]

One of the interesting factors people are reporting - if I understand correctly - is that we are not just seeing a one time drop in ranking. Instead, it's a process of dropping a bit, then a bit more, then a bit more - and sometimes the pattern breaks a bit with a return to good rankings for a period.

I would have thought this was condusive to the theory that Google's new infrastructure allows them to better measure user behaviour. Knock sites up and down, measure user behaviour in different positions for different searches and gradually a more settled pattern will emerge. This can be achieved gradually with small jumps up and down - like continual A/B testing although it could take weeks, that's anyone's guess.

It also ties in with an infrastructure change which is likely required to collect and store the data, automate movement and measure the effects of each comparison.

Plus it strikes me as common sense to use the effectiveness of a site increasingly in rankings while reducing the effect of aspects that can be manipulated.

My positions yoyo daily at the moment and if I was a betting man, that's where my money would be.

[mikeclover]

We are seeing the same thing, drop in traffic, and then some more, specifically for competitive keywords, and then BAM.....! Traffic is rocking and so are sales.Then all the sudden traffic is gone.... There does not seem to be a long period of traffic holding steady. I see everyone in my industry bouncing all over the place. Driving us crazy over here.....

[Errioxa]

And Google said

"One Large Index to rule them all, One Large Index to find them, One Large Index to bring them all, And in Google's world bind them"

[tedster]

Knock sites up and down, measure user behaviour in different positions for different searches and gradually a more settled pattern will emerge. This can be achieved gradually with small jumps up and down - like continual A/B testing although it could take weeks, that's anyone's guess.

It also ties in with an infrastructure change which is likely required to collect and store the data, automate movement and measure the effects of each comparison.

Bingo - that's my suspicion, too. We saw a little bit of this behavior beginning two years ago, but it was rare and it seemed only to be used for URLs where something that helped them rank seemed a bit unnatural - even though it was not at spammy or outside the guidelines.

The bottom line for Google is not whether the website follows their guidelines. It's whether their useers like the search results.

[potentialgeek]

How about if Google is testing user behavior for the large respectable sites which it previously assumed were relevant for long tails?

Here's the Geek Question: if you were a Google engineer and suddenly had all this extra data storage capacity, what would be at the top of your list to implement into the Algo--stuff you had wanted to do previously but couldn't because of the data limits?

[tedster]

Good question - and a lot of these geeks have degrees in statistics, too. that gets me thinking.

[Freedom]

I've seen a massive disappearance in backlinks as Google must be discounting them, and even removing many pages from the search engine. I checked out several low quality blogs and directories I used to have links on and those pages are gone. Disappeared. If you had a link from a sub-par page, changes are that page is gone from the index, or your link just doesn't count.

Around April 1, I went from 3,000 to 1,600. And then on May 1, I lost another 300 to 1,300. Despite this, Yahoo reports 15,000 links.

So Google's new interpretation of my site is that it has a lot less links which has affected it's short and long tail rankings.

Fewer links seems to have hit my adsense account with a bad bout of smart pricing.

[graeme_p]

I had my best every wek last week.

The most notable thing about my backlinks is that for a long time I have done no link building and relied on entirely natural links.

[mrez74]

For a few of my older sites (2 years+) these seem to have settled down in that they show the same spot in Google.com all the time and when I check on that one caffeine datacenter IP as well. For some of my newer sites the jumping around continues all the time. I guess that is slight good sign that hopefully things will begin to settle down once and for all.

[Shaddows]

Here's the Geek Question: if you were a Google engineer and suddenly had all this extra data storage capacity, what would be at the top of your list to implement into the Algo-stuff you had wanted to do previously but couldn't because of the data limits?

How about, making personalisation work? Feed everyone A/B testing, find demographic prefs, exploit.

FWIW, the concept of reducing the megasite Gravitas Constant* roughly correlates with my data (at least, its not inconsistant). Pages that "just ranked" for mid-tail are not stable.

Can I encourage some other data-crunchers to start looking in the wilder regions of volume keywords (past page 5)? Churn has increased. I have a sneaky suspicion that two things have been devalued- internal linking and "TrustRank". It's rippling through midtail and lower-order volume SERPs, and I reckon it will hit the big time soon.

Incidentally, Vince et al could have been a precursor to this, in protecting Branded (or otherwise "expected") results from the ensuing disruption.

*Like the Gravatational Constant, additional mass (pages) automatically increases attractiveness

[member22]

The bottom line for Google is not whether the website follows their guidelines. It's whether their useers like the search results.

Tedster do you mean that google would be looking at bounce rate to rank sites ?

I've seen a massive disappearance in backlinks as Google must be discounting them, and even removing many pages from the search engine. I checked out several low quality blogs and directories I used to have links on and those pages are gone. Disappeared. If you had a link from a sub-par page, changes are that page is gone from the index, or your link just doesn't count.

Hi Freedom,
Interesting thing that you say but if google is counting less links for you it is doing the same for everybody so to me the ranking would stay the same for everyone, it is percentage.

Then you say that google is taking less link and pages into account but on the other hand everyone says google has more data storage capacity than it used too , so why would they remove webpages from their index.

Finaly

Ultimately, Google's aim must be to eradicate any ranking factor that can be manipulated. Sure that may take years, but... baby steps...?

If this happens this is the end of google because if you don't understand how google works ( at least part of it ) you won't ever rank anywhere and by the way remember that unless you are in the TOP 3 on the 1 st page of google you will never make a single dime on the internet.

[Shaddows]

bounce rate to rank sites

Search refinement is less noisy

[Andylew]

For a few of my older sites (2 years+) these seem to have settled down in that they show the same spot in Google.com all the time and when I check on that one caffeine datacenter IP as well. For some of my newer sites the jumping around continues all the time. I guess that is slight good sign that hopefully things will begin to settle down once and for all.

I can also say this is the case, an older site is settled the new site lost 90% of traffic. However webmaster tools shows the new site is continuing to grow with more pages being listed, however searches dont yet reflect this.

Tomorrow/thursday is going to be a very telling day, the past 3 months we have seen search engine traffic increase considerably on th 5/6th of the month.

[edited by: Andylew at 9:12 am (utc) on May 4, 2010]