Google Updates and SERP Changes - June 2010

[Hissingsid]

< continued from [webmasterworld.com...] >

I've been trying to work out what happened on May 17th and May Day. I seem to spend a lot of time analysing on-page factors and back link profiles for my own ranking pages and those of competitors but have fallen into the trap of focusing on a few big traffic volume terms.

Recently I've been looking at some second tier terms and this has been much more enlightening. This has also made sense of what is happening for the big volume terms.

In site linking seems to be out ranking external back links if done in a particular way.

The pattern I'm seeing is this, when all other factors are roughly the same, pages on sites where the site is organised in such a way that there are a large number of back links containing the keywords in anchor text within the site. ie Sites with many pages each page focusing onto one page with the same anchor text and very few back links from external sites are out ranking those that have a better external back link pattern.

So sites that have Javascript or CSS drop down navigation systems with the same links on every single page seem to be doing very well.

The other thing I'm noticing and this may confirm what I'm saying above. I'm seeing SERPS loaded with home pages.

Anyone else noticed similar patterns?

Cheers

Sid

[edited by: tedster at 11:32 am (utc) on Jun 1, 2010]

[aspdesigner]

"Google IS broken."

@drall: Agreed. Ppl were saying the same thing back in 2003.

I have been going through the numerous posts regarding that fiasco (up to Part 3 now), and other than the different names & year, much of what was said and reported virtually mirrors what we are seeing today.

I mean, to the point I could take a comment from back then, and post it here now, and everybody would say they are seeing the same thing as well!

[tedster]

Ppl were saying the same thing back in 2003.

Yes - and every year in between, too, although the Florida Update was a strong example of a major change.

For me the take-away is that the often predicted "fall of Google" did not happen in the seven years since. To put it mildly! Google has thrived.

If results are as awful as they seem to us, Google will notice that dissatisfaction and take steps to change it. They have a lot of QA metrics and they do use them. Already we are seeing reports in this thread that some shifts are causing improvements.

I ran into a search on a technical issue yesterday where Google was just not cutting it. Given all the discussion, I said to myself "OK you've got an example - see how Bing does." But sadly in this case both Bing and Yahoo results were also terrible.

We do need to face the fact that the search results are going to cater to the masses first and foremost. However, if the results can't also serve serious research by professionals, that will be a problem. Overriding quote marks and plus signs IS a problem, IMO - but my family doesn't notice it.

The new emphasis does seem to be on brands for some types of searches - and yet there are others where big brands are losing. One international brand name I work with lost 15% of their non-branded long tail traffic in May. So there's a miss, if the point was to feed more brand choices. That traffic seems to have gone to informational and academic sites, rather than to any competing commercial brand. And stepping away from my own focus, I can see that those search results also do the job.

So we're working with a complex logic, not a black and white kind of thing. And the gradual return of traffic for some sites also seems to point to some kind of automated feedback system in the algo, in my view.

So again I wonder, what factors make up a brand in Google's eyes? The number of searches on just the business name itself? Unlinked mentions? Offline media mentions? Navigational searches? Whois ownership?

[backdraft7]

While I agree with the general consensus, I would have to differ that it will always be like it is today. I can not imagine how Google could make a change of this magnitude with googlebytes worth of data and NOT expect a period of upset in the SERPS. Between 2003 and May 17th 2010, we did quite well with the help of Google, so I for one am grateful for their service. Yeah, this current situation bites hard, but I doubt the effect of the shuffle will last too much longer. It's all part of the business of being self employed in the wild west online world. There's always the option of working 9 to 5 for da man, so I'll take the occasional upset over that any day. I'm just happy they did it during a relatively slow sales period for us. If they did this over the holidays, you'd hear a LOT more complaints.

I'll also add that my site is completely hand crafted, and not based on Joomla or Wordpress or some other CMS. I have 70 or so tweaked pages that are all white hat, I am a BBB OLR member, I use SSL where needed (forms etc) and I even use Alert Site to monitor my site for security vulnerabilities. I've been online for over 10 year so my trust level is probably above average. I also own a federally registered trademark for my brand. I'd love to see G index all the uspto.gov trademarks and match them up with products / sites. That way the creeps who steal or pass off on my trademark could get busted once and for all.

Today I am seeing definite improvement in traffic quality. My problem is not with Google, it's with the bums who scrape sites looking to illegally ride our wakes by stealing original IP, then over optimize in an effort to jam our own work down our throats. So if G's algo update can improve that situation in any way, I say Go Google!

[drall]

I dont think this will be the fall of Google by any means. Im sure they tested it 6 ways from Sunday and they liked what they saw. But for me and my websites (and they are my websites, not Googles) I have drawn the line in the sand.

We see the sites that have replaced us, we see how they are gaining. I wont do that to my users or my websites. Google accounts for a small portion of our overall visits.

Many of the sites that have replaced us make visitors jump through hoops to get what they want. Maybe our site being a clean and well presented experience leads to less page views or visit length because we give the user exactly what they are looking for and that is playing against in some unintended way right now?

Who knows.

Will this be the end of Google? Not a chance, I would imagine this only hurt a very small set of publishers and as Tedster pointed out the feedback systems will correct obvious errors over time.

Who knows.

All I know is what is. And what I know is that 10 years of playing by the rules just got burnt up overnight in this new and improved version of Google. I know spammy junk filled sites have replaced us. And I know NONE of it makes any sense to me. So to me at least the simple answer for us is that Google is broken. Will it fix itself someday for us?

Who knows.

[scottsonline]

@drall sadly the old saying if you cant beat them, join them. Right now to survive in this new version of google you have to go create ibuywidgets, getmewidgets, morewidgets etc etc, flat file import and XML file and Google will index 10-15% of your pages on each site for very little work. That's how our competitors are getting ahead. It seems they've all figured out a simple change to the layout and a different source IP is all you need. It's 2003-2005 all over again.

@tedster. This last 7-10 days has been really bad in our industry. We're up to almost 15% bing traffic today on very heavy traffic. Yahoo is up as well. Google is still treading water and the traffic is garbage. Our users are switching to bing and yahoo probably noticing the same thing we're noticing which is brand product name is a mess right now for searches on google.

When the #1 ranking search engine in the world ranks #5 in that same engine it tells me they've got something wrong somewhere.

[Yippee2]

Hello all, I have found this thread to be a great resource and really appreciate all the insight. I am not a webmaster by any means, I have a traditional job but the better our website does, the better off I will be. So I do help alot on our sight and believe I have a fairly decent aptitude for SEO and would like topoint out a few things I have seen in no deliberate order;

My business is a pretty small niche in the grand scheme of things and my major keyword always returned G results in the area of 30 million. for about the last 4 weeks, the returned results have been steadily recovering from a low of about 300k pages to currently about 6.5 million. I point this out because it coincides with the rebuilding index theory. It has never gone backwards from the one massive drop, just slowly increasing little by little.

I just recently signed up for GWT and cant say alot about it other than my impressions are always the same for any 30 day period either "x number" or "y number" but never anything different even if the daily number is different than the difference from x to y. And the "average position" not even close anywhere across the board. So I conclude that GWT is broke.

I have seen 2 pages dissapear from the cache: but stay in the site: only to reappear in the cache: a week later or two. I am seeing serps that say "17 hours ago" next to my snippet only to have a cache 4 days old. I last week released 50 pages and had them indexed and cached within 2 days but saw a few anomolies in descriptions and snippets showing up mismatched for about 72 hours then they all settled properly.

There have been numerous things that I noticed but didnt share and will be better about posting but from were I am sitting with a true rookie eye and trying to understand all this with no historical basis to pull from, I do believe there is a massive struggle in the G data structure causing an overall re evaluation of the WWW.

[tedster]

Let's hold this present discussion to specific observations and analysis of the current SERPs, and especially how they continue to evolve as time goes on.

We have a dedicated thread for Mayday changes [webmasterworld.com]. We also have two more general discussions going here:

Is History Repeating Itself at Google? [webmasterworld.com]
Google's Current Incarnation - how "cute little G" has changed [webmasterworld.com]

These four threads are now tending to blur and overlap. With everyone's cooperation, this monthly update thread can keep its specific focus and value.

The reappearance of traffic in some cases is a hopeful sign. The big brand I mentioned earlier is not seeing any of that, at the moment, but it's good to hear that some of it is happening for some sites.

[Lapizuli]

Warning: Wild speculation ahead...

Doesn't Google do a major "June shuffle" during this time of year? I've been writing online content for less than 2 years, but we sort of noticed this last year around the time college let out. I thought I read they hire college students to do stuff to search results. If that's true, then now would be the time - beginning mid May to early June, actually. Last year, SERPS got very weird - even weirder than usual - and then they ironed out, I don't recall when, maybe mid August?

So I'm thinking this is apart from the Mayday thing. And I don't really know what the shuffle is, or if it's just a phantom in our minds....Unfortunately, my traffic & revenue data from last year showed numbers too low to mean anything. But maybe someone else noticed it...?

Or maybe this is how crazy Internet rumors get started...

[scottsonline]

@tedster sounds good.

Back to the OP's point. The sites we know are doing well are running multiple sites offering the same exact products. In some cases 15+ sites on different www's but using the same chat software and addresses so it's pretty simple to identify. Some of those same sites are running DHTML viewers that list basically every product in a category at the top of each page. That really seems to be helping them keep things moving.

In short all the old tricks really seem to work well in the new Google :(

[aspdesigner]

I just got done reading ALL of Parts 1 - 3 of the Florida Update thread! (all 688 pages and 2,301 comments!) As well as part of another thread that actually made some strides determining what Florida was doing. There is still more reading to do, but I need to give my brain cells a rest!

While some of the details are more relevant to the other thread, I will respond to some of the points made above.

@tedster: Florida caused an upswelling of almost historic proportions - even national media attention - not just some annual event.

Nor is what Google has done over the last 7 years relevant to the validity of the Florida approach, as under intense pressure from users, SEOs, and even the media, it was reversed in early 2004.

"Google will notice that dissatisfaction and take steps to change it."

You mean like they did in 2003?

"We do need to face the fact that the search results are going to cater to the masses first and foremost."

But those are the ppl that are hurt the most by junk SERPs - the users!

"Overriding quote marks and plus signs IS a problem, IMO - but my family doesn't notice it."

Expecting the general public to have to apply advanced search features, just to be able to find what they are looking for, is a guaranteed FAIL. Microsoft's advert. dept. would have a field day with that!

But the idea that this radical change in approach is easily fixable (the current SERPs seem to indicate otherwise), or that Google does not like things just the way they are (like what they appear to claim in their video and in 2003), is IMHO simply wishful thinking.

But from what I was reading about Florida, my main concern is that this overly broad, less relevant, SEO-penalizing SERPs that may result from this basic approach, while I am sure will due wonders for AdWords sales, may prove useless for either SEOs or their business clients, let alone the poor user trying to actually find something specific in the organic search results!

"We're up to almost 15% bing traffic today on very heavy traffic. Yahoo is up as well."

scottsonline: Given how far behind Yahoo & Bing are (compared to the normal Google) in terms of search quality, the fact that users are STILL turning away from Google to use them instead is telling!

Same thing happened with our sites in 2003, but I don't think this soon...could the current SERPs actually be worse than Florida was?

[dickbaker]

Today, before I uploaded a new page for Acme Widgets with links to all of the different models, I did some checking on what's ranking well and what's not.

Years ago I'd create such a page and list each model beginning with the name "Acme." (i.e., "Acme model 123", "Acme model 234"). At some point it was seeming like that was keyword stuffing, so I only did h2 tags for categories of models with the name Acme in them, and didn't include it in the model names.

Well, all of the top-ranking pages for various searches, including my good-ranking pages, have the brand name in each link. In some cases the brand name or key phrase is repeated a hundred times on a page.

The pages that I have that don't have the brand name in each link aren't ranking as well.

So much for trying to have things read naturally.

[scottsonline]

@dickbaker exactly. Our competitors are listing 100 products as "related" per page to jam the brand name in there and Google seems clueless that this is not user friendly and is really just a new way to keyword stuff. 3-10 products are related, 75 is just keyword jamming.

@aspd the day hasn't ended yet but comparing today to the same day last month (which works well for our business), google is down 9%, Bing up 40%, Yahoo up 80%. Proportionally Bing is far larger than Yahoo but the numbers have never, ever been seen before by us.

It's a quantum shift that started towards the end of April. 3 months ago non google search engines were under 10% of our daily volume which includes directs. It will likely end up near 25% today. That's staggering to me and tells me our user base in the corporate world is finding the same stuff we are finding here.

[steerpikegg]

People seeing shifts towards Bing, are any of you in the UK? I'm ranking well in Bing, but see little traffic in the UK. Google is still a massive majority.

[scottsonline]

@steerpikegg we're in the US.

Compared to the same day last year we've seen an 8 to 9x increase with Bing over Pre-Bing/MSN. Yahoo is down slightly, Google down 30%

Compared to the same day last month as of right now google is down 20%, Bing up 22%, Yahoo about flat.

Proportionally speaking with CPC and Direct traffic removed the percentage jump of Bing in particular since April is staggering on our corporate B2B site. I doubt the powers that be at Google will see this for weeks or months to come because its very likely that people are still "trying" to search via Google before giving up and going to the other SERPs so their "share" will not change much. It's the foot soliders/web masters that will see the change first.

The three most disturbing things about the last 2-4 months for me:

1. The ease at which people are scamming Google with multiple sites that drive different keyword combinations on each site. Cheap Widget A on site A, Inexpensive Widget A on site B etc etc.
2. The slow indexing many of us are seeing and random indexing. PR5 and 6 sites are taking weeks to see pages linked from the homepage picked up.
3. The change in structure that prevents direct searches from returning direct results. If someone wants to name their page Custom Designed Colored Widget B and that's exactly what I search for even if I'm the only one that searches for it that week that's the page I should get all other things being equal. Not Custom Designed Colors in Widget, UK.

[Robert Charlton]

The three most disturbing things about the last 2-4 months for me

Thinking out loud here, and I'm not a search engineer... we can either attribute these three things to the new algo, or to the transition to Caffeine, or to a combination of these. Presumably, the new algo anticipated some of Caffeine's capabilities and possible vulnerabilities, but you never know. ;)

I can see where the slow and random indexing might be due to a limitation of resources as Google migrated from one file system and infrastructure to another. Conceivably, old data had to be cached until new data migrated to the new system. This might also explain lack of direct results. If this is the case, with the announcement of Caffeine we should be seeing a more complete and fresher index soon.

The proliferation of dupes and mulitiple site spam might be due to cached data coexisting with new data... or it might be a vulnerability of a system and/or an algo that incorporates super-fresh data. Presumably, someone anticipated this as well, but perhaps not.

I doubt that any of these behaviors was what Google desired, but they may have been what Google felt they could live with during the transition. I'm not talking about the changes that are clearly algo changes... just about these last three particular things that scottsonline has noted.

[drall]

I just did a webmaster/google related 3 word search that is a VERY common problem. I was absolutely blown away to see the results.

1st result was a spammy article from Denmark dated 2003

2nd - 9th result all offered a combination of old blogs and mash style sites and one scraper style site with almost no useful content

10th result was WebmasterWorld with mini results, the main WebmasterWorld result and the mini results without question are more on target and useful then 1-9 or any on page 2-3.

Not a snowballs chance in hell of results 1-9 being more related then WebmasterWorld. I fully expected WebmasterWorld to be result 1 having done thousands of these types of searches in the past.

Whatever they are changing, they need to seriously eval it again. I mean this wasnt a longtail obscure phrase. It was a very common problem most of us have dealt with. To not see WebmasterWorld result 1 or even ATF is disturbing to say the least.

[scottsonline]

@robert charlton/drall

I just did a search for "promotional widget" for kicks. Widget is a term for a product in this industry with roughly 50-100k searches a month. The first two results are the same company. Position 1 is the regular company. Position 2 is an affiliate/duplicate site but they don't try to hide it as they provide a huge link at the bottom of the page saying "we are result #1" also. Result 3 is an affiliate - and they make this very clear all over the page for a major distributor in the industry. Result 4 is a company with FAQs in their footer that jams the footer with the keywords over and over. 2 or 3 results in the top 10 are from real companies, not affiliates etc.

When I ran this search in November of 2009 it returned 10 unique companies in the top 10 spots. We were not one of them, but they were legitimate competitors.

Using the same widget term and firing in "custom widget" I get a #1 result for a term with 100k searches that has the word "custom widget" 30 times on the homepage. Really? This is quality?

[pexcornel]

I was somehow affected last Friday and what I see now is a problem with the canonical "www" VS "non www"

If I search for site:mysite.com, the sites that were affected also return a "non www" version and appear in Google further down the field in the "non www" version.

The sites that were not affected are in normal "www" version.

I haven't made any changes so it's somewhat strange to see this behavior since until last week all was OK in terms of canonical. (I have a .htaccess 301 redirect from "non www" to "www"

Maybe some of you can check this out and reply if you notice something.
(Or maybe this is a consequence rather than a reason...)

Cheers!

[Robert Charlton]

To get some observations in sync, one thing jumps out at me when I compare some observations on this thread and on the Is History Repeating Itself at Google? thread [webmasterworld.com], is that we're seeing keyword stuffing in some pages returned and the absence of matching keywords on others.

On the one hand...

Our competitors are listing 100 products as "related" per page to jam the brand name in there and Google seems clueless that this is not user friendly and is really just a new way to keyword stuff.

I get a #1 result for a term with 100k searches that has the word "custom widget" 30 times on the homepage.

And on the other...

If a site was "too close" of a match to what was searched for, it was kicked down in the SERPs.

And what I'm seeing at both ends of the spectrum...

I'm seeing some pages/sites that I would have thought were over-optimized, at least onpage, still hanging in at #1, and other pages/sites ranking for purely semantic associations (ie, co-occurrence) for one or more of the keywords, with hardly any keyword matches at all. Lots of synonym rankings too, particularly in long tail.

I'll probably elaborate on my thoughts about these algo factors in the History Repeating thread... will update here if we decide that the MayDay thread is more appropriate.

[mercury77]

I didn't see this pointed out by anyone yesterday...
[googlewebmastercentral.blogspot.com...]
"Today, we're announcing the completion of a new web indexing system called Caffeine."

[Robert Charlton]

mercury77 - Note current discussion...

Google announces Caffeine is completed
http://www.webmasterworld.com/google/4149255.htm
[webmasterworld.com...]

[Hissingsid]

@Scottsonline 1. The ease at which people are scamming Google with multiple sites that drive different keyword combinations on each site. Cheap Widget A on site A, Inexpensive Widget A on site B etc etc.

In the supply of real products and services particularly Google is driving the proliferation of sites and pages.

The problem is there's only one #1 slot, there's only one first page. Everyone wants to be at least on page one and then when you get there everyone wants to be #1. In my niche there are 50 real suppliers about 20 sites that try to get between Google and the real suppliers and hundreds of thousands of other pages that mention the product/service.

There seems to be three main strategies used by the 50 real suppliers.

1. Go with the sites that are getting between Google and you, ie give up on trying to rank top.

2. Try to get as many of those hundreds of thousands of pages to point to your site.

3. Make new sites and new pages to either point back to your main site or to rank for a particular term.

Or a combination of these.

This is only going to get worse. As sites designed to get between Google and real suppliers proliferate the real suppliers will have to do more to compete for the top slots.

I don't think that this is a function of the algo change or the more to caffeine it is simply coincidental.

When you think about it the number of real suppliers is only growing slowly. Google is causing the exponential growth of all of the crap making it ever harder for Google itself to find sites that offer what people really want.

Cheers

Sid

[Martin Ice Web]

Another big lost in traffic this night -30% in germany, maybe its due to the fact that caffeine is fully installed?

I´ll will do some sit and wait until the goog engineers smoked their joint and stopped laughing at us cause of all our guesses and thoughts, while the scapers bring their coints with barrows to the next bank.

[backdraft7]

From Ggogle >

With Caffeine, we analyze the web in small portions and update our search index on a continuous basis, globally. As we find new pages, or new information on existing pages, we can add these straight to the index. That means you can find fresher information than ever before — no matter when or where it was published.

This just sounds like an invitation for the over eager scammers to get their foot in the door using a new bag of black hat hat tricks. Even with the best filters, we should expect to see more garbage appear on a more frequent basis. That is frightening. Sounds like trust is out the window in favor of fresh new content. Expect your site to be scraped and shoved down your throat by the next unscrupulous webmaster. Now that they know Google's new indexing scheme, the faster refresh will encourage this bad behavior.
Buckle up again!

[ohno]

Yep i dispair. It really looks like they are happy with this so business has changed forever until the UK starts using other search engines. Adwords-never again, total waste of £££ for our niche.

[sean22]

Re: site that lost long tail traffic in April 28th

This past Saturday (June 5th) I reported that my traffic has returned. It has since been 4 complete days since then and the recovery seems stable. Today is the 5th day and although it is still very early, all trends indicate that I will have another classic pre-April 28th traffic day.

I have done nothing. No link building since April 28th and added minimal content.

This just sounds like an invitation for the over eager scammers to get their foot in the door using a new bag of black hat hat tricks. Even with the best filters, we should expect to see more garbage appear on a more frequent basis. That is frightening. Sounds like trust is out the window in favor of fresh new content. Expect your site to be scraped and shoved down your throat by the next unscrupulous webmaster. Now that they know Google's new indexing scheme, the faster refresh will encourage this bad behavior.
Buckle up again!

Maybe not so because I'm sure Google won't make fresh content from untrusted sites appear. I think trust will always be there, just more even and fair. Encouragement for older sites to freshen up because the status quo has changed.

[londrum]

they can't be ranking everything on how fresh it is because it wouldn't work.

i want my newspapers fresh every five minutes, but i don't want a music shop that only sells the charts. and no one's going to like a bookshop that doesn't stock the classics.

the vast majority of information sites must be immune from this freshness stuff, surely.

imagine if you search for a poem, or a quote from a book. what does it matter how fresh it is?

[backdraft7]

This new algo still does not address the issue of duplicate sites. It pretty easy to find these guys when the have adsense ads using the same pub-id on two separate domains that are listed in the #2 and #3 position on page 1. Both sites are hawking the exact same product using re-hash from the main site. This takes up two positions for the same site owner and violates the GWMGL's. I wonder if we'll ever see any improvement with regard to removal of dup sites.

[sean22]

they can't be ranking everything on how fresh it is because it wouldn't work.

i want my newspapers fresh every five minutes, but i don't want a music shop that only sells the charts. and no one's going to like a bookshop that doesn't stock the classics.

the vast majority of information sites must be immune from this freshness stuff, surely.

imagine if you search for a poem, or a quote from a book. what does it matter how fresh it is?

I agree londrum, for evergreen trophy KW maybe the rankings won't change. But for long tail KWs it may become based on freshness. Here is a simple example.

Say the trophy KW is Widgets.

Google's may treat the ranking for Widgets one way but not for say Green Shiny Widgets. Why?

Well content about Best Green Shiny Widgets For 2010 might be looked upon as fresh thus favorable, and could very well out rank older content targeting the same KW.

[londrum]

but what about these examples:
"elvis", "elvis presley" and "elvis presley suspicious minds".
what does it matter how fresh each answer is? its got absolutely nothing to do with it.

practically every single search term you can imagine for elvis, whether it is a two word term or longtail, derives no benefit from weighting in freshness.
the only time that freshness helps is when you're searching for some kind of elvis CD, book or TV programme -- something that you want to buy or watch.