3: Google Updates and SERP Changes - May 2010

[tyler756]

< continued from [webmasterworld.com...] >

I have been attempting to follow this thread to the best of my ability but must admit there is information in here that is a little out of my league.

Has anyone been able to do anything to help increase rankings? I lost rankings for 3 my keywords (I am not ranking for a ton of them yet) and these 3 drive a lot of my traffic. I dropped down from the 1 spot to position 5 and 6 for two of the keywords, and from the 3rd spot to the third page for the other major keyword. I am not sure what to do and we are about to enter the "slow months" for my industry and my traffic is dropping every day.

Just thought I would post to see if anyone is having any luck

[edited by: tedster at 7:41 pm (utc) on May 22, 2010]

[tedster]

OK - lots of horror stories this May. One thing that strikes me as important is that only some of the members have shared any details about what kind of searches they lost traffic on.

If anyone is in this "I lost lots of traffic" boat and they don't have the data analysis programs installed that they need to dig deep - or they don't know how to use the programs they have - this should be a wake-up call to get that happening. This month we see that evening being scrupulous about Google's guidelines is no guarantee that even long standing rankings won't vanish for you.

How closely do you look at the keywords in your referrers? Do you note the number of words in each? Do you bucket that traffic by various types, such as navigational or informational? Do you know which keywords are "just traffic" and which bring you revenue? Do you know what googlebot's crawling pattern looks like for your site? Cn you tell the difference between a real googlebot request and a rogue bot that's just spoofing a googlebot user-agent?

There are ways to do all this, and more - and no they don't necessarily require Google Analytics. If you can't do in-depth analysis and your revenues are taking a dive, I highly recommend studying up and taking some intelligent steps.

Our Website Analytics [webmasterworld.com] forum is a good place to start.
The Analytics Library [webmasterworld.com] will give you an overview, and
Which Stats Package is right for you? [webmasterworld.com] is a good thread for getting practical.

With more detailed information at hand, you can contribute more in-depth information to discussions like this one and be part of finding some answers.

If I had to guess at a critical change in this algo change right now, I'd guess that backlinks are being scored in a whole new way. I think lots of backlink credit has been wiped out, especially the kind that any webmaster can just dummy up on their own - as opposed to the kind that takes another website thinking your site is worth recommending to their own visitors.

[tkik]

@walkman, tedster:

i posted that before, here is a link to a live blog [themechanism.com...]

via twitter i knew that the update was already done and is finished. Dont remember the twitter account, you might have to search for "#io2010 matt cutts" or something like that

[mrez74]

Here's what I've noticed lately with my sites:

I have noticed two sets of SERPs for me in the past 2 weeks. 1 set shows up about 80% of the time (and has been up all the time for the last week) and then 2nd set that shows up maybe for 1 day or so every like 5-7 days.

What I have noticed is that when the 2nd set SERP shows up several of my sites disappear from the page 1 for their main keyword(can't be found past the first 10 pages) while the sites that are on page 2 on the 1st SERP set show up in the top 3 of the on the first page. This is for like 5-6 sites. Then for the few sites that have been around for like 2 years or so they remain stable and are at the same position whether SERP 1 or SERP2. This movement back and forth are to sites that are about 1 or less years up.

It's funny though that when I see a group of sites reappear on the first page for their main keyword I know that the other set of sites are gone from the front page (and when I do a query for their main keyword of course that is the case).

But over the past 2 weeks the 1st SERP is there most of the time. Just today I'm getting the SERP 2 set. The problem is my traffic evens out since I lose the ones from SERP1, but gain those from SERP2. I wish the 2 sets of SERPs can be one so I can see my whole traffic.

[walkman]

If I had to guess at a critical change in this algo change right now, I'd guess that backlinks are being scored in a whole new way. I think lots of backlink credit has been wiped out, especially the kind that any webmaster can just dummy up on their own - as opposed to the kind that takes another website thinking your site is worth recommending to their own visitors.

I can report the exact opposite tedster, and I'm not dramatizing. My main bread and butter site is the one I'm very careful with links, especially from my own sites. That site has many links from sites that are valued by Google (they have plenty of varied backlinks, largely on my topic, fresh tags, good traffic) yet it lost traffic. That site also has many links from Twitter, FB and others socials.

Other sites that have ROS from my other sites and many lazy links saw an increase on indexed pages and traffic.

Most importantly I'm baffled at the inconsistency of Google results and referrals I get, there's no way it's a daily flux thing given the huge discrepancy. If anyone thinks that Caff is here, they are wrong. First Google would have announced it, and second, this [209.85.225.103...] the original caff, is different from any Google.com I have searched on manually. Caff shows on most IP searches via SEO tools but apparently they aren't getting much play time on Google.com

On Edit: maybe, just maybe Google decided that I got to many links too fast and placed the 'new' ones all on hold. That's the only logical thing I can think of. The link rate was faster than normal but that's because I got involved with social nets and many links flowed.

[tedster]

Well, walkman, I said it was a guess :(

Truth is I'm a bit in the dark here, too -- mostly because I don't have a site that's run into trouble. So what I said was based on a very small sample of websites that were shared with me, and I ought to know better than to do that.

< edited for spelling >

[edited by: tedster at 4:34 am (utc) on May 23, 2010]

[arizonadude]

First Google would have announced it, and second, this [209.85.225.103...] the original caff, is different from any Google.com I have searched on manually.

I've been seeing the exact results you see on the ip you gave over 90% of the time for the last 3 weeks anytime I search on Google.

I do not use personalized search and get those results whether I use private browsing or not.

Not sure why you or others are not seeing those results more than others but from AZ those results have been in the wild for quite some time now.

Also want to add, I've got several very deep sites with a lot of pages and numerous niche focused sites with less than a 100 pages and none of the large sites have been effected and only a few of the newer smaller sites have been hit. Over all, I've seen a very large traffic increase since caf has started rolling out over 3 weeks ago.

[walkman]

Well, walkman, I said it was a guess

:) Mine's a guess too, no real way of knowing. You maybe right, we both might be, in a way. As I keep repeating I get great traffic (in spurts), too bad it does not last, so I have hope.

I'll just tidy something here and there on my site and that's about it.

[Plan_D]

I'm finding that pages that used to rank on the first page have actually disappeared from the index. These have been replaced by pages further down in that section.

For example if I have a long article running to 5 pages, page 1 used to rank very well for it's key words. But has now disappeared so I can now only find page 5 (this is now ranked 200+ in the serps though). This is because all the in-site links are to page 1 and probably any links from outside as well. Pages 2 to 5 also link to page 1.

I can't see any logical reason why google would rather index page 5 than page 1. So I am hoping that page 1 will reappear at some point soon.

[tessmac]

One thing that I have picked up upon in this thread is a recurring theme of "Old Results" showing.

I also check my rankings daily on a few keywords, and while it is difficult to remember exactly results from 6 months ago, it does have the look of deja vu.

There also seems to be two different issues repoted here:

1. Rankings the same, but traffic down...this I think should be a seperate issue.
2. Rankings greatly diminished....which I think is the issue at hand.

My D Day if you like was on the 5th of May. We had top ten rankings for virtually every keyword we target. Now while our site is 8 years old, we only started seriously building links within the last 3 months ( for the sake of argument I'm going to assume we have not got a penalty of some sort )

So on the 5th of May we dropped off the map literally. We didnt even rank for "my-domain" without the www & .com. We did however rank for www.mydomain.com.

After about 1 week we started to rank again for "mydomain".

We started to find minor pages in the site outranking highly optimised pages for a keyword.

We also started to find https pages that had somehow been crawled and were outranking the http equivalent ( the http pages were backlinked )

2 weeks on and we have some rankings in the .com version of G, not great, but better than it was. Now traditionally we see rankings improve in G.com, which then seem to get updated in G.co.uk ( the web ) then finally filter over to G.co.uk. This does not seem to be happening now.

For example I have tracked serps from G.com, making a note of who was above us and who below, for a given keyword. Then I have gone back to G.co.uk ( the web ) and the serps are accurate except for where our site should be ( we are pushed way lower, with a different url to the higher one in G.com )

The only difference for our site is that 95% of our link building has been done in the last 3 months, whereas our competitors have been doing it for a lot longer.

To me it looks like everything done in the last 3/6 months has been discounted, which would explain the above. I dont think I'm alone either as there have been similar comments on this thread.

Dont know if any of this helps, and I know its a bit of a ramble, but can anyone see a pattern here.

[aristotle]

I found some recent comments about SEO by Matt Cutts and other Google employees at

[labnol.org ]

They talk about page loading speed, duplicate content problems, paid links, length of domain registration, and various other SEO issues.

[walkman]

To me it looks like everything done in the last 3/6 months has been discounted, which would explain the above. I dont think I'm alone either as there have been similar comments on this thread.

I have said the same thing. If you read many comments you will that many talk about 'links not being counted yet' or 'trust still being held back until re-evaluated.' A lot of people have the same issues. I rank for my site name, get fresh cache and everything. Basically I rank at -6 months level

Maybe Google decided to to apply a sandbox to new links at higher than normal rate? Who knows, but the version where all links are counted exists. I know because I see the very good referrals.

[BillyS]

If we lost traffic (hard to tell due to seasonality), it was around 10%. But one thing I noticed is hourly spikes in traffic. In the past our traffic would grow slowly over the course of a day then start to decline. Over the last two weeks I see spikes that exceed our week over week "norm."

I'm going to see if I can look at those hours, I don't think it's possible with the tools I have...

[tessmac]

but the version where all links are counted exists. I know because I see the very good referrals.

What does that mean?

Do you mean you are seeing better rankings from different DC's?

[anand84]

I conducted a minor study for my website. What I did was go through Google SERPS for several of my page keywords. My website caters to several different topics and so a particular keyword is only relevant to a particular page. I checked this for several pages right from something I had posted yesterday to those written back in Nov '09..So if the title of my page was 'Number of Red widgets in USA', I would search for 'red widgets', 'red widgets in united states', 'red widgets number',etc. and see where my articles ranked. Here are some observations I made.

1) I could not find a pattern..I was able to rank on top for 'red widgets' itself for some pages and sometimes ranked only if I searched the entire title - no problem here since I guess it is based on the competitiveness of the keywords alone.

2) I tried to see if the rankings are affected either for recent or old articles. Pages written since February are ranking very well. And while pages before that have been ranking too, there were a lot more times when my page didnt rank. Interestingly, all articles written on Feb 1 and 2 were ranking well. Jan 28,29,20 did not rank at all - while I must say I still could not find a pattern, it is interesting that there was a clear demarcation at this point.

3) Connecting words (eg: in, at, on, among) seem to be having a lot more relevance now. There were quite a few instances when I could not find my page for 'red widgets usa' but ranked within the first two pages for 'red widgets in usa'.

4) I could not confirm this point, but here is an observation : Google recognizes that "USA" is the same as "United States" (since searching for "united states" also boldens USA). But if your page does not contain "United States" explicitly, your page is not displayed when searching for 'red widgets united states'..

I have lost close to 50% of traffic and the numbers are marginally falling every week. Despite this, the pages have ranked moderately well. So, what I presume is that Google is now giving a lot of emphasis to keywords in titles or URL. I can only guess that my website used to rank for keywords not inside title/url and possibly this has fallen now. Otherwise, I cannot understand why my site should have dropped 50% of traffic now.

Has anybody else find a drop in non-title keywords? Also, have you observed changes that I noted in point 3?

[tedster]

Interestingly, all articles written on Feb 1 and 2 were ranking well. Jan 28,29,20 did not rank at all - while I must say I still could not find a pattern, it is interesting that there was a clear demarcation at this point.

Thanks for that - I'm going to take a closer look for the same.

Connecting words (eg: in, at, on, among) seem to be having a lot more relevance now

Yes, I am definitely seeing that pattern. Earlier in this Mayday discussion [webmasterworld.com] I made an educated guess that something had changed in the phrase-based indexing algorithm, and I based that guess on this kind of observation.

Suppose you search on a 4-word phrase (a 4-gram) - word1 word2 word3 word4. If a page on a strong site contains word1-word2-word3 in one group and word4 appears elsewhere on the page, it can still outrank pages with the full phrase on "weaker" sites, but not as easily as before.

This pattern seems to include "connecting words" more often - words than might previously have caused the full phrase to be ignored as "a good phrase". The observation points to a shift that is quite complex in the indexing/ranking logic. Maybe something that required more computing power to bring live?

It certainly has shaken up some long-tail traffic patterns. Other reports in this thread show both directions of long-tail change, significantly up and significantly down.

Interestingly, very large sites with lots of UGC seem to be the ones that went down, (admittedly this is based on a very small sample). I wonder if that drop was because pages with the exact, full phrase now get "more credit", even when other signals are relatively weak.

[pontifex]

Tedster: that makes perfect sense and relates to the patents you mentioned, too. Giving weight to sentences or "snippets" of sentences that give a pretty good idea of the intention of the user would also explain the long tail change.

It also means that pages with more sentences would get more traffic - strange enough: this favors articles for SEO reasons over casual content and explains the rise of spammy sites, too...

"red widgets for something"

would outrank

"red widgets that you can buy and own for something"

if all other aspects of both pages are roughly the same (age, link juice, etc.) and I just "like" that theory much more than the AI approach, because it is simpler and as I know Google, they want to achieve a lot with little effort :-)

So, besides their database of "common keywords", which might covered around 3 Mio. terms in English, they pumped it up to 30-40 Mio. terms and try to match that better...

As always: just wild guessing, but from my monitoring, I got 1.8 Mio. different phrases from live searches over the time in German. More people search in English - so lets just round it up and say, Google was pre-calculating with 3 Mio. phrases, needing special attention, because they are typed in more than twice in a month.

Now (and they do that, because we know Google Trends, don't we) they have recorded all search terms over the years and thought: well, 3 Mio. is not good enough, lets add 2 or 3 words to all these queries that we store and calculate relevance for those phrases for our main index.

I like the theory, it makes sense and it explains what I am seeing in the long tail! It would also enhance Google Trends, which I will take a look at now :-)

P!

[sGroup]

A site we manage dropped from #2/3 to #11/12 for a few top KW's that we mainly build inbound links for.

Our site is a 10+ year old site with plenty of inbound links and basically dropped about 10 spots. The rest of the sites that were there before are still there. The drop happened about 2-3 weeks ago.

Are others here also noticing the same type of rank drops for KW's that they have been building ongoing links for?

[aristotle]

tedster said:

This pattern seems to include "connecting words" more often - words than might previously have caused the full phrase to be ignored as "a good phrase". The observation points to a shift that is quite complex in the indexing/ranking logic. Maybe something that required more computing power to bring live?

If a big increase in computing power was required to bring this live, couldn't this be an indication that the Caffeine rollout is finished?

[walkman]

" A site we manage dropped from #2/3 to #11/12 for a few top KW's that we mainly build inbound links for. "

Were you #2/3 last year or did you build that with the new links this year?

[Sgt_Kickaxe]

I took a beating by a site that scrapes content from craigslist and serves it up on their main site, 2 sub-domains and an entirely different domain that has a one letter difference. Thats FOUR new pages ahead of mine, all the same scraped content. Not impressed.

[tedster]

...couldn't this be an indication that the Caffeine rollout is finished?

Maybe not completely finished, but at least being served a large proportion of users. However, this idea about a change in phrase-based indexing is not anything that's confirmed at all - it's just a working hypothesis for me.

[edited by: tedster at 11:00 pm (utc) on May 23, 2010]

[Sgt_Kickaxe]

Is it fashionable to repeat content on sub-domains again? I'm seeing some new spam on quite a few keywords. A Google search for something like "[item name] for sale" shows a site displaying the same expired scraped craigslist content on 3 different subdomains as well as on a completely separate domain, and all are in the top 20.

It's so blatant two of the sub-domains only have switched url keywords for the same pages! (ie: blue-widget and widget-blue). I'm not giving this update a passing grade just yet, it needs work.

[edited by: tedster at 11:02 pm (utc) on May 23, 2010]
[edit reason] no specific search terms, please [/edit]

[Plan_D]

This must be a mistake. The sites that have replaced us are very poor. Surely there has been some catastrophy at google that they cannot tell us about and they are having to go through some kind of emergency procedure.

Otherwise, I understand they are looking for lots of people to teach maths or science...

[imbckagn]

I spent a couple days digging into my niche trying to figure out what is going on. I've come to the conclusion that something is definitely wrong with Google. I don't want to share the SERPs or methods the websites I analyzed are using to rank but it's a joke.

Google has to be relying on an old algorithm and data right now. There is no way these websites would be ranking where they are unless something was seriously wrong.

For example just one keyword phrase that is insanely competitive has 4 different websites that are HORRIBLE created by the same webmaster in the 2,7, 13 and 21 position. This webmaster has managed to move these websites ( 20 pages of crap max ) into just about all the competitive phrases I watch.

If Google doesn't fix what ever is wrong soon the SERP's will be infested with websites like these. Hell at this point I am considering pumping these sites out myself.

[sGroup]

" A site we manage dropped from #2/3 to #11/12 for a few top KW's that we mainly build inbound links for. "

Were you #2/3 last year or did you build that with the new links this year?

We were #2/3 for at least the last year or two.

Over the last 6+ years we have held 1st page positions anywhere from #2 to #5. What is odd is that we have now moved to 2nd page holding positions #11/12.

[ohno]

Ok, weekend traffic was poor but understandable considering the weather! Seems dead today & Googlebot paid us a visit early this morning & crawled a LOAD of pages(normally seems to crawl one page at a time?).

Edited to add-just gone back further through the logs and Gbot has been all over us over the weekend.

[steerpikegg]

@Ohno

Exactly the same here - quiet weekend (understandable), poor Monday so far and mad Googlebot over the weekend.

[walkman]

Slower than normal Monday, actually probably even slower than yesterday. Googlebot is always active, today's no exception

[mark_roach]

anand84 said:

3) Connecting words (eg: in, at, on, among) seem to be having a lot more relevance now. There were quite a few instances when I could not find my page for 'red widgets usa' but ranked within the first two pages for 'red widgets in usa'.

I ran a few searches comparing the results of

red/blue/green widgets location and red/blue/green widgets in location.

The 1st page results were broadly the same with only some minor shuffling of the order, however I noticed something strange which might be relevant.

In my niche you can either typically search for widgets or "widget suppliers". The two words are only loosely related, "widget suppliers" is a generic single word not containing the word widget.

On a search for red widgets location the words red, widgets & location are all bolded in the results as expected.

On a search for red widgets in location the words red, widgets & location are still all bolded, but in addition "widget suppliers" is also bolded.

I tried these 2 searches for 6 different "colours" and 4 of the 6 exhibited the behaviour above. In the other 2 searches "widget suppliers" was not bolded.

I repeated the 6 searches for colour "widget suppliers" in location and the word widget was not highlighted in any of the result sets.

I don't know if the highlighting is significant because the results didn't alter much. However it might be an indicator of Google trying to guess what you are looking for. If you are searching for "widget in location" then you might be looking to buy that widget and therefore are interested in widget suppliers.

[ohno]

@steerpikegg, do you normally see so much Gbot activity? I can't recall seeing it like this before but i am checking the logs more at the moment!