Google Updates and SERP Changes - March 2010

[drall]

< continued from [webmasterworld.com...] >

I have been watching all of my competitors which pretty much live off of large amounts of longtail traffic loose overall traffic the past 4-6 weeks and that includes us.

I dont know whats going on but for big sites in my section of the web site page count has been dropping at a constant decay and new content is coming in very very slowly and not nearly enough to counter the drops.

Overall traffic seems down about 8-10% for all of us and those drops are graphing perfectly to the drops in page counts since mid January.

At the same time the crawl graph in WMT and my local stats package show a flatline since mid Jan with no deep crawls and about a tenth of usual crawl activity.

All of this for us and our competitors started mid January
1. dropoff of the googlebot deepcrawl with a flatline stable crawl at a much lower rate.
2. daily reductions in pagecount across a dozen sites by exact same percentages
3. similar gradual traffic declines matching page count falls
4. uber slow new page caching and ranking

When I see this happening to not only us but many pr6-8 sites in my sector it gives me comfort to see that it isnt only us but also concern because it isnt only us.

Just going to keep chugging along, not much you can do but sit back and watch this stuff unfold.

[edited by: tedster at 5:49 pm (utc) on Mar 1, 2010]
[edit reason] split from earlier thread [/edit]

[brinked]

I am noticing slightly different SERP's on california proxies. It's really weird. They are no where like the caffeine serps, they are more like the regular SERP's I see now but 1 or 2 sites will be shuffled around. It feels like they are outdated. The only logical guess I can make of this is that maybe they are trying to show SERP's based on california search trends?

WN, have you noticed anything similar to this in your findings?

[whitenight]

How can we recognize indications of subtle spyware?

Good question vstevens.

Well the first issue was i was getting Caff on google.com live.
And then once i got it, i couldn't NOT get it.
Obviously i know how to clear everything and then baseballguy/dude (i think) reported the same thing.

Check with various other comps in the immediate area.
Same thing. Once gotten, impossible to "un-get".
Then checked with my associates in different states.
Same thing.

And exact identical SERPs EVERY SINGLE TIME?!
Don't think so, Gorg!

It was like i had specially formatted my personalization results to ONLY include those Caff datasets in the exact same SERP order.

See the past updates threads to see how i was pinpointing the targeted groups before the public announcement of Caff's failure and supposedly "starting over"

Also, what was caffeine showing last time out of the box when it wad obvious?

Don't quite understand this question.

It's obvious to me everytime i see it as i have "control" keywords/SERPs that are radically different from "regular" SERPS
and yet nearly exactly the same as the SERPs that were on sandbox.gorg 9 months ago.

(Yes, with the updated link/PR/etc calculations included in those datasets)

This is not even including the control keywords that would only receive the traffic they are getting if Caff datasets weren't live "somewhere"

Ie. SERP #50 on regular SERPs = no traffic
SERP #3 on Caff dataset = tons of traffic

[whitenight]

They are no where like the caffeine serps, they are more like the regular SERP's I see now but 1 or 2 sites will be shuffled around WN, have you noticed anything similar to this in your findings?

Yes, this is pretty normal of the "variations of the variations" datasets that appear and then disappear/get folded in, during major updates.

Most time, they are minor "tweaks" of the "200 variables" that need to be specifically added and only of minor interest.

Once in a blue moon, you'll see the "Ghost Datasets" (if Gorg can't sneak them in under the radar) that are obviously different and should be screenshot-ed, photocopied, and kept under lock and key for reasons that will become apparent to someone like you.

[louieramos]

brinked, I am seeing 4 sets of different results in all US Google DC using a rank checker software that I have..

[slinky101]

All affected sites are in different niches and as such very little or no interlinking. All sites had different standing before the trouble: some had great traffic, some not so much. Some had many incoming links, some less, some had new links added recently, some had not. Sites are evenly distributed between different servers on different IPs. Most have dedicated IPs. A couple had server issues and were down for more than 24 hours just couple days ago but most run perfectly for years. All built on the same CMS (not WP) but templates vary widely - a product of many years of disparate development.

Just about the only thing that ties them together (other than most are under the same WMT account, but even then not all are) is that they are all forums.

Am I just feeding the flames of my paranoia or some of you guys can also confirm weaker standing for forums in this new update?

Both my forums have had a big drop in traffic today too. Analytics has us down about 30%. Yet the main keywords remain at No1 spots. Went to a (californian) proxy to check the site: command. Pages are down from 278,000 to 34,000 and the other down from 128,000 to 4,000.

A UK proxy is still showing the 278,000 results.

My other sites ( not forums ).. are as normal. Don't know if this is caffine exactly..but if you're paranoid, then sign me up to the club ! My forums have definately had a big drop today too as opposed to my static sites.

[BillyS]

There is no half caffeine because it does not exist. Google is only releasing caffeine. Google is not going to release 2 versions of caffeine,

I am noticing slightly different SERP's on california proxies. It's really weird.

This does not compute, someone please explain what seems to be a series of inconsistent statements to my feeble mind...

[whitenight]

This does not compute, someone please explain

The semantics are VERY important here.

Just because one is seeing a 3rd dataset doesn't mean it's an any way related to Caff.

If SOMEONE would read the OCT 08 thread where i painstakingly detailed the rollout of updates,
one would understand WHY it's not Caff or 1/2 Caff
but simply a "3rd dataset"....
and one could see up to 10 (or more) different datasets over the next few days/week, if one is watching closely enough to differentiate.

[tedster]

Those who are seeing big drops in rankings for their forums, are you using common forum software?

[cangoou]

It was like i had specially formatted my personalization results to ONLY include those Caff datasets in the exact same SERP order.

What could be a reason for G to put so much work in such a thing? Just to make sure no-one can analyse the different datasets during the update? Or has this something to do with the target groups you mentioned before?

[Pass the Dutchie]

can anyone tell me if the stored cache of a page has anything to do with when the site's ranking is calculated? Despite recent daily updates our home page cache is now 14 days old.

[Marvin Hlavac]

Those who are seeing big drops in rankings for their forums, are you using common forum software?

... and to add to the Tedster's question, if you've experienced a drop in traffic on your forum, have you upgraded to a new version of forum software recently? A popular forum software recently released a new version, which was a major rewrite.

[Shaddows]

Despite recent daily updates our home page cache is now 14 days old

Very often the case pre- and mid-update. The Cache date does not necessarily imply your ranking is out of date.

You might want to consider the noarchive tag though. Search the site for details, incrediBill being a strong advocate, from memory.
-------------------------

Further to WN, the fact that slightly different data is held on different datacentres necessitates different datasets.

Consider this. On an average day, you can see slightly different results on different DCs. These chatter to each other, keeping them largely aligned.

During an update, due to imperfectly matched data, each DC builds its index in slightly different ways. Small changes in initial conditions mean the index is built in different ways (priority given to differnet data shards). These converge as iteration reduces the significance of the the order of processing, and also as base data becomes more complete.

However, convergence is not absolute. To deal with this, two things happen. First, Google has predefined "important" datapoints, such as particular homepages. These act as data anchors, starting points if you will.

Secondly, there is the inter-DC chatter I mentioned earlier.

Under Caffiene, I'm expecting this folding process to disappear, as physical DCs become irrelevant, and all aspects of the Google world become distributed (and not just the loadbalancing we see today).

Disclaimer: This is a mix of fact, analysis and conjecture, brought together into a single cohesive view for the purpose of clarity. It is also overly simplistic, avoiding certain subtleties. Free forum advice is worth what you pay for it

[cangoou]

Shaddows, thanks for sharing your thoughts.

What I don't understand: Why is it that every DC gets slighty different results? Do they work on different sets of data? Or is just that the algorithm (especially calculating internal PR) is so complex and recursive and works on so much data that the DCs use mathematical approximations which could produce different results? Or does one DC needs less time folding in a datasets than another because of different computer-power or more intelligent workers or something like that?

[jgold454]

Today I noticed a big change in the long tail SERPs for my niche....

[1script]

@slinky101, @tedster, @Marvin Hlavac regarding forum sites:

I'm using a very UNcommon forum software, so definitely no recent software changes would affect me. However, the structure of pages is common for most forums: homepage has categories, each with summary of the most recent post, categories are paginated and are basically just lists of posts. Page 1 lists most recent posts. As posts age, they move to Page 2, 3 etc. Individual posts have links to other posts with related subjects. So, again, very common page/internal linking structure found on most forums.

In addition, most of the sites I've been able to analyze so far look exactly the same as they did in terms of number of incoming links (showed by Y!) , freshness of G cache for the homepage, number of Googlebot visits, searches for domain name etc.
Everything is the same BUT the traffic for most of the keywords that used to rank. Very long tail still brings some traffic but it actually looks like the situation deteriorates by the hour.

Hope someone can post more data about their forums so we can search for some common traits.

[1script]

@Pass the Dutchie:

I don't think homepage's cache freshness matters much. I have a site that has been obliterated yesterday but the cache is still only 4-6 hours old - as it has been during recent months. There is still healthy Googlebot activity and the homepage cache updates VERY quickly (2-3 times a day) but Google traffic plummeted to 1/20th of its volume just 48 hours ago.

To add insult to the injury Google has just added sitelinks for that site. I had a tendency to think that sitelinks are a mark of a site that they think is good enough. Guess, not good enough for ranking...

[SEOPTI]

Today one site I own suddenly went from 100k indexed URLs to 400k indexed URLs and traffic increased, never saw this kind of thing before.

The traffic increased by 10% but not by 300% this again shows how buggy and completely useless the site command is.

[edited by: SEOPTI at 12:06 am (utc) on Mar 17, 2010]

[jonny0000]

< moved from another location >

Since January and the release of Caffeine I've had drops in search traffic reported to internal sections of several sites. There has also been an impact on traffic arriving at internal pages from queries relevant to different sections.

Section related traffic landing on the homepages tends not to be affected. Together this suggests to me there has been a shift in the algorithm in regard to site architecture that is specifically impacting traffic to site sub-sections.

The first thing I would like to is disprove is that the Caffeine spider is not using a unique useragent. Anyone discovered this?

Positions have fluctuated quite a bit but the more established stuff tends to be ok. Thoughts?

[edited by: tedster at 11:39 pm (utc) on Mar 16, 2010]

[trakkerguy]

I'm seeing Caffeine results again in Northern Calif (first time in 3 days) on Google.com, IP 74.125.19.105.

[slinky101]

@1script, @tedster, @Marvin Hlavac regarding forum sites :-

Those who are seeing big drops in rankings for their forums, are you using common forum software?

I'm using IPB for both forums.

and to add to the Tedster's question, if you've experienced a drop in traffic on your forum, have you upgraded to a new version of forum software recently? A popular forum software recently released a new version, which was a major rewrite.

Well that's why it's so weird. One forum was upgraded 3 weeks ago. The other remains on the old 2.3.6 version for now, no changes for about 18 months. Both sites regardless dropped like stones traffic-wise yesterday, and the same today.

Everything is the same BUT the traffic for most of the keywords that used to rank. Very long tail still brings some traffic but it actually looks like the situation deteriorates by the hour.

Same here. Googlebot activity seems healthy, it's on the forum 24/7. Some posts cached and served after 3 or 4 hours, main keywords are just fine both on proxies and 'normal' Google searches.

The only thing that has changed seems to be the pages indexed in some of the proxies. 282k down to 34k for the larger forum. Uk specific proxies still show the 282k.

I also did a search on another large forum I visit frequently and got the same results. Everything as normal, except that some proxies are showing a huge drop in pages indexed. 583k to just 47k on some of them. Uk specific still showing 583k.

I'm no expert in caffine etc, in fact a lot of it goes over my head. I have no idea about datasets, ip's and datacentres rotating results and things like that.
I simply compared normal searches ( non-personalised), with a load of proxy one's. Ranking for main keywords were the same. However, some proxies showed an alarming drop of indexed pages. Thousands and thousands short of what I've come to regard as 'normal' for my forums anyway.

I'm usually a lurker here, but when the traffic dropped massively yesterday and I saw that 1script had reported the same results for his forums. I thought it might be significant to add that I was experiencing the same.

My larger forum is 9 years old, very large and well established in it's niche, backlinks and rankings. There have been no significant changes the last 2 years ( it's the one on the old IPB software ).
The much younger forum has been very recently upgraded, and is still finding it's 'feet' in regards rankings and activity. Like 1script..both dropped significant traffic on the same day at the same time.

What to make of that then ?

[louieramos]

Its happening guys! I am seeing caff results on DC's that were not showing caff results before.. I think its happening now.. more and more DC are getting caffeinated.. fingers crossed

[JoeSinkwitz]

Which blend of hormel do you like the most? As is typical of a lot of updates and churn, some of the datasets you may see promoted are going to look funny due to the sheer amount of sites that rank solely based on comment/guestbook posts.

If you have a cleaner site that got bumped behind these sites, just keep your head down and continue diversifying your link profile. Data looks to be getting folded and despite what was mentioned on MC's blog, I believe I'm seeing caffeine in quite a few geolocations (follow the internet pipes).

[Kelowna]

I am see caff on aol now... :-)

[maximus12]

I am seeing caffeine on all the proxies I watch. A few hours ago it was caffeine on the east coast proxies and non caffeine on the west cost. Now they are all showing caffeine. As louieramos said, let's keep our fingers crossed

[brinked]

nice, seeing caffeine in cali and aol. will check the other in a few and report my findings

[Pass the Dutchie]

big change in SERPs on Google.de Not sure how to check if its Caffeine on non-English Google sites.

[Pass the Dutchie]

Must be switching between old and new datasets as we are now back to yesterdays results.

[cien]

My Vbulletin forum traffic has dropped to a crawl overnight since 3 days ago. This forum has over 100k pages under site:mysite.com. I lost rankings for all keywords overnight. Adsense dropped from $20/day to 0. No, I haven't upgraded vbulletin to 4.0 nor done any updates for months now. Still have 3.8.4

[cangoou]

I see domains dropping again from #1 to nirvana for no reason since 4 days. Even on really small-you-need-only-4-links-to-get-to-the-top-keywords for sites that have been there for years or for keywords like the company-name. I can only explain that by the usual data-chaos G might get during every update. Hope this whole roller coaster will stop soon.

[member22]

what are those datasets that some of you talk about , could you tell me what a dataset is ? and by the way isn't google installing caffeine with the december rankings.

It seems to me that they are installing caffeine first and and will redo their ranking calculations once caffeine is all in order, am I the only one noticing that ?