Google Updates and SERP Changes - April 2010

[wmxpert]

< continued from [webmasterworld.com...] >

We are in the same situation, since March 15, 2010 our traffic is continuously declining with the average of 6% every week. In current week it gets worst, approx. 15% decline!

Overall, 30% to 35% traffic has been reduced compare to peak week of January 2010.

"site:" is also showing huge drop in index pages, though it only provide the avg. not actual.

Don't know what's going on, Is it really the effect of Caffeine roll-out?

[edited by: tedster at 9:46 pm (utc) on Apr 1, 2010]

[arizonadude]

I'm seeing it on Google.com in the Phoenix area and have seen it all day today and half the day yesterday.

[dusky]

1script:

Why would you keep elsewhere only 10% of the data belonging to one site. Also, the drop affects a site (as in all or most pages of the same site) yet I think it makes sense to think that they arrange the data per page (individually).

The answer IMO is the site main keyword, usually the homepage or name and major pages titles (homepage, subdomain homepage and main directory titles) are stored in one table, and in most cases they are only a few dozen pages. The rest of the pages on all the site, they can be millions of them are stored in other table/s. When G* moves whole sites to the new structure, they move the INDEX of a particular site first, the rest of subpages may take from 2-5 days to weeks and recover again like nothing had happened.

Example of a database index structure they may be using on an idex table:

- ..19998888466780: widet.com | widget resources LLC | other records of subdirectories / pages which are located in a different table in the new structure...

- ..19998888466781 | widget.com/redwidgets on widget.com | other records of subdirectories / pages which are located in a different table in the new structure...

- ..19998888466782 | widget.com/greenwidgets on widget.com | other records of subdirectories / pages which are located in a different table in the new structure...

etc...

The above is an index only, a skeleton of a particular site, which in most cases is still available when you search for the MAIN company name or keyword, any other keywords and none returned. The reason is it'd have only taken few minutes to migrate the above index to the new structure, hence a search is returned, OR it is still served from the old structure BUT the other thousands or millions of pages are still being migrated, gbot may still be busy spidering even more...

member22:

From the experience I have everytime google does an update if your site is meant to rank better than it used to be...google moves your site further in the ranking for some time ( it is because it doing its ranking calculations ) and soon you will be back where you use to be if not better.

Yes, I second that as I seen it from my own experience and I spoke about here.

The process may take few weeks especially if the site has hundreds of thousands of pages, but the good thing is, the site will recover with better rank.

1script, I noticed what I would think may be problems you'll face with SEs or are facing with the site on your profile, can't say it here, PM me and I'll point them out!

Could someone confirm this from Mountain View either here or on the google blog, surely it can't be a security or algo giveaway, it'll surely help a lot of webmasters understand why the sudden drop in traffic for days to weeks then recover like nothing had happened!

[cangoou]

In my niches the search for "widgéts" (misspellig) instead of "widgets" (main-/money-keyword) still produces almost the same good, old positions for sites that got kicked down, which might be a hint that a (new?) filter is applied to the sites for the keyword.

So... don't want to sound pessimistic, but might this filter be permanent? Why should a site drop for "widget" but not for "widgét" during an update? Or is this observation only coincidences in my cases?

[caribguy]

I'd bet money on the fact that Gorg is not using an RDBMS but an object oriented structure instead... Objects (pages) can have attributes and immediately inherit from algo changes.

It makes sense when you think about the way several shards of data are collated together (i.e. freshness attributes are possibly stored in a different database, and on and on).

[mercury77]

Hi, is anyone still having problems with google indexing 301 headers very slowly, someone mentioned about this like a month ago. I have made 301 headers to my site 4 weeks ago, and still nothing happening, old removed sites still show on Google.

[dusky]

caribguy

I'd bet money on the fact that Gorg is not using an RDBMS but an object oriented structure instead... Objects (pages) can have attributes and immediately inherit from algo changes.

I agree with most of what you think caribguy, with a slight difference and an addition:

RDBMS structures are too slow for any SQL with the magnitude of G*'s databases, because they rely on tables seeking other tables for relational data, even their database indexing is inadequate and causes bottlenecks, so yes it's almost a certainty G* is not using any RDBMS structure at all and never used for years. They almost certainly used an in-house system housing the big daddy system and found that has outgrown its sleeping bag. Now, they may be using a similar structure to what Microsoft is rolling for Bi*g, and I think is based on a distributed / table independent matrix database system instead of the hierarchical / relational system most of us are familiar with. That is derived from the wolfram alpha concept with a small difference. This matrix system has tables that are independent from the database except a small ID in the index, tables themselves are like a whole self contained and "self questioning" databases on their own.

The flat / matrix structure of a database infrastructure obviously is not hierarchical and the SQL is entirely different, it is a "straight to the point" seek system, bypassing all conventional query language, there is no select this from or select all from done on the tables themselves, it's based on "get this ID's info" from the database structure by country, region, language..., then that particular ID's info is returned, say index ID of green widget 100999976654788...for example which is for all sites ranking for green widgets, and info is returned from the green widgets tables: gree-widgets-PR10, gree-widgets-PR9, gree-widgets-PR8, and of course sites from gree-widgets-PR10 are returned the highest ranking in the SERPs. The algo does not directly affect site ranks, but IMO is allocated or assigned through their algo calculations which may take weeks or months, and then sites are stored within the terms tables above by rank. The draw back of such a system is the large tables which are larger than any conventional MYISAM, InnoDB or other known system, hence the need for a much larger storage system, and therefore Caffeine!

Has anybody worked out what the abbreviation of C.A.F.F.E.I.N.E is?, do some brain storming and you'll see what I mean, the guys at Mountain View are geniuses, I am still scratching my head!

[internetheaven]

Has anybody worked out what the abbreviation of C.A.F.F.E.I.N.E is?

Catch All Fresh Fluff Endorsing Instantly Not Evaluating

[dusky]

Close :), but I am still scratching my head though!

[giggle]

If you shift CAFFEINE +1 character you get DBGGFJOF

Data Base Google Grows For Joy Of Franchises

[suratmedia]

G store data on large B+ Tree with their own records structure.

Other search engine using similar infrastructure called it XFS [X-File System]

[Angelis]

Yesterday position 66, now position 4. Um, go figure! Either way I like whatever is going on.

[cangoou]

Either way I like whatever is going on.

... if it lasts I would agree ;-)

[1script]

@dusky:

The answer IMO is the site main keyword, usually the homepage or name and major pages titles (homepage, subdomain homepage and main directory titles) are stored in one table, and in most cases they are only a few dozen pages.

.
In my case neither the homepage nor index pages of subdirectories or pages with listings of posts (picture a forum site structure) have ever ranked for anything. Only individual posts / pages at the very end of the site's linking structure tree did. In fact, I'm experimenting (after the crash, so the experiment should not be the cause) with no-indexing (meta robots) most of the site's structure except for those final pages.
So, getting back to my original concern: where did 90% of the same data go? What I mean by that is that all that still ranks are the individual pages, just like before March 15th. There is really no difference in the type of pages that still rank. In fact, many pages (hundreds) have not seen any change in rankings whereas most (tens of thousands) have simply vanished from SERPs. To make matters even more complicated some KWs have clearly lost rankings and have moved to page 2 or 3 of SERPs. But some gained.

So, the big picture looks like this: you have a site. Throw away 90% of its pages. Then run ranking algos on the remaining 10% and place those pages where they belong.

When I say "throw away" I don't mean de-index. All the pages that no longer appear in SERPs sill have cache. In most cases I've checked (although few dozens checked do not correctly represent tens of thousands of pages in question) all pages had cache dated AFTER the crash. Gbot activity neither stopped - same visit rates. So, back end - business as usual, front end - 90% of business gone.

So, again it appears as if out of 10 chunks of the same data 9 have been lost (invalidated, moved to supplemental, trash bin, archive - whatever it is that happened to it). Not as if some of the higher value data about the site has moved faster than the rest.

[internetheaven]

Did all hyphenated domains just get dropped a bit?

I've always been waiting for G to start using hyphenations as a spam filter. They like easy (but daft) spam filters so this would be a perfect one for them to try.

[Vimes]

Things appear to have settled down again, from my data there is a 24-48 hour settling period and then they throw another twist of lemon in to the sauce.

Seems to have happened in my part of the world on the 15th-16th March and then again on the 22nd-23rd on the 7th-8th April and now again.

any one else seeing this?

Vimes.

[internetheaven]

UK and Caffeine don't seem to be working so well together.

I just saw 2 phishing sites appear under the bank they were trying to scam ... they were fresh content with most likely a sudden influx of visitors from GMail and Toolbar entries ... that doesn't make them relevant though.

They are really switching their entire system over to avoid another Michael Jackson moment and it has to be the stupidest thing I've ever seen. If they wanted to help Bing's ad campaign in the UK, this'll do it!

P.S. I'm not just a bitter webmaster, most of my sites are doing better with Caffeine, I'm a bitter searcher who does not like being forced to look at the paid ads on the side because the organics are so awful. I have most of my family and friends on Bing now.

[gouri]

Vimes,

If possible, can you please tell me, for these 3 sets of dates, is there a lot of difference in what you are seeing in terms of rankings in the SERP or are the rankings looking similar for all three sets?

[gford]

Internetheaven - I was doing a search on G yesterday not related to my sites for a particular scanner wanting to get some software (non drivers) for it.

Well, wouldn't you know, my norton sonar saw 3 of the top 6 results as attack sites.

And what is with all the article repository type sites ranking top 10 now too? sheesh..

[arizonadude]

I'm a bitter searcher who does not like being forced to look at the paid ads on the side because the organics are so awful.

Their plan is working perfectly. How do you think they were able to post such a huge quarterly profit?

Money is driving the plex now, not user satisfaction.

That said, I love the new Caff index. It has treated most all my sites very well.

[Shaddows]

Money is driving the plex now, not user satisfaction.

That said, I love the new Caff index. It has treated most all my sites very well.

A couple of points on this.
1) Declining user satisfaction -> declining traffic -> declining revenue/profit. User satisfaction IS money

2) Rephrased "My sites are doing well now user satisfaction is out the window!"

[arizonadude]

A couple of points on this.
1) Declining user satisfaction -> declining traffic -> declining revenue/profit. User satisfaction IS money

2) Rephrased "My sites are doing well now user satisfaction is out the window!"

Let me rephrase that.

My sites traffic are up and I'm making more money so I could care less if others people traffic is down and based on the fact the Google just posted nice profits for the quarter, user satisfaction does not equal money because the serps have been in turmoil for most of that reporting quarter.

People are like trained monkeys when it comes to using Google. If they don't get the results they want, they click an ad and that makes Google very happy. The trained monkeys would never think of using another search engine because it's Google. After all, they have a verb named after them in the dictionary. ;)

[topvet]

I was at pubcon in London 2003 - and Google had just made a major change - and the same talk was going around.
"bad serps = more money for google"

If you think about it from that point of view - it doesn't really make any sense. Transitions are exactly that. Especially if tuning is involved.

[member22]

I currently have google new interface on one of my browsers and its seems that on the new interface google is doing a ranking update ( at least for one of the keywords I am looking at )

Unless you have this new interface you won't see any changes in the rankings at least from what I have seen and the different tests I have done... Has anyone experienced that too ?

[walkman]

Guys, what's the status of 'caffeine' as far as you can tell? is this it or what?

[BillyS]

Their plan is working perfectly. How do you think they were able to post such a huge quarterly profit?

Money is driving the plex now, not user satisfaction.

It's doubtful the above could be true. Google's here for the long run, this is good tactic if you're closing down operations.

[Jeffhyde]

Here is what I have noticed....

I use both google chrome and firefox. Both have consistently been having different results. It seems the new SERPS are located in google chrome and firefox is still using the old algo. However the new SERPS are flooded with spam and nonsense. Spammy sites with very little content but massive backlinks are holding ground in the new SERPS and my site has taken massive hits for all major keywords. My site also is stuffed with original content and a nice level amount of backlinks... I went from page 1 to page 3 and all keywords are down.

Can anyone verify which results are the ones that will stay?

[arizonadude]

It's doubtful the above could be true. Google's here for the long run, this is good tactic if you're closing down operations.

A public company is not driven by profit? I don't know what planet you live on, but on this planet every single public company is driven by profit whether they are here for the long run or not.

No profit, no company.

[topvet]

Jeffhyde

I'm seeing the same results in chrome and firefox. (us)

[tedster]

I don't think anyone is saying that Google isn't driven by profit (although they do have other motivations going too). It's just that Google is savvy enough not to use tactics that create short term gains that generate long term trouble.

For years webmasters who lose rankings and feel pushed into using Adwords have conjectured that this is Google's intentional purpose. That doesn't fly very well in the biggest picture, especially because any time one site loses rankings another steps into its place.

Google knows that user satisfaction means profits - long term and sustainable profits, not an extra little bump just for this quarter.

[edited by: tedster at 10:14 pm (utc) on Apr 20, 2010]

[internetheaven]

People are like trained monkeys when it comes to using Google.

Too true. No way Google would have got off the ground if this was their organic offering in the beginning.

Though I must say that they have been working overtime on making the Adverts relevant! Most relevant sets of advertisements I've seen so far.

Guys, what's the status of 'caffeine' as far as you can tell? is this it or what?

It was IT here in the UK ... for about 2 days. Now we're seeing old results again with Caffeine flickering from time to time.