Cloaking in Amazon?

[moopy]

In Amazon, there's a page showing a "sort" dropdown.

Looking at the cached version of the page, the "sort" dropdown code isn't found. In addition, checking that page, it returnes a weird 405 "method is not allowed" status code.

Can anyone explain to me what's going on here?

Thanks :)

[edited by: tedster at 4:45 pm (utc) on Jan. 12, 2010]
[edit reason] removed the specific url [/edit]

[tedster]

Normally we would not discuss methods used on a specific website by name. But in this case we'll make an exception, partly because Amazon is such a major company. I removed the specific URL, however. Amazon does this "sort" treatment on a variety of pages.

The key to understanding it is this -- it is not a good idea to allow Google to index different "sorted" versions for the same list of product results. It creates a kind of near-duplicate content.

As to how, exactly, Amazon is serving this slightly altered content to Google it's hard to say. Especially with gigantic websites, Google has been known to work out some agreement for "helpful cloaking" that really does benefit Google and its users. The infrastructure challenges faced by any mega site can be quite daunting and Google does bend a bit.

The bottom line here is that there is nothing truly deceptive about this, and nothing that artificially boosts rankings on important keywords. Rather, it is something helpful to all involved. And so it is one of those allowed cases where Google will accept being served something slightly different than the average browser gets.

They also watch this kind of thing closely for any attempted abuse. I know of one case where a big company tried cloaking that was aimed at improved keyword rankings, and Google dropped their product pages from the results until they cleaned it up and asked for reconsideration.

[moopy]

Thanks for the exception tedster, I also belive it's an important topic to discuss. Google has very clear rules against cloaking.

[google.com...]

My first question is about "helpful cloaking" and how to go about it. As an SEO company which SEOs big companies around the globe, in some cases such as different sorted versions of the same content, it makes sense to use "helpful cloaking" and hide the sort options from Google, doesn't it? How would you go about it?

The second question is about what Amazon uses. What exactly is the 405 status and how do they manage hiding the sort options from Google, technologically?

I'm very curious about these questions...

[tedster]

There are ways to handle "sort" options that work for most all but the very tangled, large, infrastructures. One simple way is to generate the dropdown box with an external script, and disallow that script in robots.txt.

You can also generate some component in the URL and disallow that type of URL to avoid having the "sorted" URLs spidered. However, I have worked with some major sites, and I know that even the most apparently simple approaches can trip all kinds of problems within a big infrastructure like Amazon.

--

A common reason for a 405 HTTP error occurs when a browser submits a "Post" to a server that is not configured to accept that method. I didn't pull apart Amazon's code, but I would assume that something like this is what you discovered.

[edited by: tedster at 6:48 pm (utc) on Jan. 14, 2010]

[CainIV]

Google has been known to work out some agreement for "helpful cloaking" that really does benefit Google and its users.

Quote of the day right there, tedster.