Summary of Duplicate Content Discussion at SMX

Vanessa Fox summarizes; many interesting points

[netmeg]

The Webmaster Central Blog has a new entry which has (what I think is) a lot of very interesting information about duplicate content:

Duplicate content doesn't cause your site to be placed in the supplemental index. Duplication may indirectly influence this however, if links to your pages are split among the various versions, causing lower per-page PageRank.

There are also a number of suggestions that were made that were of interest:

Specifying the preferred version of a URL in the site's Sitemap file
One thing we discussed was the possibility of specifying the preferred version of a URL in a Sitemap file, with the suggestion that if we encountered multiple URLs that point to the same content, we could consolidate links to that page and could index the preferred version.

I would love this; in this day and age of buyouts and takeovers, I have boatloads of clients with multiple domain issues.

Making a duplicate content report available for site owners
There was great support for the idea of a duplicate content report that would list pages within a site that search engines see as duplicate, as well as pages that are seen as duplicates of pages on other sites. In addition, we discussed the possibility of adding an alert system to this report so site owners could be notified via email or RSS of new duplication issues (particularly external duplication).

WAY cool...

There's a lot more - go read it.

[googlewebmastercentral.blogspot.com ]

[Whitey]

That's certainly a good communication which means G is listening hard on this issue and the confusion faced by webmasters / siteowners.

I particularily like the proposed reporting of "duplicate pages" via Webmaster Central which could potentially save years of difficulties for webmasters. It appears to be a much more transparent approach.

[edited by: Whitey at 9:53 pm (utc) on June 13, 2007]

[g1smd]

I can see some of those reports as being useful to spammers too.

How much do I need to alter that scraped content to no longer be classed as a duplicate of where it was nicked from?

Scary.

[Whitey]

If somehow G could validate an increased level of trust with webmasters / siteowners that register at Webmaster Central , they could potentially cover this to a large extent IMO

[SEOPTI]

"Duplicate content doesn't cause your site to be placed in the supplemental index."

What does it mean? As an example, I always thought that if 10 sites use 10 Amazon or affiliate feeds, 9 of them will be supplemental because of using duplicate content.

[g1smd]

Some 18 months ago, or more, we saw lots of Duplicate Content tagged as Supplemental.

It was assumed that this was a direct relationship.

Some years later we find out that it is indirect.

Supplemental is all about Pagerank, and Duplicate Content lowers PageRank for individual URLs. Lowered PageRank leads to Supplemental.

There are several "types" of Duplicate Content, though, so this scenario does not cover all possibilities, by a very long way.

[jdMorgan]

I'm not real impressed with the idea of "preferred domain" selection using Webmaster Tools or Sitemaps. The HTTP protocol has had a non-proprietary method for dealing with this from day one... The 301-Moved Permanently redirect response.

If installed from the second a server goes live, multiple domains are never a problem, and if care is taken not to allow duplicate URLs --for example, "www.example.com/" and "www.example.com/index.php"-- there is no need for a "fix-it-in-google-only" solution.

I don't mean to imply that there's a limit to the 301 redirect's effectiveness, just that if duplication-prevention is taken care of during the site set-up phase, there's no chance for a problem to develop. The 301 is equally effective at fixing the problem after the fact -- but it may takes several weeks or a few months if done after damage has become evident.

I am glad to see a peek behind the veil -- the statement about the indirect relationship between dup-content and Supplementals -- and hope for a bit more of the same, with the proviso that I'd rather they not publish info that might make it easier for scrapers... But then, they usually don't.

Jim