Removing Duplicate Pages - Best Way?

[c41lum]

Hi guys I think I have been hit with a Duplicate Content Penalty. We operate a large site with over 500,000 pages. I have checked webmaster Tools and it states that we have over 9500 duplicate pages.

We have recently took a big hit in the serps and think its got something to do with these dups.

Whats the fastest way to remove them we have done a 301 but Google seems to be taking very little notice of it. Should I use the removal tool and 404 the pages.

Any help would be appreciated.

Many Thanks

[tedster]

Accidental duplicates almost never cause a penalty - a site has to be doing something really spammy with duplicates to get penalized. I'd say the 301 should be all you need to do to handle the duplicates, but don't get your hopes too high for that action fixing the penalty.

If you do want to try the url removal process, then you'll need to either set a robots.txt disallow rule for those urls or make them 404. In fact, if you can easily create a robots.txt disallow rule, that would probably be faster than 301 redirects, even if you don't use the url removal tool.