homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

www, www21, wwww, etc. Displaying Same Content & Indexed By SEs
What to do?

 10:30 am on Jun 8, 2011 (gmt 0)

We have a website www.example.com and the same has also been set as preferred domain from Google Webmaster Tools. But the problem is for some reasons there are more than 50 duplicate sub-domains (not sure if this the right terminology) indexed and listed in Google, Yahoo, Bing. All of them shows the same content as the original domain because in UltraDNS *.example.com points to www.example.com

For example wwwww.example.com, www31.example.com, www.wwww46.example.com, etc. are all crawled and displayed in the SERPs (may be in the 3rd page or may be in the 9th page). Even a few have the same Page Rank as the main website!

I guess this is a case of url canonicalization and are negatively affecting the rankings of the main domain as i) They are considered as duplicate contents ii) PR/weightage getting shared

I think the best way to handle them is to do a 301 redirection for all of them for the next 60-90 days, let the SEs update their indexes and them just terminate them.

Please let me know your thoughts.



 11:49 pm on Jun 8, 2011 (gmt 0)

Yes. 301 redirect to retain traffic.

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.(html?|php)\ HTTP/
RewriteRule ^(([^/]+/)*)index\.(html?|php)$ http://www.example.com/$1 [R=301,L]

RewriteCond %{HTTP_HOST} !^(www\.example\.com)?$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]


 12:00 am on Jul 21, 2011 (gmt 0)

i have the same problem 500 pages with www1.domain..... your redirect code is that 2 different or one whole and would it help in my situation also


 7:18 am on Jul 21, 2011 (gmt 0)

Use all of the code.

The first chunk fixes index page canonicalisation and the second chunk fixes domain canonicalisation.


 6:48 am on Jul 22, 2011 (gmt 0)

let the SEs update their indexes and them just terminate them

keep the redirects in place.
otherwise the next time a non-canonical hostname is discovered by a SE the duplicate content will get reindexed.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved