homepage Welcome to WebmasterWorld Guest from 54.204.94.228
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
www, www21, wwww, etc. Displaying Same Content & Indexed By SEs
What to do?
getxb

5+ Year Member



 
Msg#: 4323360 posted 10:30 am on Jun 8, 2011 (gmt 0)

We have a website www.example.com and the same has also been set as preferred domain from Google Webmaster Tools. But the problem is for some reasons there are more than 50 duplicate sub-domains (not sure if this the right terminology) indexed and listed in Google, Yahoo, Bing. All of them shows the same content as the original domain because in UltraDNS *.example.com points to www.example.com

For example wwwww.example.com, www31.example.com, www.wwww46.example.com, etc. are all crawled and displayed in the SERPs (may be in the 3rd page or may be in the 9th page). Even a few have the same Page Rank as the main website!

I guess this is a case of url canonicalization and are negatively affecting the rankings of the main domain as i) They are considered as duplicate contents ii) PR/weightage getting shared

I think the best way to handle them is to do a 301 redirection for all of them for the next 60-90 days, let the SEs update their indexes and them just terminate them.

Please let me know your thoughts.

 

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4323360 posted 11:49 pm on Jun 8, 2011 (gmt 0)

Yes. 301 redirect to retain traffic.

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.(html?|php)\ HTTP/
RewriteRule ^(([^/]+/)*)index\.(html?|php)$ http://www.example.com/$1 [R=301,L]

RewriteCond %{HTTP_HOST} !^(www\.example\.com)?$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]

zeus

WebmasterWorld Senior Member zeus us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4323360 posted 12:00 am on Jul 21, 2011 (gmt 0)

i have the same problem 500 pages with www1.domain..... your redirect code is that 2 different or one whole and would it help in my situation also

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4323360 posted 7:18 am on Jul 21, 2011 (gmt 0)

Use all of the code.

The first chunk fixes index page canonicalisation and the second chunk fixes domain canonicalisation.

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4323360 posted 6:48 am on Jul 22, 2011 (gmt 0)

let the SEs update their indexes and them just terminate them

keep the redirects in place.
otherwise the next time a non-canonical hostname is discovered by a SE the duplicate content will get reindexed.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved