Forum Moderators: goodroi
I have 2 domains and my website can be accessed with both addresses and www & non-www versions.
Now I only want www.domain2.com to be indexed by SE, because it has the best PR.
So if I create a robot.txt file with following content, will that work?
--------------------------------------------
User-agent: WebCrawler
Disallow: [domain1.com...]
Disallow: [domain1.com...]
Disallow: [domain2.com...]
--------------------------------------------
I tried redirets and mod rewrited before, but I don't really care if people can access the site differently... ( only care about SE indexing )
Hope somebody can help me with this
Reards
2 everybody
MAVI
The correct solution to this problem is to use a 301-Moved Permanently redirect to redirect all non-canonical domain variations to the single canonical domain. This is a popular subject, and searches on WebmasterWorld for "domain canonicalization," "canonical domain," and "www non-www domain" will turn up hundreds of threads with discussion and code examples.
Leaving your site as it is now will result in PageRank and link-popularity being 'split' across the multiple domain variants -- in effect, making your site compete against itself in the search results. Attempts to promote more than one domain may even result in search engine duplicate-content penalties -- if the promotion is too heavy or the domains too numerous.
Jim