Robots.txt exclusion help

[webguy7]

Hello,

I have a section of my website that is accessed through

1) mydomain.com/myfolder

2) myfolder.mydomain.com

3) newdomainpointingtomyfolder.com (addon domain)

How can I prevent search engines from indexing pages using 1 and 2 but without excluding 3 (the new domain name that points to that section of my website)?

Thank you

[jdMorgan]

I'd strongly recommend that you fix this problem at the root, not with some robots.txt work-around: Use a 301-Moved Permanently response to redirect all requests for the non-canonical domains to the canonical domain using mod_rewrite on Apache or ISAPI Rewrite on IIS. If the site is dynamic, then you also have the option of implementing this redirect in your main script.

Having the same content appear at multiple URLs is the root problem. Even something as simple as having your pages accessible at both domain.com and www.domain.com can cause trouble, so why not fix it all at once.

Jim

[webguy7]

Hi Jim,

Thank you for your post.

I thought about using redirects but I wasn't sure if that would prevent search engines from indexing the pages with multiple urls. But I will try that. Thanks.