Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

how to manage cname in robot

don't was to be indexed

5:31 am on Nov 16, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 2, 2003
votes: 0


I have a site where i have used 2 of the cname like cname.domain.com and cname2.domain.com. Now i cna see that these cnames urls are indexed and i dont want them in engines at all. how should i stop these to be indexed through robot file. is something other way except this.

help appreciated in advance..



9:17 am on Dec 14, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 2, 2003
votes: 0

cud anybody tell me pls...

i have cname made for my domain and i dont want them in engines.

should i block the cname like this

disallow [cname.domain.com...]

Will this work or this will also effect the parent site domain.com..?

thanks a lot for sugestions.


7:58 am on Dec 21, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 2, 2003
votes: 0


ThomasB & goodroi,



2:10 pm on Dec 21, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
votes: 0


Please be aware that this is a forum, not instant messaging. Due to differences in time zones, you may be posting when the person best able to answer your questions is sleeping. :)

Also, the moderators are not primarily responsible for replying to posts, they are responsible for moderating posts.

Unless the CNAMES map to a different document root -- in other words, unless they each have their own file space on your server and serve different pages than your primary domain, then robots.txt won't help you. This is because robots.txt assumes that each domain/hostname has it's own robots.txt file, and it has no provisions for specifying which domain/hostname the restrictions that it contains apply to.

If your CNAMES do map to separate filespace, then you can put a unique robots.txt into the root directory for each CNAME, containing

User-agent: *
Disallow: /
to disallow all robots from fetching all resources.

However, in many cases, all hostnames are 'pointed' to the same file space, and a unique robots.txt for each domain can't be used. The usual solution is to redirect all non-canonical domain name variants to the main domain using a 301-Moved Permanently redirect. This can be done with mod_rewrite or PHP on Apache, ISAPI Rewrite or .asp on IIS, and several other methods, depending on your server technology. A search for "redirect canonical domain" on WebmasterWorld may turn up some useful threads to help get you started.



Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members