Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Disallowing Robot's Effect on Sub-Domain

Will it disallow the sub domains as well

4:39 am on Jul 14, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:May 28, 2004
votes: 0

I've got a main domain www.example.com that I don't want any spiders to visit (since it's a duplicate of another site I run and don't want the duplicate site penalty - it's just a much easier url for people to remember than my main one).
Now, I've got a few 'sub-domain' sites that I still DO want the spiders to visit, ex:


Now, if I dissallow robots to www.example.com, will they still index abc.example.com, def.example.com, or will they consider all of those 'dissallowed' since they are under the www.example.com domain?

Hopefully someone out there has this situation in place on there site and knows the outcome so that I don't just need to take the blind leap of faith to find out.

Thanks for any help on this.


5:53 am on July 14, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
votes: 0

You just need to be sure that each subdomain returns its own unique robots.txt file, and put the appropriate exclusions in each robots.txt file.



Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members