Welcome to WebmasterWorld Guest from 54.211.17.91

Forum Moderators: goodroi

Message Too Old, No Replies

Disallowing Robot's Effect on Sub-Domain

Will it disallow the sub domains as well

   
4:39 am on Jul 14, 2004 (gmt 0)

10+ Year Member



I've got a main domain www.example.com that I don't want any spiders to visit (since it's a duplicate of another site I run and don't want the duplicate site penalty - it's just a much easier url for people to remember than my main one).
Now, I've got a few 'sub-domain' sites that I still DO want the spiders to visit, ex:

abc.example.com
def.example.com

Now, if I dissallow robots to www.example.com, will they still index abc.example.com, def.example.com, or will they consider all of those 'dissallowed' since they are under the www.example.com domain?

Hopefully someone out there has this situation in place on there site and knows the outcome so that I don't just need to take the blind leap of faith to find out.

Thanks for any help on this.

BPPilot

5:53 am on Jul 14, 2004 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



You just need to be sure that each subdomain returns its own unique robots.txt file, and put the appropriate exclusions in each robots.txt file.

Jim

 

Featured Threads

Hot Threads This Week

Hot Threads This Month