homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Robots.txt - Disallow subdomain only

 11:50 pm on Aug 16, 2010 (gmt 0)

I am new to using robots.txt on subdomains.

We have recently added a new subdomain forums.mysite.com and I'd like to exclude bits until we are ready to launch.

If I use:
User-agent: *
Disallow: /

Will that exclude the robots from the whole domain?

I am planning to put the robots.txt in the subdomain folder - should I be putting it in the main root domain instead?

Thanks for any advice



 8:50 am on Aug 17, 2010 (gmt 0)

the robots.txt file must exist in the root directory of the relevant hostname.
for example you could have a /path/to/http/docs/robots.txt file for http://example.com/robots.txt and another /path/to/http/docs/subdomain/robots.txt file for http://subdomain.example.com/robots.txt


 7:30 am on Aug 26, 2010 (gmt 0)

Sorry for the late reply phranque - I missed your reply.

And thanks for the info - I'll be implementing what you suggested.


Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved