homepage Welcome to WebmasterWorld Guest from 54.204.231.253
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Robots.txt - Disallow subdomain only
pavlovapete




msg:4188013
 11:50 pm on Aug 16, 2010 (gmt 0)

I am new to using robots.txt on subdomains.

We have recently added a new subdomain forums.mysite.com and I'd like to exclude bits until we are ready to launch.

If I use:
User-agent: *
Disallow: /

Will that exclude the robots from the whole domain?

I am planning to put the robots.txt in the subdomain folder - should I be putting it in the main root domain instead?

Thanks for any advice

 

phranque




msg:4188167
 8:50 am on Aug 17, 2010 (gmt 0)

the robots.txt file must exist in the root directory of the relevant hostname.
for example you could have a /path/to/http/docs/robots.txt file for http://example.com/robots.txt and another /path/to/http/docs/subdomain/robots.txt file for http://subdomain.example.com/robots.txt

pavlovapete




msg:4192384
 7:30 am on Aug 26, 2010 (gmt 0)

Sorry for the late reply phranque - I missed your reply.

And thanks for the info - I'll be implementing what you suggested.

Cheers

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved