homepage Welcome to WebmasterWorld Guest from 54.197.215.146
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
How can I stop search engine to go to my subdomain?
virtualreality




msg:4245764
 2:37 am on Dec 24, 2010 (gmt 0)

How can I stop search engines to go to a sub domain of my site?

 

mack




msg:4245768
 2:48 am on Dec 24, 2010 (gmt 0)

The sub domain will be viewed by a bot as a different site. It will therefore read the robots.txt file located at subdmain.example.com/robots.txt

If you disallow robots there, the sub domain wont be crawled.

Mack.

virtualreality




msg:4245791
 5:02 am on Dec 24, 2010 (gmt 0)

Thank you, Mack.

Joshb




msg:4248312
 4:16 pm on Jan 2, 2011 (gmt 0)

You could also twick the robots.txt file to control which sub-domain will be scanned. could be a powerful tool to 'trick' crawlers "last-scan" timers.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved