Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

How can I stop search engine to go to my subdomain?



2:37 am on Dec 24, 2010 (gmt 0)

5+ Year Member

How can I stop search engines to go to a sub domain of my site?


2:48 am on Dec 24, 2010 (gmt 0)

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

The sub domain will be viewed by a bot as a different site. It will therefore read the robots.txt file located at subdmain.example.com/robots.txt

If you disallow robots there, the sub domain wont be crawled.



5:02 am on Dec 24, 2010 (gmt 0)

5+ Year Member

Thank you, Mack.


4:16 pm on Jan 2, 2011 (gmt 0)

You could also twick the robots.txt file to control which sub-domain will be scanned. could be a powerful tool to 'trick' crawlers "last-scan" timers.

Featured Threads

Hot Threads This Week

Hot Threads This Month