Welcome to WebmasterWorld Guest from 54.226.46.6

Forum Moderators: goodroi

Message Too Old, No Replies

How can I stop search engine to go to my subdomain?

     

virtualreality

2:37 am on Dec 24, 2010 (gmt 0)

5+ Year Member



How can I stop search engines to go to a sub domain of my site?

mack

2:48 am on Dec 24, 2010 (gmt 0)

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The sub domain will be viewed by a bot as a different site. It will therefore read the robots.txt file located at subdmain.example.com/robots.txt

If you disallow robots there, the sub domain wont be crawled.

Mack.

virtualreality

5:02 am on Dec 24, 2010 (gmt 0)

5+ Year Member



Thank you, Mack.

Joshb

4:16 pm on Jan 2, 2011 (gmt 0)



You could also twick the robots.txt file to control which sub-domain will be scanned. could be a powerful tool to 'trick' crawlers "last-scan" timers.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month