Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Advanced robots.txt question


Buddha - 5:47 pm on Feb 14, 2005 (gmt 0)


I have a development server at test.domain.com on a separate dedicated server from my live site, www.domain.com.

test.domain.com is a mirror dev site. It's not linked to from anywhere but google has indexed it. For fear of dup content, I don't want Google to crawl it.

How do I exclude the test.domain.com? Do I block it using the robots.txt from the www.domain.com (which is on a different server)? Or do I block it from the robots.txt from the subdomain of the actual test.domain.com server?


Thread source:: http://www.webmasterworld.com/robots_txt/551.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com