Page is a not externally linkable
not2easy - 4:34 am on Jan 8, 2013 (gmt 0)
You cannot block robots with robots.txt but you can disallow areas of your site or the whole site for compliant robots only. Usually, compliant robots show a URL where you can read their policies. The robots.txt file does not force robots to obey your instructions. On Windows servers, look into using isapi instead of .htaccess, try a search here or ask your host.