Robots.txt and password-protection

[hwy2hel]

A stupid question really... My client has a testing server (subdomain.example.com) that is password-protected. When I was doing a backlink check for example.com, I've noticed that subdomain.example.com was listed there, hence it can probably be found through other means as well. I do not want the subdomain to appear at all (or at least appear as little as humanly possible). There is a robots.txt in place, but again, since the whole subdomain is password-protected, does that mean that SEs are not able to read that robots.txt file? And if so, what should be done to make it visible to SEs?

Thanks much in advance!

[goodroi]

hi hwy2hel,

the search engines are probably seeing your home page which has the username and password input screen becuase that page is not password protected. if your subdomain is properly password protected than the bots will not be able to crawl it.

if you want a more aggressive approach to restricting access you probably will want to make changes to your .htaccess file

[hwy2hel]

thanks much! this helps :)