dmorison - 12:19 pm on Sep 23, 2003 (gmt 0)

If you're on Apache you could use mod_rewrite. I'm not sure of the details, you'll have to check the docs at httpd.apache.org; but you should be able to test for HOSTNAME and robots.txt, and then have it either return 404 or serve robots.txt as you wish.