Page is a not externally linkable
dougmcc1 - 2:25 pm on Sep 19, 2003 (gmt 0)
In your tld root directory: Remember to put a blank line at the end of your robots.txt and use a validator to make sure you dont have any errors.
Where would I put it?
The root directories of the subdomain and tld. What would the syntax look like for disallowing, say, subdomain.domain.com/mailer/mail.php?
In your subdomain:
User-agent: *
Disallow: /mailer/mail.php
User-agent: *
Disallow: /subdomain/mailer/mail.php