Page is a not externally linkable
victor - 7:09 pm on Sep 12, 2005 (gmt 0)
If a site has a link to its robots.txt, then it is directing non-bot visitors to it. Makes it fair game for search engines to index.
Reserving robots.txt as a file only for web bots is a convention. Its usage is neither compulsary nor universal.