Page is a not externally linkable
rogerd - 3:53 am on Jan 2, 2007 (gmt 0)
Even if you aren't concerned about inappropriate content being spidered by the major SEs, a robots.txt file will avoid clogging up your logs with 404 errors.
It's a handy place for a blog [webmasterworld.com], too. Of course, that approach may not be for everybody. ;)