Key_Master - 10:03 pm on Jul 31, 2007 (gmt 0)

If your only intention is to disallow access to a file or files then a robots.txt would work just fine.

However, you can't use noarchive, nofollow, nosnippet, or unavailable_after in a robots.txt file. The header X-Robots-Tag is a much more powerful tool. It allows us to use these directives without needing to edit files. It also allows us to use these directives for media files, pdf files, etc, that can't have meta tags directives inserted in them. It can also be used for user-agent/ip delivery.