jdMorgan - 2:18 pm on May 20, 2004 (gmt 0)

> Where do you place that information? In an alternate file or within the head tags of each page?

In a plain-text file called "robots.txt [robotstxt.org]" in the root directory of your site - along with your "home page." The on-page robots tag is useful for people writing html pages to put on a server where they do not have access to robots.txt, and so can't change it.

> something that doesn't really matter?

One very good reason to put up a robots.txt file even if it is blank is to prevent hundreds of 404-Not Found errors in your server log, caused by robots trying to check your robots.txt file. This reduces error log file clutter, and makes it more useful for finding real problems.

Jim