dmorison - 8:19 am on Sep 1, 2003 (gmt 0)

If you just want to use robots.txt to ban particular user-agents (and therefore the same file would be ok for each site), then you can, if you're on a *nix box, maintain a single central copy; and create a soft link to it in each directory.

You still have to create all the soft links, but after that you can quickly make one change that will impact all sites.

The Linux command is "ln" to create a link, so if you were in the wwwroot directory of SiteA, you might type:

ln robots.txt /path/to/robots.txt

...where /path/to/robots.txt is your central copy.