microcars - 3:38 pm on Feb 20, 2006 (gmt 0)

I recall reading (here maybe?) a while back about someone that DID use Robots.txt to separate the good bots from bad bots.

I don't have the specific details but a Robots.txt rule was set up to disallow ANY bots from going to a certain directory.
If a bot went to that directory anyways, a script added its name and/or IP address to the .htaccess file which then disallowed it from any future access to ANY area of the website.