GaryK - 5:03 am on Jun 24, 2009 (gmt 0)
It would have been nice if there was a standard plugin/module deployed in all webservers that would auto ban bots that either did not read robots.txt or disobey it.
It's a good idea in theory. In practice it's not doable. Cause with automation it's not always possible to ensure what you're banning is really a bot.