Page is a not externally linkable
-- Webmaster General
---- bandwidth Issue
tangor - 8:02 pm on May 21, 2011 (gmt 0)Thread source:: http://www.webmasterworld.com/webmaster/4315608.htm
This is true, but will more swiftly reveal which are the bad bots. :)
Other method it to whitelist what UA is allowed and not worry.
Brought to you by WebmasterWorld: http://www.webmasterworld.com