dougwilson - 1:55 am on Mar 1, 2013 (gmt 0)
Is it March already?
Recently cleaned my htaccess. Not really blocking much anymore. But I still watch and below are some user agents and requests that annoyed me enough to add to server process time. Yes I use "" for strings and ^ sometimes. Checking http viewer (rex swain) for each.
# bots naughty or no return
SetEnvIfNoCase User-agent ^\xff blocked
SetEnvIfNoCase User-agent Curl blocked
SetEnvIfNoCase User-agent ".exe" blocked
SetEnvIfNoCase User-agent HTTrack blocked
SetEnvIfNoCase User-agent magpie-crawler blocked
SetEnvIfNoCase User-agent "Web Core / Roots" blocked
SetEnvIfNoCase user-agent "MSIE\ 5.0" blocked
SetEnvIfNoCase Request_URI //db blocked
SetEnvIfNoCase Request_URI /user/soapCaller.bs blocked
deny from env=blocked
Most of this (above) is just the difference between a 403 and a 404.
What I am serious about denying are legit data mining outfits - because I despise them.