Do you use .htaccess to block bad agent/bot?

I want to know if this is worth to add these bot to my .htaccess file

[iProgram]

Hello, I want to block email collectors and website downloaders using my .htaccess, for example:

SetEnvIfNoCase User-Agent "^WebZIP" bad_bot
..... (A very long list here)
SetEnvIfNoCase User-Agent "^e-collector" bad_bot
deny from env=bad_bot

However, I am afraid this will slow down the opening speed of all web pages. Which method are you using to block these bad agents? (The robots.txt is useless, IMO)

[jdMorgan]

iProgram,

Yes, it does slow them down, but you might want to consider the other delays in the system, such as the transmission delay from server to client, server disk access latency, etc. These other delays are measured in milliseconds rather than microseconds, and will probably swamp any delay due to native Apache directive execution.

If your site gets fewer that, say, 7200 unique visitors per hour, I doubt you'll notice *any* additional delay if you use fewer than 200 additonal access-control directives.

I've implemented access controls using both mod_access and mod_rewrite directives. As you point out, robots.txt must be recognized as a request to properly-implemented, non-malicious automated user-agents. It does no good if the robot is not coded properly or if it is malicious.

Jim

[iProgram]

hello Jim, thank you for your message. I use mod_rewrite too. Now I decide to apply this to all websites on my server. I think the best place to block bad agents is not in .htaccess file, but httpd.conf file. It will work for all my websites on this server, right? My next two questions are:
1. In httpd.conf file, where sould I place the SetEnvIfNoCase User-Agent "^WebZIP" bad_bot... codes?
2. Need I restart Apache to make this work?

[jdMorgan]

iProgram,

1a. I'm not sure, I haven't installed access restrictions in http.conf -- I rent my server space. :(
1b. Please let us know what works for you when you get it working. :)
2. Yes, you must restart Apache for this stuff to take effect.

Jim