Ban bad bots with Apache 1.3.6

[anneli]

I have a site hosted on someone else's server.

It's Apache 1.3.6

I tried a while ago to ban bad bots but didn't succeed. Now I see that there's a problem with that version and banning bots with .htaccess.

Are there any solutions for me?

I do not have root access, but may succeed in getting the admin to make changes if I spell them out for him in EVERY DETAIL.

[jdMorgan]

anneli,

Welcome to WebmasterWorld [webmasterworld.com]!

> there's a problem with that version and banning bots with .htaccess.

What specific problem are you referring to? A quick review of the Apache docs turned up nothing obvious...

Jim

[anneli]

According to the docs I've read, you can't use BrowserMatch, or SetEnVif (sp?) in .htaccess with that version of Apache.

Are there any other options?

[jdMorgan]

anneli,

I'm not sure what documentation you referred to, but I read:

Module mod_setenvif
Compatibility: Available in Apache 1.3 and later.
(This module includes the SetEnvIf and BrowserMatch directives)

Apache HTTP Server Version 1.3 Documentation [httpd.apache.org]

I suggest you try a very simple test to see if it works, such as banning your own IP address. You can then un-ban yourself by replacing the test .htaccess file with an unmodified original backup via FTP:

SetEnvIf Remote_Addr ^192\.***\.0\.19$ myIP
<Files *>
Order Deny,Allow
Deny from env=myIP
Allow from all
</Files>

Put your own IP address in the first line, escaping all periods by preceding them with a backslash as shown.

Jim

[anneli]

Banning IP addresses works fine. I tried your example (with my IP address), but it produced a 500 error. However, I've done it before, this way, and that works:

order allow,deny
allow from all
deny from 63.148.99

(the IP number is an old Cyveillance range)

But when I tried banning user agents, that's when I didn't succeed.

I should also quote to you from the page you'd been reading:
About BrowserMatch:
Compatibility: Apache 1.2 and above (in Apache 1.2 this directive was found in the now-obsolete mod_browser module); use in .htaccess files only supported with 1.3.13 and later

SetEnvIf:
Compatibility: Apache 1.3 and above; the Request_Protocol keyword and environment-variable matching are only available with 1.3.7 and later; use in .htaccess files only supported with 1.3.13 and later

So, the question still remains. Is there a way for me to ban bots? Oh, I see where we misunderstood each other. You thought I asked about banning IP addresses... No, that works. It's user agents I'm having trouble with.

[jdMorgan]

anneli,

Since BrowserMatch is just a short-cut version of SetEnvIf User-Agent, let's just talk about the latter.

SetEnvIf:
Compatibility: Apache 1.3 and above; the Request_Protocol keyword and environment-variable matching are only available with 1.3.7 and later; use in .htaccess files only supported with 1.3.13 and later.

This means you can't use

SetEnvIf Request_Protocol HTTP/0.9 old_http
SetEnvIf Request_Protocol HTTP/1.0 old_http

-or-

SetEnvIf old_http pre11

and you can't use SetEnvIf in .htaccess. BUt you stated that you might be able to get the code installed in httpd.conf if you spelled it out completely.

So, SetEnvIf may not be used to check Request_Protocol or to check user-defined environment variables prior to version 1.3.13. Since neither is the case in this code, it should work.

SetEnvIf User-Agent ^HARVEST badbot
SetEnvIf User-Agent DOWNLOADER badbot
SetEnvIf User-Agent downloader badbot
<Files *>
Order Deny,Allow
Deny from env=badbot
Allow from all
</Files>

If that code really doesn't work, or if you can't get the server owner to install it in httpd.conf, then you could use mod_rewrite in either httpd.conf or in .htaccess:

RewriteCond %{HTTP_USER_AGENT} ^HARVEST [OR]
RewriteCond %{HTTP_USER_AGENT} downloader [NC]
RewriteRule .* - [F]

You'll need the owner to set (Options FollowSymLinks or Options All) and ((AllowOverride FileInfo and AllowOverride Options) or (AllowOverride All)) for your account in order to use mod_rewrite.

Jim