.htaccess banning hosts (not working)

[barns101]

I've recently found that certain hosts (e.g. abc.abc.abc.ny.us) were using a lot more bandwidth than I was happy with, so I implemented the following .htaccess ban:

order allow,deny
deny from ny.us
allow from all

As I understood the manual, this would ban any host ending ny.us

However, upon checking my logs I can still see abc.abc.abc.ny.us making loads of requests and using lots of bandwidth.

Have I missed something?

** Edit:
I know that the server is looking up the host name because they appear in the logs (as opposed to just IP addresses).

[jdMorgan]

A mod_access Deny directive does not stop the remote host's requests from reaching your server and being logged -- Only a firewall can do that.

However, you should see a 403-Forbidden server response code in the access log entries for these requests. If that's not the case, then it's possible that mod_access is not available to you, either because it's not loaded, or because your host has not granted your account permission to use it.

One thing that can help if you are serving a 403 response is to use a very small custom 403 error page to minimize bandwidth wasted -- Set it up as a the smallest valid HTML page possible, including only an "Access Denied" message, and then provide a link to a second, more detailed page explaining the error. In most cases, only humans will 'click' this link.

However, in order to use a custom 403 error page (and the optional 'more info' page described above), you will need to Allow anybody to fetch it. I also suggest you allow anybody to fetch your robots.txt file as well. This can be done with some mod_setenvif code, and a slight change to your Allow/Deny code:

SetEnvIf Request_URI "(403[^.]*\.html¦robots\.txt)$" allowit
SetEnvIf Referer "spam-site\.com" ban
<Files *>
Order Deny,Allow
Deny from env=ban
Allow from env=allowit
#
Deny from 38.0.0.0/8
Deny from ny.us
</Files>

Note that with Order set to "Deny,Allow" an explicit "Allow all" is not needed and must not be used.

Here I assume that your 'more info' page for 403 errors is named "403<something>.html".

The second SetEnvIf directive, the Deny from env=ban, and the IP address CIDR Deny have nothing to do with your question, but are shown for completeness.

Jim

[edited by: jdMorgan at 1:35 pm (utc) on Oct. 11, 2007]

[barns101]

Hi Jim, that's a wonderfully detailed reply, thank you very much. :)

The strange thing is that I tested it by banning my own ISP and it did work. I realise that the requests from the banned host will still show in the logs but there are no 403s. All the requests from abc.abc.abc.ny.us get status 200.

I've just tested it with a well-known server header checker and after banning their host in the same manner, they returned a 403 and this was shown in the server logs.

Any other ideas?

[g1smd]

Are your requests in HTTP/1.0 or HTTP/1.1 format?

[barns101]

Thanks for the reply. It's HTTP/1.1, an example of a request that was denied as follows:

24.213.67.178 - - [13/Oct/2007:06:48:23 +0100] "GET /index.php HTTP/1.1" 403 479 "-" "Mozilla/4.0 (compatible;)"

And a status 200 request as follows:

210.213.141.120.pldt.net - - [13/Oct/2007:11:09:51 +0100] "GET /index.php HTTP/1.1" 200 534 "-" "Mozilla/4.0 (compatible;)"