Effect of changing .htaccess-file

A question about blocking banned IPs

[Snoader]

Suppose, the administrator of a YABBSE-forum uses the following .htaccess file to block banned users:

AuthName "Access Restrictions"
AuthType Basic
<Limit GET POST>
order allow,deny
allow from all
deny from 202.102.3.9
deny from 80.126.25.19
deny from 216.112.42.61
deny from 64.246.11.20
</Limit>

We discovered that, when he adds an IP-address to this file, hits from this added IP-address will go on for a while.

Is this normal, and if not, what can we do about it? Is it possible to instantly deny access from a certain IP-address?

Thanks in advance!

Sander

[BlueSky]

Welcome to WebmasterWorld

A log entry will always be generated each and every time a person accesses a page even for those who are banned. What you guys need to look for is the code on the log entries to see whether or not these folks are getting content or an error message. If your block is working correctly, I think you'll see a 401 (since you're asking for a login) then maybe a 403 not a 200 or 304. Some banned folks get really mad and could be pounding on your site for awhile until they get tired and go away. Each time they request a page, you'll see a log entry.

I'm not sure why you have these two lines.

AuthName "Access Restrictions"
AuthType Basic

If you're allowing everyone except certain IPs, why not delete those lines and just immediately feed the banned guys 403's instead?

[Snoader]

Yeah, this makes sense. We'll try this out later this evening, and I'll let you know how this works out for us.

Thanks! :)

[Snoader]

The file now looks like this:

<Files *>
order allow,deny
allow from all
deny from 202.102.3.9
deny from 80.126.25.19
deny from 216.112.42.61
deny from 64.246.11.20
deny from 212.76.68.
</Files>

But we still see entries in the forum-errorlog from 212.76.68.*!

We don't want 212.76.68.* to cause any traffic, what are we doing wrong?!?

Please help...

[closed]

Just like BlueSky said earlier, as long as members are trying to access your site, you'll see entries from them in your logs. You have to check what kinds of hits you're getting. It would be a lot more helpful if you gave more details about what you get from your forum's error log.

[Snoader]

The .htaccess now looks like this again:

<Limit GET POST>
order allow,deny
allow from all
deny from 202.102.3.9
deny from 80.126.25.19
deny from 216.112.42.61
deny from 64.246.11.20
</Limit>

This should affect the whole site (it's in the root), am I right?

Suppose a new IP is added to the .htaccess (deny from), I still can't imagine why someone using this IP is able to trigger forum-software on the site. I do not have access to site-logs, only to the forum-logs.

What I can see in the forum access-log is dat this IP is still able to make calls to forum pages, and sometimes (when a lot of calls are made within a short period of time), error-entries are added to the error log, because of lost connections to the database. We can ban this IP by using a forum-ban, and then it's logical to me that this IP still generates entries in the forum access log (because he's hitting the forum and is getting "You are banned" answers).

Why doesn't adding this IP to the .htaccess prohibit this guy reaching the forum? If he's not allowed to enter the site, why can he reach the forum-software which is running on the site?

And then another question.

From the forum-logs it has become clear that his user-agent ID is "Mozilla/4.7 [en]". It should be possible to disallow access for this type of agent by using the following lines:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4.7 [en]$ [NC]
RewriteRule ^.*$ - [F]

But, when the site-owner adds this to his .htaccess, we get an Internal Server Error (500). Yesterday, I've been browsing through Apache .htaccess docs, but I can't figure out what's wrong with these 3 lines... Does the position in the .htaccess file of these 3 lines have anything to do with it?

Someone?

[jdMorgan]

Snoader,

You have included reserved characters in your code. These characters must be 'escaped' by preceding them with a slash:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.7\ \[en\]$ [NC]
RewriteRule ^.*$ - [F]

You may also need to add "Options +FollowSymLinks" in front of the code above. If neither of these helps, then your server may not support mod_rewrite.

> deny from 212.76.68.

Leave off the trailing period. It should be

deny from 212.76.68

> This <deny code> should affect the whole site (it's in the root), am I right?

Yes, you are right. The point that several posters have tried to make is that .htaccess cannot stop anyone from *requesting* a page. It can only stop the server from *serving* the requested page. If you cannot see your raw access logs, then there may be no way to tell if the request was successful - that is, returns a 200-OK response and the requested page, or if the request was denied, and the server responded with 403-Forbidden and an error page.

The only way to stop requests is to black-hole them at the router. Otherwise, the server will always respond, whether with the requested resource, or with an error page.

If the code above is not working, then there is a problem with the server set-up, or it is not set up as you think it is. Many of us use these directives to block hundreds of "bad IPs" from accessing our content, and the method works fine.

HTH,
Jim

[Snoader]

Thank you for answering, jdMorgan!

It took a while to determine whether HTTP_USER_AGENT Mozilla 4.7 [en] was denied access to the forum, but today it showed up again in the forum-log.

Also, several kinds of IP-addresses which are denied showed up.

This means, that our .htaccess doesn't do the job. :(

Here's a small part:

Options +FollowSymLinks

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.7\ \[en\]$ [NC]
RewriteRule ^.*$ - [F]

<Limit GET POST>
order deny,allow
deny from .qa
deny from .sa
deny from .vn
deny from .ye
deny from 12.148.209.198
deny from 157.91.76.65
deny from 163.24.133.117
deny from 164.58.18.25
deny from 164.58.28.250
...
deny from yuluma.wa.edu.au
allow from all

</Limit>

Is

"deny from .qa"

a valid entry?

What's 'better', using

"order deny,allow"

or using

"order allow,deny"

?

If .htaccess is correct, but doesn't work, what should we check (or have checked) on the server? MOD_ACCESS and MOD_REWRITE are present on the server.

I heard something about httpd.conf, has this file something to do with the functioning of .htaccess?

Thanks in advance!
Sander

[BlueSky]

Please post one of the entries from your server log (not the forum log) which shows this person accessed pages on your forum.

[Snoader]

The problem is, that the webmaster does not have access to the server-logfiles.

At least, that's what he says. Maybe he's wrong.

In which directory should he be able to find this logfile? And what's the name of this logfile?

[jdMorgan]

Snoader,

> Is "deny from .qa" a valid entry?

Denies access from hosts in Quatar.

> What's 'better', using "order deny,allow" or using "order allow,deny"?

For your case, "Order allow,deny" is required: [httpd.apache.org...]

> In which directory should he be able to find this logfile? And what's the name of this logfile?

It varies wildly. Ask your Web host.

Jim

[Snoader]

Thanks Jim, 'allow,deny' does the trick. Someone had assured us that it had to be 'deny, allow'...

Hopefully my last question: the following part doesn't have any effect, what can be the reason?

===
Options +FollowSymLinks

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.7\ \[en\]$ [NC]
RewriteRule ^.*$ - [F]
===

MOD_REWRITE is present on the server.

Kind regards,
Sander

[jdMorgan]

The most likely cause is that your pattern does not match the actual user-agent that is being used. Going through my (admittedly non-comprehensive) records, I found no cases where anyone used a user-agent of exactly "Mozilla/4.7 [en]".

I *have* found several that use variations such as "Mozilla/4.7 [en]C-CCK-MCD <using organization> (Win95; U)" but your pattern won't match those because you have end-anchored [etext.lib.virginia.edu] it right after "[en]".

Note on the Order issue: When dealing with .htaccess/httpd.conf directives, don't take anyone's assurances, except for the source [httpd.apache.org]. It's all there and it's free.

Jim

[richmondsteve]

When using mod_rewrite, instead of waiting for the desired IP or user agent to visit, test using your own IP or user agent. Then modify to suit your needs after you know that the rules in place work properly.

BTW, this is my first time posting to this forum. I've been posting to the AdSense forum for some time, but so you know a little about my background I've been doing Linux server administration professionally since 1999.