Forum Moderators: phranque
I have a long list of IP addresses and CIDRs that are denied access via the <Files *> ... </Files> section in my .htaccess file. I do allow some files to be accessed using "SetEnvIf Request_URI ... allowit" - such as robots.txt, my custom 403, my stylesheet, and certain cgi scripts. After reviewing my recent logs it seems that I am losing a lot of possibly legitimate traffic that is banned because their IP is inside a CIDR range in my deny from lists (because of log spammers, harvesters, etc, from various IP addresses and ISPs).
What I would like to do is instead of blanket banning everybody whose IP is within a range on my blocklist from accessing anything but these few special pages, allow most of them to access all pages except the ones with contact information, email address scripts, or forms, while reserving the * section for the worst offenders. This would let the innocent view not only my technical pages, but also my javascript and cgi driven ads, which they will hopefully click on once in a while. The thought came to me that I might be able to do this with a second <Files> section, using a RegExp group of filenames and extensions.
I Googled on this and found this listed in the Apache Docs ([httpd.apache.org ]):
The filename argument should include a filename, or a wild-card string, where `?' matches any single character, and `*' matches any sequences of characters. Extended regular expressions can also be used, with the addition of the ~ character.
For example:<Files ~ "\.(gif¦jpe?g¦png)$">
would match most common Internet graphics formats.
With this in mind, examine my test directive:
<Files ~ "(about¦contact¦foo¦bar)\.html¦(wxyz?¦abcdef)\.js">
order deny,allow
deny from env=ban
deny from (long list of ISPs IP addresses and CIDRs)
</Files>
Wiz
You could put the "$" on the end of each alternative (i.e. after ".html" and after ".js", or yes, wrap another set of parentheses around it and put the $ at the end. Either way will work.
Apache deprecated <Files ~ "regex"> in favor of <FilesMatch "regex">.
An alternate approach would be to keep the code simaple as before, but be more selective in the size of ranges you block.
Jim
The method should work, but you can have only one Order directive in your file. Just move Order outside of the <Files> containers.
Do I understand correctly that I can only use "order deny,allow" (or allow,deny) once, anywhere, in my .htaccess file? Since all of my <Files> directives use the same order anyway (deny,allow), I should place "order deny,allow" on a separate line, outside of any <Files> directive sections? IOW, this becomes a global directive?
Apache deprecated <Files ~ "regex"> in favor of <FilesMatch "regex">.
Concerning the blocklist, it has grown huge over the lat year as I see more formmail probes, email harvesters, site scrapers, hack attempts, etc. The incidence of zombie computers being used to spam my logs and try to exploit formmail has gone up dramatically. That's why I am blocking entire countries and blackhat ISPs, including one based in Texas.
I'd rather keep the worst offenders (env=ban, log spammers, surveillance bots, and blackhat ISPs) inside the "deny from all" Files section, and move the general population of my blocked IPs to a "deny from a few pages" section.
Thanks again, Wiz