why is this REWRITE not working

[dupres01]

The intent of the code given below is to block all HEAD requests except those from the IPs listed. Yet IP 199.101.132.166 got a "200".
What am I doing wrong?





RewriteEngine On
RewriteCond %{REQUEST_METHOD} ^HEAD [NC]
RewriteCond %{REMOTE_ADDR} !^64\.68\.[89][0-9]\.
RewriteCond %{REMOTE_ADDR} !^64\.233\.1[6-9][0-9]\.
RewriteCond %{REMOTE_ADDR} !^65\.5[2-5]\.
RewriteCond %{REMOTE_ADDR} !^66\.249\.[6-9][0-9]\.
RewriteCond %{REMOTE_ADDR} !^67\.195\.
RewriteCond %{REMOTE_ADDR} !^72\.14\.[12][0-9][0-9]\.
RewriteCond %{REMOTE_ADDR} !^72\.30\.
RewriteCond %{REMOTE_ADDR} !^74\.125\.
RewriteCond %{REMOTE_ADDR} !^74\.6\.
RewriteCond %{REMOTE_ADDR} !^131\.10[67]\.
RewriteCond %{REMOTE_ADDR} !^207\.46\.
RewriteCond %{REMOTE_ADDR} !^207\.[67][0-9]\.
RewriteCond %{REMOTE_ADDR} !^209\.85\.[12][0-9][0-9]\.
RewriteCond %{REMOTE_ADDR} !^216\.239\.[3-6][0-9]\.
RewriteRule .* - [F]

[wilderness]

for this line try:
RewriteCond %{REQUEST_METHOD} ^HEAD$

[dupres01]

Thanks for the recommendation, wilderness, but it doesn't seem to work. It appears that (with or without the trailing $) any IP can successfully issue a HEAD request. Is my IP "list" incorrect- is it somehow saying "let all IPs through"?

[lucy24]

Do you get the same result if you replace
%{REQUEST_METHOD} ^HEAD
with
%{THE_REQUEST} ^HEAD
?

You may be getting off easy. Cursory search in Apache brings up people getting inexplicable 501s in response to similar directives. ("Inexplicable" here means that nobody responds to the question by screaming "Noooo! You can't do it that way!" ;))

[phranque]

if you have access to your server config you could turn on rewrite logging and see if that shows anything helpful.

[g1smd]

Try:

RewriteCond %{THE_REQUEST} ^HEAD [NC]

[dupres01]

Thanks guys, will try RewriteCond %{THE_REQUEST} ^HEAD [NC] and report back.

[dupres01]

Still does not seem to work. The relevant section of the .htaccess file is:
RewriteEngine On
RewriteCond %{THE_REQUEST} ^HEAD [NC]
RewriteCond %{REMOTE_ADDR} !^[64\.68\.[89][0-9]\.]
RewriteCond %{REMOTE_ADDR} !^[64\.233\.1[6-9][0-9]\.]
RewriteCond %{REMOTE_ADDR} !^[65\.5[2-5]\.]
RewriteCond %{REMOTE_ADDR} !^[66\.249\.[6-9][0-9]\.]
RewriteCond %{REMOTE_ADDR} !^[67\.195\.]
RewriteCond %{REMOTE_ADDR} !^[72\.14\.[12][0-9][0-9]\.]
RewriteCond %{REMOTE_ADDR} !^[72\.30\.]
RewriteCond %{REMOTE_ADDR} !^[74\.125\.]
RewriteCond %{REMOTE_ADDR} !^[74\.6\.]
RewriteCond %{REMOTE_ADDR} !^[131\.10[67]\.]
RewriteCond %{REMOTE_ADDR} !^[207\.46\.]
RewriteCond %{REMOTE_ADDR} !^[207\.[67][0-9]\.]
RewriteCond %{REMOTE_ADDR} !^[209\.85\.[12][0-9][0-9]\.]
RewriteCond %{REMOTE_ADDR} !^[216\.239\.[3-6][0-9]\.]
RewriteRule .* - [F]

Yet this showed up in my log file:
14.160.91.174 - - [07/May/2012:01:24:37 -0600] "HEAD / HTTP/1.0" 200 - "-" "-"

Shouldn't have this request gotten a 403 instead of a 200?

[phranque]

it looks like you have extraneous square brackets around each of your IP address patterns...

[dupres01]

yes, i know they are square brackets. i placed them there after failing to stop head requests, hoping that would cure the problem. it did not.

[lucy24]

To take one at random:

RewriteCond %{REMOTE_ADDR} !^[64\.68\.[89][0-9]\.]

"Offending IP does not begin with anything in the group [64\.8[]9 ... "

Holy ###. Your server didn't melt? I didn't realize you were even allowed to have brackets inside of brackets. Tried it in my text editor-- which is obviously less lethal than putting it into mod_rewrite-- and it didn't object.

Anyway, that Condition alone says "IP cannot begin with a digit", so by using grouping brackets you've effectively wiped out the entire Rule.

[wilderness]

despite the misplaced brackets that you've been previously advised.

I'm wondering if this line is the problem?

RewriteCond %{THE_REQUEST} ^HEAD [NC]


Apache mod-rewrite docs [httpd.apache.org] provide the following explanation:
THE_REQUEST
The full HTTP request line sent by the browser to the server (e.g., "GET /index.html HTTP/1.1"). This does not include any additional headers sent by the browser. This value has not been unescaped (decoded), unlike most other variables below.
end of quote

This implies to me that more criteria is required.
either a page request or some regex for all pages.

There's simply not much available on RewriteCond %{THE_REQUEST} to compare with past practices.

[phranque]

wilderness - that RewriteCond directive is merely testing for a HEAD request.
if THE_REQUEST begins with "HEAD" it's a HEAD request.
there is no end anchor in the regex.

[wilderness]

wilderness - that RewriteCond directive is merely testing for a HEAD request.
if THE_REQUEST begins with "HEAD" it's a HEAD request.
there is no end anchor in the regex.

phranque,
It may be testing for "a begins with", however its not denying the request.
FWIW, if the "ends with anchor" is bad syntax, than the "begins with anchor" is also bad syntax.

The IP ranges are simple straight-up exceptions and I don's see in any errors.

THE_REQUEST line is the only other possibilty for the lines failing.

[wilderness]

dupres,
I suggest the following (incomplete IP ranges) as a TEST ONLY.

Remark out the existing lines and place the following above your mod_rewrite section (prior to RewriteEngine ON:

<Limit HEAD>
Order Deny,Allow
Allow from 157.60.0.0/16
Allow from 157.56.0.0/14
Allow from 157.54.0.0/15
Allow from 65.52.0.0/14
Allow from 66.249.64.0/19
Allow from env=pass
</Limit>

If it works, than add the other IP exceptions, and remove your old mod_rewrite lines for same task.

In summary, I believe this entire task is overkill, as I don't recall ever (more than a decade) seeing the major SE's make HEAD requests.
AOL always did and a few other networks that cache images and/or files make similar HEAD requests, just NOT the major SE's.