CIDR vs Mod Rewrite

[wilderness]

A few years back (maybe not that long) I converted many of my IP mod_rewrite lines to Deny From and CIDR (for faster access).

Over the past few days I'm getting IP's that have been denied for the longest time, with instant access (and repeated access). All of which are clearly designated in lines.
Many Amazon.
The very first line of that section was
Deny from 1. 2. 5.

FWIW, I've seen syntax errors in mod_rewrite that result in weird application of ranges (some not detected for months or even years), however don't recall seeing those same weird applications of IP's using deny from.

Poured over the deny from lines (there are many) and was unable to locate any significant syntax errors (there were a few lines with extra spaces at the line-end, however they've not created any issues in the past).

Give the ongoing discussions regarding Apache 2.2 to Apache 2.4?
I contacted my host (my host a reseller for a major hosting company) to see if perhaps this update had taken place in the last few days?
They replied NO.

Tech support at hosting companies rarely comprehend an extensive (much less complicated htaccess beyond a few lines; much less regex).
Their only solution was to upload a BU file to which I responded a firm NO. (I'd already uploaded an older file absent any improvement)
I also asked to host if there had been any system-wide changes in http.conf and they replied NO.

Summary, I'm fact with the daunting task (already begun) of re-converting the CIDR lines back to mod_rewrite. (I've already seen these access' growing over the past few days and I'm not waiting to determine whether is a host-server syntax error that may be resolved in the distant future.

There's rarely any talk these days of using mod_rewrite for IP ranges, thus these issues shouldn't hinder many (at least hopefully).

[lucy24]

Sure, you can do global replaces. There will be a lot of them, though.

(\d+\.\d+) (i.e. a CIDR /16 block) >> ^$1\.

(\d+\.\d+\.)0\.0/17 >> ^$1(12[0-7]|1[01]\d|\d\d?)\.
(\d+\.\d+\.)128\.0/17 >> ^$1(12[89]|1[3-9]\d|2\d\d)\.

and then four global replaces for /18, eight for /19 and so on. The anchoring and escaping in the targets is not a mistake; it's because they will be going into your htaccess that way. Since you're doing ranges, you can't use the =literaltext alternative.

Moving to 2.4 should not be an issue, because initially it will come with mod_compat (don't quote me on the exact name) so Allow/Deny directives will continue working.

I strongly recommend constraining most access-control rules to requests for pages, so the server doesn't get bogged down on every human visit.

But really, I'd look for alternative solutions a/o explanations before changing over to this wholesale shooting-flies-with-an-elephant-rifle approach. Is everyone from blocked ranges now getting in, or is there some subset, and if so, what's the unifying feature? URI, range, or something else?

[wilderness]

Is everyone from blocked ranges now getting in, or is there some subset, and if so, what's the unifying feature? URI, range, or something else?

Many thanks lucy.

I'm inclined to reply with a NO, however I'm not doing a whois (or searching my data) for every IP. Rather, there are just some ranges that JUMP out at me, and whenI check they are included in the deny from.

as to the deny from 1. 2. 5.
I've had multiple requests from the 1's and 5's over the past few days. Those changes (from CIDR to mod_rewrite) may be the easiest to catch in the logs (as well as the Amazon ranges).

I'm going to hold off on proceeding further with the CIDR-mod_rewrite conversion until I'm able to confirm that the changes I've made already are solving the issue.

[lucy24]

deny from 1. 2. 5.

Is your actual "Deny from" statement written that way, with trailing . dot after each number? Try deleting the dot and see if anything changes.

I keep forgetting that 1 is a legitimate IP range. Asia, I guess; I never see them. Of course there are plenty of 5, though it mainly seems to be Russian robots.

:: detour to raw logs ::

Oh, my. A lot of robots, but also plenty of humans from 1.a.b.c

[keyplyr]

IMO you may just have a few CIDRs wrong. What are you using to convert the IP addresses to CIDR?

There's a tool here: [ipaddressguide.com...] to check if your CIDR is accurate,

You may want to reconsider your frustration and stay with mod_access since mod_rewrite is much more server intensive.

[edited by: keyplyr at 3:56 am (utc) on Mar 25, 2016]

[wilderness]

Online IP CIDR Supernet Calculator

keyplr,
FWIW, I've had deny from's in place for more than 15-years.
I've never seen an IP error in ''deny from' corrupt the process of the entire file?

I've seen over and again, syntax errors in mod_rewrite corrupt an entire file.

[keyplyr]

FWIW, I've had deny from's in place for more than 15-years.
I've never seen an IP error in ''deny from' corrupt the process of the entire file?

That's not what I said.
I suggested that the UA getting through, where you assumed you had them blocked via CIDR, might be because the CIDR was not actually covering the entire range you thought it was. Since you could not find any syntax errors, this would be the next thing I would look at.

Another thing to consider is the line-wrapping in your text editor. It may appear you have a space between each CIDR, but when the line_wrap is removed, it may not be (an extra space or two doesn't matter.)

[wilderness]

That's not what I said.
I suggested that the UA getting through, where you assumed you had them blocked via CIDR, might be because the CIDR was not actually covering the entire range you thought it was. Since you could not find any syntax errors, this would be the next thing I would look at.

Another thing to consider is the line-wrapping in your text editor. It may appear you have a space between each CIDR, but when the line_wrap is removed, it may not be (an extra space or two doesn't matter.)

Thanks I've already checked the line-wraps.
I've line-wrap turned off in multiple tools (including my text editor; Notepad)

FWIW, before proceeding with any editing of files?
I restored a file from approximately one-month-ago. There was no change.

If I've the entire Class A of 1's and 5's denied with a simple (solitary) line:
deny from 1. 2. 5.

Yet there have been an unusual amount of theses subnets gaining access?
Where's the error? It's the first line in the CIDR's

Amazon ranges are something that JUMP out at me and seeing them gain access when I know they've been previously denied is not an oversight or CIDR error.

Thanks for the suggestion though.

[wilderness]

Note the following line (complete) has been in place for some years:
Deny from 185. 186. 187. 188. 189.

The 2d realization that IP's were getting in came with the following
188.29.31.zzz

[keyplyr]

IPv5?

[lucy24]

Did you try taking out the trailing dots? They're unneeded at best (because this isn't a RegEx, so "5" means only that, regardless, and not 125 or 53 or anything else with 5 in it). If nothing else, you'll save a few bytes. fwiw, the docs for mod_authzwhatsit don't use trailing dots, though admittedly there are things in the docs I would say differently.

but when the line_wrap is removed, it may not be

? Unless your line wrapping is set to "by character" instead of "by word"-- and how often do people need to do that?-- lines will only wrap when there's a space.

[keyplyr]

Nature of some text editors

[wilderness]

Did you try taking out the trailing dots? They're unneeded at best (because this isn't a RegEx, so "5" means only that, regardless, and not 125 or 53 or anything else with 5 in it). If nothing else, you'll save a few bytes. fwiw, the docs for mod_authzwhatsit don't use trailing dots, though admittedly there are things in the docs I would say differently.

Many thanks lucy.
I've used trailing dots since the beginning (2000 or 2001).
All the examples for deny from at that time suggested so.
Same applies to 'deny from' or 'Deny From'.

The few bytes are insignificant.
What really matters is consistency in procedures, which as you know makes looking for errors easier..

[wilderness]

Update.

After six days (with minimal changes) of pouring over htaccess?
I've gone over each IP individually in mod_authz_host (i. e., deny from) to checked syntax.
Despite finding what I believed were three significant syntax errors and then correcting (and waiting for effect).
Last night, and than again this morning two IP's that were clearly denied (Hell! One was on the first line; again) in mod_authz_host. (This has happened so many times in the past eight days; and continues despite my efforts to locate an error with cause on my end).

I'm still inclined to believe this is a host and/or Apache update issue, unfortunately I'm weary of playing!
I'm abandoning mod_authz_host and CIDR and reconverting all those ranges to mod_rewrite.