Blocking Ranges with Prejudice

[TorontoBoy]

I block all googleusercontent.com (Google Cloud) ranges with prejudice (allowing some UAs through)

@keyplyr, how do you do this? This statement always confuses me.

If I have a large IP range, for example say from 0-255 and I need to poke a hole for, say 87, I block 0-86 and 88-255, with:

deny from a.b.c.d/16

With CIDR format this gets messy quickly. with combinations of /24, /23, /19 and others. Is there a better way?

[keyplyr]

A simple way...

RewriteCond %{REMOTE_ADDR} ^123\.45\. [OR]
RewriteCond %{REMOTE_ADDR} ^234\.56\. 
RewriteCond %{HTTP_USER_AGENT} !(UA-example1|UA-example2)
RewriteCond %{HTTP_REFERER} !(example1\.com|example2\.com)
RewriteRule !^(forbidden\.html|ads\.txt|robots\.txt)$ - [F,L]

I use a multi-tiered set of rules/conditions combining other Blocking Methods [webmasterworld.com]

I have my CIDRs set up to allow different conditions using SetEnvIf.

[TorontoBoy]

Thanks. Can you provide an example for your SetEndIfs? I have use htaccess inheritance, so Rewrites are not as useful as SetEndIfs.

[keyplyr]

Sorry no. I don't do what you're asking with SetEndIfs. I have that batch of filters set up to do different things.

IMO, using the IP ranges allows a more surgical alternative, which is why I chose to use it in the example. I can visualize where a range starts/ends easier than I can with CIDRs, plus with ranges, you can do more nesting.

[lucy24]

I rarely use SetEnvIf for this purpose, but when I do, it goes like this:

SetEnvIf Remote_Addr ^1\.2\. bad_range

BrowserMatch NiceGuy !bad_range

Note that mod_setenvif uses Regular Expressions rather than CIDR ranges (unless they've added the functionality in 2.4 and nobody told me), so you use the same syntax as in a RewriteCond, rather than the syntax you'd use in a Deny or Allow line. If you're lucky, it will be an exact /16 or /24 so you don't have to say anything complicated, like

^1\.2\.(1[6-9]|2\d|3[01])

for the equivalent of 1.2.16.0/20. And if you're really lucky it would be a whole /8, like ^52 (to pick a number wholly at random).

[wilderness]

If I have a large IP range, for example say from 0-255 and I need to poke a hole for, say 87, I block 0-86 and 88-255, with:

Using mod rewrite and poking hole for 87

([0-9]|[1-79][0-9|8[0-689]|1[0-9][0-9]|2[0-5][0-9])
Note; depending upon what CLASS (A,B,C or D) the location of parenthesis and the leading or trailing characters will vary.

[TorontoBoy]

Here's an AZN range. I want to poke a hole:
23.20.0.0 - 23.23.255.255 23.20.0.0/14 AMAZON
poke a hole for: 23.21.226.191 DuckDuckBot

# 23.20.0.0 - 23.23.255.255 AZN
SetEnvIf Remote_Addr ^23\.2[0-3]\.[0-9]{1,3}\.[0-9]{1,3} bad_azn
# DuckDuckBot 23.21.226.191
SetEnvIf Remote_Addr ^23\.21\.226\.[0-9]{1,3} !bad_azn

or maybe even shorter:

SetEnvIf Remote_Addr ^23\.2[0-3]\. bad_azn

My usual way:

# 23.20.0.0/14
# DuckDuckBot 23.21.226.191
deny from 23.20.0.0/16 23.21.0.0/17 23.21.128.0/18 23.21.192.0/19 23.21.224.0/23 23.21.226.0/32 23.21.226.255/32 23.21.227.0/24 23.21.228.0/22 23.21.232.0/21 23.21.240.0/20 23.22.0.0/15

Is this correct? SetEnvIf is much easier and shorter.

[lucy24]

or maybe even shorter:

Yes, you’re right. The second version is all that’s needed, since there's nothing after that second \. that would not match.

One option is to say
BrowserMatch DuckDuckBot !bad_azn
The other option is your version.

It's a shame Allow/Deny doesn't allow nesting and toggling (Deny from all A except B-within-A, but then do deny from C-within-B-within-A, unless it’s D which is within C within B within A). Once we all move on to Require (2.4 and above) syntax, there should be more options.

[TorontoBoy]

Thanks @Lucy24. This method is a lot cleaner and easier than poking holes in my Deny From rules. I will need to rewrite quite a few AZN rules.

[TorontoBoy]

Can anyone spot an error?

# 52.192.0.0 - 52.223.255.255 52.192.0.0/11 AZN
SetEnvIf Remote_Addr ^52\.19[2-9]\. bad_farm
SetEnvIf Remote_Addr ^52\.2[0-1][0-9]\. bad_farm
SetEnvIf Remote_Addr ^52\.22[0-3]\. bad_farm
# Pinterest 52.201.248.0/24 52.201.249.0/24
# SetEnvIf Remote_Addr ^52\.201\.24[8-9]\. !bad_farm
BrowserMatch Pinterestbot !bad_farm
# XDA 52.201.216.184
BrowserMatch "XDA\ Image" !bad_farm

[lucy24]

Looks right. You don't actually need to escape spaces inside quotation marks in mod_setenvif--in fact that's the main purpose* of the quotation marks--but I don't suppose it will do any harm.

Edit: In this construction

52\.201\.24[8-9]\.

the final \. is technically superfluous, since nothing else could come after three digits, but you may be retaining it for consistency. Again, does no harm.


* At least, it's the only thing I ever use them for.

[TorontoBoy]

@lucy24 Thanks again for the confirmation and the tip.