Welcome to WebmasterWorld Guest from 54.80.87.166

Forum Moderators: Ocean10000 & phranque

Blocking Ranges with Prejudice

     
12:32 am on Aug 9, 2018 (gmt 0)

Preferred Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 489
votes: 43


I block all googleusercontent.com (Google Cloud) ranges with prejudice (allowing some UAs through)

@keyplyr, how do you do this? This statement always confuses me.

If I have a large IP range, for example say from 0-255 and I need to poke a hole for, say 87, I block 0-86 and 88-255, with:
deny from a.b.c.d/16

With CIDR format this gets messy quickly. with combinations of /24, /23, /19 and others. Is there a better way?
1:57 am on Aug 9, 2018 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12841
votes: 881


A simple way...
RewriteCond %{REMOTE_ADDR} ^123\.45\. [OR]
RewriteCond %{REMOTE_ADDR} ^234\.56\.
RewriteCond %{HTTP_USER_AGENT} !(UA-example1|UA-example2)
RewriteCond %{HTTP_REFERER} !(example1\.com|example2\.com)
RewriteRule !^(forbidden\.html|ads\.txt|robots\.txt)$ - [F,L]

I use a multi-tiered set of rules/conditions combining other Blocking Methods [webmasterworld.com]

I have my CIDRs set up to allow different conditions using SetEnvIf.
2:23 am on Aug 9, 2018 (gmt 0)

Preferred Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 489
votes: 43


Thanks. Can you provide an example for your SetEndIfs? I have use htaccess inheritance, so Rewrites are not as useful as SetEndIfs.
3:24 am on Aug 9, 2018 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12841
votes: 881


Sorry no. I don't do what you're asking with SetEndIfs. I have that batch of filters set up to do different things.

IMO, using the IP ranges allows a more surgical alternative, which is why I chose to use it in the example. I can visualize where a range starts/ends easier than I can with CIDRs, plus with ranges, you can do more nesting.
4:11 am on Aug 9, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15198
votes: 682


I rarely use SetEnvIf for this purpose, but when I do, it goes like this:
SetEnvIf Remote_Addr ^1\.2\. bad_range

BrowserMatch NiceGuy !bad_range
Note that mod_setenvif uses Regular Expressions rather than CIDR ranges (unless they've added the functionality in 2.4 and nobody told me), so you use the same syntax as in a RewriteCond, rather than the syntax you'd use in a Deny or Allow line. If you're lucky, it will be an exact /16 or /24 so you don't have to say anything complicated, like
^1\.2\.(1[6-9]|2\d|3[01])
for the equivalent of 1.2.16.0/20. And if you're really lucky it would be a whole /8, like ^52 (to pick a number wholly at random).
10:28 pm on Aug 9, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5496
votes: 3


If I have a large IP range, for example say from 0-255 and I need to poke a hole for, say 87, I block 0-86 and 88-255, with:


Using mod rewrite and poking hole for 87

([0-9]|[1-79][0-9|8[0-689]|1[0-9][0-9]|2[0-5][0-9])
Note; depending upon what CLASS (A,B,C or D) the location of parenthesis and the leading or trailing characters will vary.
10:18 pm on Sept 1, 2018 (gmt 0)

Preferred Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts:489
votes: 43


Here's an AZN range. I want to poke a hole:
23.20.0.0 - 23.23.255.255 23.20.0.0/14 AMAZON
poke a hole for: 23.21.226.191 DuckDuckBot
# 23.20.0.0 - 23.23.255.255 AZN
SetEnvIf Remote_Addr ^23\.2[0-3]\.[0-9]{1,3}\.[0-9]{1,3} bad_azn
# DuckDuckBot 23.21.226.191
SetEnvIf Remote_Addr ^23\.21\.226\.[0-9]{1,3} !bad_azn

or maybe even shorter:
SetEnvIf Remote_Addr ^23\.2[0-3]\. bad_azn

My usual way:
# 23.20.0.0/14
# DuckDuckBot 23.21.226.191
deny from 23.20.0.0/16 23.21.0.0/17 23.21.128.0/18 23.21.192.0/19 23.21.224.0/23 23.21.226.0/32 23.21.226.255/32 23.21.227.0/24 23.21.228.0/22 23.21.232.0/21 23.21.240.0/20 23.22.0.0/15

Is this correct? SetEnvIf is much easier and shorter.
1:09 am on Sept 2, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15198
votes: 682


or maybe even shorter:
Yes, youíre right. The second version is all thatís needed, since there's nothing after that second \. that would not match.

One option is to say
BrowserMatch DuckDuckBot !bad_azn
The other option is your version.

It's a shame Allow/Deny doesn't allow nesting and toggling (Deny from all A except B-within-A, but then do deny from C-within-B-within-A, unless itís D which is within C within B within A). Once we all move on to Require (2.4 and above) syntax, there should be more options.
1:28 am on Sept 2, 2018 (gmt 0)

Preferred Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts:489
votes: 43


Thanks @Lucy24. This method is a lot cleaner and easier than poking holes in my Deny From rules. I will need to rewrite quite a few AZN rules.
6:39 pm on Sept 5, 2018 (gmt 0)

Preferred Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts:489
votes: 43


Can anyone spot an error?
# 52.192.0.0 - 52.223.255.255 52.192.0.0/11 AZN
SetEnvIf Remote_Addr ^52\.19[2-9]\. bad_farm
SetEnvIf Remote_Addr ^52\.2[0-1][0-9]\. bad_farm
SetEnvIf Remote_Addr ^52\.22[0-3]\. bad_farm
# Pinterest 52.201.248.0/24 52.201.249.0/24
# SetEnvIf Remote_Addr ^52\.201\.24[8-9]\. !bad_farm
BrowserMatch Pinterestbot !bad_farm
# XDA 52.201.216.184
BrowserMatch "XDA\ Image" !bad_farm
8:37 pm on Sept 5, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15198
votes: 682


Looks right. You don't actually need to escape spaces inside quotation marks in mod_setenvif--in fact that's the main purpose* of the quotation marks--but I don't suppose it will do any harm.

Edit: In this construction
52\.201\.24[8-9]\.
the final \. is technically superfluous, since nothing else could come after three digits, but you may be retaining it for consistency. Again, does no harm.


* At least, it's the only thing I ever use them for.
1:17 am on Sept 6, 2018 (gmt 0)

Preferred Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts:489
votes: 43


@lucy24 Thanks again for the confirmation and the tip.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members