SetEnvIf with Apache 2.4

[JamesSC]

So I have finally roused myself to quit relying on my host's mod_compat module and to convert my .htaccess file over from 2.2 to 2.4.

The mod_authz stuff seems straightforward: Allow/Deny --> Require Any/All / Require not, although I'm going to have to study the possibilities of nesting commands more thoroughly.

At the same time, though, I'm also going to be converting some portion of my mod_authz Allow/Deny stuff to mod_setenvif in order to get more granular control.

Right now, I'm already using mod_setenvif to cut holes for DuckDuckBot in a block against AWS ranges. My current Apache 2.2

SetEnvIf Remote_Addr ^107\.20\.0\.0\/14 bad_range
SetEnvIf Remote_Addr ^107\.21\.1\.8$ !bad_range # DuckDuckBot
deny from env=bad_range

(There's an Allow from command already in place above where the deny from env=bad_range is placed.)

My new Apache 2.4 configuration for the above and for using mod_setenvif to control any other IP range or address under 2.4, as I understand it, should be

SetEnvIf Remote_Addr ^107\.20\.0\.0\/14 bad_range
SetEnvIf Remote_Addr ^107\.21\.1\.8$ !bad_range # DuckDuckBot
<RequireAll>
Require not env=bad_range
</RequireAll>

My questions are, before I blow things up, is the above new 2.4 configuration correct, and is the RequireAll envelope even necessary? My understand as to the latter is that it is necessary, taking the place of the prior 2.2 Allow from all preceding any subsequent Deny from commands.

Thanks.

[lucy24]

Heh. When my host finally moved up to 2.4, the SetEnvIf directives were the one thing I didn't have to change at all. There are some new ones, like SetEnvIfExpr, but all the old directives are unchanged.

When it comes to Require, you can either have a long list of “Require not”, OR you can make a RequireNone envelope and then omit the “not” from each line. So it would look like

<RequireAll>
Require all granted
<RequireNone>
Require env keep_out
Require env bad_protocol
Require env bad_range
[et cetera]
</RequireNone>
</RequireAll>

The preliminary “Require all granted” is equivalent to "Allow from all” in the old Allow,Deny ordering.

Not long ago I counted on my fingers and established just how many “Require” lines you need before you start saving bytes by using the <RequireNone> envelope, but really I think it's more a matter of personal coding style.

The bad_range construct is useful if you need to poke holes: block the whole range except suchandsuch named user-agents, or some smaller subset of the range. If you're unconditionally blocking a whole range, you can say

Require ip 37.187
Require ip 45.145

and so on--assuming a RequireNone envelope--exactly the way you used to say “Deny from 37.187” and so on. In fact, if you have a whole lot of them you can just globally replace in a text editor.

The RequireNone envelope has to be inside a RequireAll envelope, or your server will explode.

Edit:

SetEnvIf Remote_Addr ^107\.20\.0\.0\/14 bad_range

Waitwaitwait? You can do this? I thought SetenvIf worked only with Regular Expressions, in this case
^107\.2[0-3]\b
(In this situation, either \. or \b will work.)

[JamesSC]

Hey, Lucy,

So let me understand these one by one.

First, are you saying you're putting the

<RequireAll>
Require all granted

directive at the top of the entire file, or at least above any areas containing subordinate SetEnvIf directives, with

</RequireAll>

bringing up the rear, and similarly with the next nested envelope tier, in this case

<RequireNone>
</RequireNone>

If, as I understand it, Apache runs each module separately - all SetEnvIf directives, then all mod_rewrite, etc., it seems there wouldn't be any reason not to so long as all internal directives were consistent with their outer envelopes. If true, this would certainly be more efficient, it would seem, and pare server load by whatever it did.

Waitwaitwait? You can do this?

Like the bumblebee, apparently, at least on 2.2. Since it flies, I haven't yet had any reason to think it shouldn't.

While we are trading inexplicables, it happened for some reason that, at the moment my host implemented http/2 my header control section blew up the server. I determined that by stripping my .htaccess file to the bone, then adding back each section until the header controls proved to be the culprit. I haven't yet added them back.

The 2.2 configuration was/is:

Order Allow,Deny
Allow from all

SetEnvIf Accept ^$ no_accept
SetEnvIf Accept-Charset ^$ no_accept-charset
SetEnvIf Accept-Encoding ^$ no_accept-encoding
SetEnvIf Accept-Language ^$ no_accept-language
SetEnvIf Connection ^$ no_connection

BrowserMatchNoCase bingbot !no_accept
BrowserMatchNoCase bingbot !no_accept-charset
BrowserMatchNoCase bingbot !no_accept-encoding
BrowserMatchNoCase bingbot !no_accept-language
BrowserMatchNoCase bingbot !no_connection

Deny from env=no_accept
Deny from env=no_accept-charset
Deny from env=no_accept-encoding
Deny from env=no_accept-language
Deny from env=no_connection

I don't know why the above wouldn't work with whatever accompanied the changeover to http/2, but at the time it was only a nascent part of my control efforts, so I elected not to poke the bear further and since just haven't gotten back around to it.

[JamesSC]

Also, and I should have asked this above, is there any reason SetInvIf would dislike a configuration like

SetEnvIf Remote_Addr (^107\.20\.0\.0\/14|^23\.20\.0\.0\/14|^50\.16\.0\.0\/14|etc) bad_range

similar to the

Require not 123.123.123.123 123.123.123.123 123.123.123.123 123.123.123.123

often used in controlling IP ranges?

[lucy24]

If, as I understand it, Apache runs each module separately - all SetEnvIf directives, then all mod_rewrite, etc., it seems there wouldn't be any reason not to so long as all internal directives were consistent with their outer envelopes.

Since each module is an island, the server doesn't care what order you put things in. You could even interlace directives from a bunch of different modules; the server will ignore everything except the mod it's currently working on. (I do not, of course, recommend that you do this. Just saying you could if you wanted to. If, say, the site administration was about to be taken over by someone you dislike and you wanted to get them off to a miserable start.)

Most people find it most practical, convenient and sanity-saving to group each module's directives together. And, although it makes no functional difference, it may be intuitively easier to put the modules in execution order, with SetEnvIf before AuthzWhatsit.

I currently have one big RequireAll envelope, which functionally contains two things: the “Require all granted” line and a RequireNone envelope, which in turn contains all the lines that used to be “Deny from”. I don't think the server cares if you have more than one envelope of the same kind--but generally why would you?

Like the bumblebee, apparently, at least on 2.2.

Slowly, clumsily and loudly, but it works? I’ll be darned. I’ll try it on my test site one day. Incidentally, I doubt you need to escape the / slash, although it does no harm. Generally you only need to escape it if the / slash itself serves as a RegEx delimiter, as in javascript and one or two obscure Apache mods that I don't personally use.

is there any reason SetInvIf would dislike a configuration like

Dunno, never tried it :) But I really doubt it would make the server throw a fit. At worst it just wouldn't work, so go ahead and try. And you could almost certainly put the ^ opening anchor outside the parentheses, since it applies to every item.

at the moment my host implemented http/2 my header control section blew up the server. I determined that by stripping my .htaccess file to the bone, then adding back each section until the header controls proved to be the culprit

How odd and alarming. Mine went to http/2 a while back but I don't think anything changed. Possibly some detail of header logging, but then again that might be a php upgrade that just happened to come around the same time. Did you check each header item individually? Maybe there's some particular header that http/2 dislikes.

Edit: In general, I wouldn't use BrowserMatchNoCase. It's more work for the server, since it has to do the extra step of flattening the case on both sides, and most of the time if something is wrongly cased it's a faker. (For a while there was a fairly active “GoogleBot”, for example.) It might be appropriate in disallows, like
BrowserMatchNoCase stupidevilbot bad_agent
so the unwanted robot can't try to outsmart you by changing its name to StupidEvilBot next week.

[JamesSC]

Thanks, Lucy. The bumblebee is supposed to be incapable of flight, and yet it does.

How odd and alarming. Did you check each header item individually? Maybe there's some particular header that http/2 dislikes

Yeah, that's what I'm going to have to do, as long as I'm revamping the whole thing anyway.

So just BrowserMatch, then, except for shapeshifters.

Incidentally,

SetEnvIf Remote_Addr (^107\.20\.0\.0\/14|^23\.20\.0\.0\/14|^50\.16\.0\.0\/14|etc) bad_range

throws a 500. I really prefer horizontal for long strings of entries, like AWS IPs; I don't like reading my own .htaccess file like a scroll.

[dstiles]

DuckDuck has a wider range of IPs than you've shown; prehaps you are not showing them all in your example. They have a list of IPs on their site. I use...

<if " -R '23.21.227.69' || -R '40.88.21.235' || -R '50.16.241.113' || -R '50.16.241.114' || -R '50.16.241.117' || -R '50.16.247.234' || -R '52.5.190.19' || -R '52.204.97.54' || -R '54.197.234.188' || -R '54.208.100.253' || -R '54.208.102.37' || -R '107.21.1.8' ">
 SetEnvIfExpr "%{REMOTE_ADDR} =~ /(.+)/" ips=duck:$0
 BrowserMatch DuckDuckBot duck bot=duck
 BrowserMatch DuckDuckGo-Favicons-Bot duck bot=iconduck
 Require env duck
</if>

The setenvifexpr and "bot=" bits are for my own logging. Most of my handling is in a file included in the site's config rather than in htaccess. That means I only maintain one file.

[JamesSC]

Thanks, dstiles. Yes, I had all of the ducks but the 40; thanks for the add.

Frankly, I've never seen that SetEnfIfExpr lateral construction using the double pipes before; that may be what I'm looking for. I'm going to have to dig more deeply into that as an option

I suppose being able to string 50 IPs out horizontally as I can in 2.2 is too much of a convenience to hope for; I haven't yet seen it replicated in the 2.4 Require directives.

[lucy24]

I haven't yet seen it replicated in the 2.4 Require directives

You can still say things like

Require ip 51.15 51.68 51.91 51.161

exactly like the former “Deny from” construction. (What I, personally, like to do is group each /8 on a single line.) Note the quirk that, since this is not a RegEx, “15” only means .15 and not .151 and so on.

[JamesSC]

Ah, good. Thanks, Lucy. You've spared Support another of my sometimes unpleasant experiments.

One would think someone so inclined would have already built an Apache Docs Fully Illustrated Web site where this sort of material would all be collected in one place. Instead, one gets a million sites all providing the same basic, simple examples - "Allow from all Deny from 123.123.123.123" and little else.

[lucy24]

Not everything is in the apache docs, but this one is:

http://httpd.apache.org/docs/2.4/mod/mod_authz_host.html#requiredirectives

(Code markup so the fragment is retained, and fortunately “Require ip” with all its variants comes first.)

[JamesSC]

Thanks, Lucy. This is why God made women, so that men wouldn't just say "Eff-it", pull the trigger, and crash the server.

In all seriousness, I don't fault Apache at all for being relatively taciturn purists about their own documentation; again, it just seemed to me that, with all the possibilities the Internet offers even unto Tik-Tok dancing, this still remains remarkably unexploited fertile ground.

[lucy24]

More mundanely: This is why God made test sites. The server can explode, melt down, or go into a corner to sulk, and nobody is hurt.

Just remember that when Apache docs say “Unintended consequences”, it is liable to mean that the world as we know it will come to an end.

[JamesSC]

Yes, well, it's been the usual smacking a rhino across the snout with a mackerel. Most of the problems have been unrelated typos. I should also mention I'm not mixing 2.2 directives with 2.4 directives.

But there's clearly something I'm still missing with regard to applying Require directives to mod_setenvif.

This in an entirely alternate (i.e., unmixed) .htaccess file works perfectly

 Order Allow,Deny
 Allow from all

  BrowserMatchNoCase 2icommerce bad_bot
  BrowserMatchNoCase 4SeoHuntBot bad_bot
  BrowserMatchNoCase 7Siters bad_bot
  BrowserMatchNoCase 80bot bad_bot
  BrowserMatchNoCase 80legs bad_bot
  BrowserMatchNoCase 8LEGS bad_bot
  BrowserMatchNoCase A6-Indexer bad_bot

 Deny from env=bad_bot

This blows up:

 <RequireAll>
 Require all granted
 <RequireNone>

  BrowserMatchNoCase 2icommerce bad_bot
  BrowserMatchNoCase 4SeoHuntBot bad_bot
  BrowserMatchNoCase 7Siters bad_bot
  BrowserMatchNoCase 80bot bad_bot
  BrowserMatchNoCase 80legs bad_bot
  BrowserMatchNoCase 8LEGS bad_bot
  BrowserMatchNoCase A6-Indexer bad_bot

  Require env=bad_bot

 </RequireNone>
 </RequireAll>

It also blows up if I use the simpler

 <RequireAll>
 Require all granted

  BrowserMatchNoCase 2icommerce bad_bot
  BrowserMatchNoCase 4SeoHuntBot bad_bot
  BrowserMatchNoCase 7Siters bad_bot
  BrowserMatchNoCase 80bot bad_bot
  BrowserMatchNoCase 80legs bad_bot
  BrowserMatchNoCase 8LEGS bad_bot
  BrowserMatchNoCase A6-Indexer bad_bot

  Require not env=bad_bot

 </RequireAll>

Nor does the = sign make any difference. Rather than one big envelope I'm using a few smaller ones, each self-contained, and the ones controlling IPs are working fine

There's just something in the Require - SetEnvIf syntax I'm still missing and haven't been able to track down.

[lucy24]

Nooooo!

All those BrowserMatch directives are mod_setenvif. They do NOT go inside the Require envelope, which is mod_authwhatever.

Put all the SetEnvIf stuff outside the Require envelopes. Since each module is an island, it doesn't matter if they go before or after, but most people will find it more intuitive to put them first, matching the mods' execution order.

[JamesSC]

Oh...kaaay ::Wile waits patiently as Roadrunner drops anvil on head::

So a setenvif retains the orginal 2.2 Allow/Deny/Deny from language, though outside the Require envelope(s)?

Msg 5029161 above led me to believe they (mod_authz_host & mod_setenvif) effectively merged in practical application in 2.4, with IPs taking Require (not) ip and setenvifs taking Require (not) env var_iable as replacements for the prior common Deny from, regardless that one might be read on a separate pass some nanoseconds after the other.

[lucy24]

They remain two entirely separate things.

mod_setenvif (SetEnvIf, BrowserMatch and so on) is basically the same between 2.2 and 2.4, except that 2.4 adds some extra stuff. It executes before mod_authwhatever, which is what lets you use environmental variables in access control.

The 2.2 “Allow/Deny” directive is replaced by the 2.4 “Require” directive. (Setting aside the mod_compat option.) Each of course uses its own syntax, but fundamentally you're still saying the same thing: allow/deny suchandsuch IP range, or allow/deny suchandsuch environmental variable.

[JamesSC]

Okay, thanks, Lucy, I think I've got it now.

I was trying to place my mod_setenvif carveouts adjacent to my mod_authz_host CIDR ranges in an attempt to simplify and foolproof things whenever I wanted to turn a simple CIDR block into a SetEnvIf bad_range I could then carve a !bad_range exception out of, but that simply won't work, it seems.

The only thing I'll be able to do is place a duplicate server farm or geographic region within its own, separate SetEnvIf section, remove the CIDR I want to carve out from its mod_authz_host server farm location to its new, duplicate mod_setenvif server farm location and deal with it there.

Even though I've turned up from elsewhere virtually identical code to yours above (ignoring the extra/different envelopes)

<Directory /var/www/html>
<RequireAny>
 <RequireAll>
 Require all granted
 Require not ip 104.197.51.76
 Require not ip 54.242.250.203
 Require not env bad_bot
 Require not env spam_ref
 </RequireAll>

 <RequireAny>
 Require ip 131.253.24.0/22
 Require ip 131.253.46.0/23
 Require env good_ref
 Require env good_bot
 </RequireAny>

</RequireAny>
</Directory>

apparently mixing mod_authz_host IP directives with mod_setenvif directives within Require envelopes and declaring itself the definitive new 2.4 way, nothing like that will work on my server.

I can only use the identical Apache 2.2 syntax (which, who knows, may indeed also be Apache 2.4 syntax);

Order Allow,Deny
Allow from all

BrowserMatchNoCase 2icommerce bad_bot

Deny from env=bad_bot

every other combination of allow/deny/require fails.

If this ends up being "mixing Apache 2.2 and Apache 2.4 directives, " then either they're simply going to have to get mixed, or else I'm going to have to run only Apache 2.2 directives under my host's existing mod_compat module.

There simply isn't going to be any

Require env bad_range

or

 Require ip 131.253.46.0/23
 Require env good_ref
 Require env good_bot

within any SetEnvIf sections of my .htaccess file that won't throw a server error.

[lucy24]

I can only use the identical Apache 2.2 syntax (which, who knows, may indeed also be Apache 2.4 syntax);

Well, the SetEnvIf parts are identical; it's the Allow/Deny that changes.

Order Allow,Deny
Allow from all

BrowserMatchNoCase 2icommerce bad_bot

Deny from env=bad_bot

Is this the way you've been accustomed to do it in 2.2? Be aware that what you have there is:
two lines mod_authzzz
one line mod_setenvif
one line mod_authzzz

It works because each line is separate; no envelopes involved. But it may in the long term be conceptually harmful--to you, not the server--because it encourages the idea that the two mods are in some way connected, when in fact they're not. The server reads and executes the one mod_setenvif line before all three of the mod_authdoodad lines.

Are you positive that your server is, in fact, on 2.4?

Having both 2.4 and 2.2 (i.e. mod_compat) directives along the same filepath will not blow up the server, but things won't execute exactly as expected. (I know this only because one or two directories have their own access-control rules that I forgot about when changing the main htaccess to Require syntax.) If it's strictly alphabetic, then I guess mod_compat--meaning, your old Allow/Deny lines--will be overridden by any Require lines.

There simply isn't going to be any
  Require env bad_range
or
  Require ip 131.253.46.0/23
  Require env good_ref
  Require env good_bot
within any SetEnvIf sections

I wish you would stop saying it in those words, because Require isn't a SetEnvIf directive. Any Require directive has to be inside a <RequireSomething> envelope, while any SetEnvIf directive has to not be inside the envelope.

[JamesSC]

Well, the SetEnvIf parts are identical; it's the Allow/Deny that changes.

Someday I hope to find out how. Until then, this is the only thing I've found that works.

Is this the way you've been accustomed to do it in 2.2?

Yes, this is the way I learned it from you.

it encourages the idea that the two mods are in some way connected, when in fact they're not.

This is what I found confusing from the two differently sourced code fragments I've quoted.

Are you positive that your server is, in fact, on 2.4?

Yes, going on three years now. I figured I'd better get with the program.

I wish you would stop saying it in those words, because Require isn't a SetEnvIf directive. Any Require directive has to be inside a <RequireSomething> envelope, while any SetEnvIf directive has to not be inside the envelope.

I'm only saying it in those words because that's how I've encountered the code, both in your message above and in the virtually identical one I quoted from someone on another forum, but thanks for the clarification. I am currently using

Require all denied

inside a FilesMatch envelope but with no Require envelopes, verified by a test file. The SetEnvIf and BrowserMatch (same thing) sections are the only ones where I'm using the deny language; in all other denial situations - well, the FilesMatch and IPs are the only ones - I'm using Require.

I've gotten everything switched over successfully now, although, as I mentioned, I had to break the SetEnvIf carveouts off separately into their own sections with the Allow/Deny syntax you dislike above. As I mentioned previously, there was no other permutation of the words allow, deny and require than the ones quoted above that didn't throw a server error, and I tried every one of them. When I find better ones, I will, of course, upgrade.

Thanks again for all the time you've poured into this, Lucy. I appreciate your efforts immensely.

[dstiles]

I've given an example of my bot-all sections in the duckduck one above. Below are a few of my "block baddies" sections. All are from a file included in every site config file but I think most should also work in htaccess. I hope this may help.

# allow letsencrypt but block other nasties
<if "! (%{HTTP_USER_AGENT} =~ m#letsencrypt.org#)">
 BrowserMatch [\"\'\$%&\*~#\{\}\[\]\<\>\?\|\\\!] useragent=symbol:$0
</if>

# block bad user-agents
BrowserMatchNoCase curl|(go-)?http-?client|fetch|grab|HTTrack|extract|indy\slibrary|java|larbin|libwww|perl|php|python|scan|wget|winhttp useragent=fetch:$0

# block stupid/injection symbols in URI
SetEnvIf REQUEST_URI (\=|\@|\[|\]|\^|\`|\{|\}|\~) badrequri=uri_sym:$0
SetEnvIfNoCase REQUEST_URI (\'|%0A|%0D|%27|%3C|%3E|%00) badrequri=uri_sym:$0

# reject bad query-strings
<if " %{QUERY_STRING} =~ m#(md5|benchmark|debug|union|select|insert|cast|session|set|declare|drop|update)#i ">
 SetEnvIfExpr "%{QUERY_STRING} =~ /(.+)/" query=inject:$0
</if>

# block bad referers
SetEnvIfNoCase Referer \}__test|base64_decode|bash|chr\(|disconnectHandlers|echo|eval\(|EXEC\(\@S\)|JDatabaseDriverMysqli|JSimplepieFactory|\$_REQUEST|JFactory|getConfig refer=code:$0

# if not known bot, flag as bad
<if " ! %{HTTP_USER_AGENT} =~ m#((Apple|bing|Exa|Google|istella|Twitter)bot|(Mojeek|Seznam|Yandex)Bot|BingPreview|DuckDuck|facebook|letsencrypt|Linespider|Qwantify|Vagabondo|Yeti)# && ! %{REQUEST_URI} =~ m#/robots\.txt#">
 BrowserMatch .{0,10}([Bb]ot|[Cc]rawl|rank|review|spider).{0,10} bot_is=bad_robot:$0
</if>

# and finally, block on the set-env values above
# (some of these are not shown in the code above for brevity)
Require env favicon
Require env netsurf
Require expr %{REQUEST_URI} =~ m#/robots\.txt#
Require expr %{REQUEST_URI} =~ m#favicon\.ico|apple-touch-icon\.png|apple-touch-icon-precomposed\.png#i
<RequireAll>
 Require method GET POST HEAD
 <RequireNone>
 Require env bot_is
 Require env useragent
 Require env badua
 Require env badrequri
 Require env refer
 Require env query
 Require env BlockCountry
 </RequireNone>
</RequireAll>

[JamesSC]

Thanks, dstiles!

Unfortunately, I seem to have a more fundamental problem. Neither

 <RequireAll>
 Require all granted
 <RequireNone>

 Require ip 13.66.
 Require ip 13.66.139.139

 </RequireNone>
 </RequireAll>

nor

<RequireAll>
 Require all granted

 Require not ip 13.66.
 Require not ip 13.66.139.139

 </RequireAll>

are actually blocking either the IPs shown above nor any others also within those respectively tested envelopes, which means none of my other identical IP blocks are working either. This with an htaccess file containing only Apache 2.4 Require ip directives and with Apache 2.4.9 positively verified by my host. Reverting to the Apache 2.2 .htaccess file, OTOH, knocked them dead (via mod_compat, no doubt).

So I obviously have some remedial work to do.

[lucy24]

neither
...
nor
...
are actually blocking either the IPs shown above

Is it possible there are other access-control rules along the same (physical) filepath)? I think I mentioned earlier in this thread that I ran into this issue after converting to Require syntax, and it took a while before I remembered that certain subdirectories had their own htaccess files. (Since you can't have a <Directory> section in htaccess, specialized access controls--or different Options settings--can only be done by giving selected directories their own little supplementary htaccess.)

[JamesSC]

In my case, there's only one directory with one .htaccess file.

The only other references to either bing or 13.66 are a mod_write condition to prevent hotlinking while excerpting bing; existing blocks of two single 13.66. Microsoft Azure IPs which are not bingbots using the same Require directive, and a commented-out BrowserMatchNoCase bing line, all of which exist in the parallel 2.2 .htaccess file which works using the 2.2 Deny from directive instead.

That particular little section is used exclusively in both 2.2 and 2.4 versions to toggle Bingbot on/off - you might have recognized the IP - when it goes nuts and, say, hits the server 1200 times in an hour across half a dozen IPs.

I have a utility allowing me to view my logs remotely in real time, and there it was, blip, blip, blip ringing up 200s when they should have been 403s. Switch out the .htaccess file: 403, 403, 403. Switch back to the 2.4 version and change the double negative <RequireNone> construction to the basic, more simple one: 200, 200, 200. Switch out back to 2.2 version: 403, 403, 403.

Had I not seen this accidentally, it might have been some time before I realized I had basically dropped trou for the world, at least with the require/deny IP parameter.

I thought, well maybe Lucy's right, somehow I'm on a 2.2 machine, so I chatted up support and he looked it up: nope, 2.4.29 (yes, I know I said 2.4.9 above); you might recognize the version.

Perhaps I alone have been confined to some special, training wheels server; not a rational thought path. Perhaps somehow, somehow I've gotten the directive wrong; no, too basic, too simple, and I even triple checked. Perhaps some other process is counterfering: those listed at the top are the only ones I can find remotely connected, and they have no effect in the 2.2 universe.

What element, no matter how crazed, might I be overlooking? That bingbot IP/range should be blocked; it is not.

The one useful datum I have escaped with at this point is knowing that the button doesn't necessarily do anything when you push it, even if you haven't blown up the server.

[JamesSC]

Confirmed original log entries from this morning:

2.4 .htaccess file

13.66.139.139 - - [09/Mar/2021:05:19:48 -0800] "GET /blah/blah/2017/11/blah.jpg HTTP/1.1" 200 207485 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

157.55.39.158 - - [09/Mar/2021:05:27:05 -0800] "GET /2012/01/23/blah/ HTTP/1.1" 200 57095 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

2.2 .htaccess file

13.66.139.3 - - [09/Mar/2021:05:57:42 -0800] "GET /2011/11/30/blah/ HTTP/1.1" 403 4266 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

157.55.39.146 - - [09/Mar/2021:06:09:34 -0800] "GET /2013/02/06/blah/ HTTP/1.1" 403 4266 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

[lucy24]

change the double negative <RequireNone> construction

You do have a knack for saying things that make me anxious, don’t you. To me, “double negative” would mean something like
<RequireNone>
Require not ip blahblah
Require not ip doodah
</RequireNone>
which of course means nobody ever gets in--except the specific ip's you want to block. (Do not think about this too hard, unless you have a particular reason for wanting to get a headache. It's logically the same as RequireAny leaving out the nots.)

Mystery remains, though I'm pretty certain the explanation will be something that has us all whapping outselves upside the head with a resounding “D’oh!”

[JamesSC]

I even tried removing a commented out Bingbot IP that I might alternatively block that lay between the opening and closing tags of the envelopes so that there was absolutely no material involved but the directive (for a better connection, don't you know, but I am consulting goat entrails at this point). Nada.

I did run across a thread where someone with a more complex setup than mine had solved an almost identical problem of non-functionality by improving/correcting his <Directory> wrapper around the whole shooting match, e.g.,

<Directory>
<RequireAll>

Blah blah

</RequireAll>
</Directory>

and that solved his problem.

This wouldn't be something that everyone just happens to know, so that no one need write it down so that the village idiot does, too, such as "oh, yeah, when you've finished with all your directives enclose everything in a <Directory> wrapper to make everything actually work", would it?

[JamesSC]

Well, here's the last thing I could think of to check, which turned out it was not.

Way back in the day askapache actually spelled out the language for using mod_setenvif under 2.2:

SetEnvIfNoCase User-Agent "^Bandit" bad_bot
SetEnvIfNoCase User-Agent "^Baiduspider" bad_bot
SetEnvIfNoCase User-Agent "^BatchFTP" bad_bot
SetEnvIfNoCase User-Agent "^Bigfoot" bad_bot
SetEnvIfNoCase User-Agent "^Black.Hole" bad_bot

Order Allow,Deny
Allow from All
Deny from env=bad_bot

There hasn't been anything I've been able to find that tells me mod_setenvif uses any different language under 2.4 - it may; it may not; I don't know - so I retained the 2.2 formulation when rebuilding my .htaccess for 2.4, and, AFAIK, it still works; I've seen it catching flies. But I worried that maybe the "Allow from all" somehow preempted and precluded my new Require directives, so I commented out both the Order and Allow lines leaving only the Deny line, and nothing broke.

But it had no effect. The new Require stuff still doesn't work.

So at this point my only real liability is that, while all the cool kids are wearing new 2.4 shoes, I'm still wearing old 2.2 shoes. All my .htaccess security and other functions are still working just as well as they were last week, and any speed/efficiency premium I might have gotten from 2.4 hasn't been measurable. Nothing will effectively change for me except the inability to render the complex new formulations available under 2.4 until my host finally pulls the plug on mod_compat.

If anyone runs across anything about 2.4 Require reformulations arriving stillborn or otherwise unable to thrive, please be so kind as to hit me up here. Otherwise, I've run out of road.

[lucy24]

<Directory> wrapper around the whole shooting match

Yikes. I thought this was all happening in htaccess--which, in effect, is its own <Directory> wrapper.

But I worried that maybe the "Allow from all" somehow preempted and precluded my new Require directives, so I commented out both the Order and Allow lines leaving only the Deny line.

Now, wait a minute. There shouldn't be ANY lines using Allow/Deny syntax in the same htaccess/Directory as lines with Require syntax.

Commenting-out the Order line will often have no effect, because order only matters when a given request matches both an Allow directive AND a Deny directive. The default is Deny,Allow.

:: poring over docs [httpd.apache.org] ::

Oh, whoops! I always thought it was called mod_compat, but in fact it is mod_access_compat. This matters, because it means it executes AFTER mod_authzheehaw.

[JamesSC]

Yikes. I thought this was all happening in htaccess--which, in effect, is its own <Directory> wrapper.

It is. I was hypothesizing.

Now, wait a minute. There shouldn't be ANY lines using Allow/Deny syntax in the same htaccess/Directory as lines with Require syntax.

No doubt. When I plumb the Delphic mysteries of what a SetEnvIf directive should look like nowadays when neighboring with Require directives, I'll correct the prior 2.2 version. But testing with such lines commented out earlier today left the impotent Require directives unaffected, so the issue is moot with respect to the problem at hand.