Blocking any request uri that doesn't begin with [a-z0-9]

[csdude55]

This is currently in .htaccess, but will be moved to Apache conf once I'm done...

I have a list of RewriteCond where I try to block anything suspicious. I keep track of any 404 errors, and I see a lot like /.example/ (opening with a dot, %, //, etc). They don't really hurt anything, but I'm trying to block them and save some resources.

The only thing on my sites that should start with something that's not [a-z0-9] is /.well-known/, which is used by cPanel to validate the domain for AutoSSL. That's important, so I need to modify my list to block [^a-z0-9] UNLESS it's .well-known.

Technically, the .well-known directory has 2 empty directories in it with no files. So unless files are created and deleted whenever AutoSSL runs, maybe I should block any file attempt in that directory? But that might be overkill and cause more problems than it helps.

I have a pretty detailed list of 28 conditions that looks like:

RewriteCond %{HTTP_REFERER} service.dropdowndeals.com [NC,OR]
RewriteCond %{REQUEST_URI} ^/(?:[^a-z0-9]|crossdomain|(?:phpmy)?admin|p2|ui|impl\.|editor|phpunit|header-rollup|system|bitcoin|rev/|script|public) [NC,OR]
...

RewriteCond %{THE_REQUEST} \?\ HTTP/|\/\*\ HTTP/|etc/passwd|%0A|%0D [NC]
RewriteRule ^ - [F]

Can you guys and gals suggest a way to allow this .well-known exception within the RewriteCond? The only way I could think of is to make it a separate rule, but that seems kinda inefficient so I'm just trying to find a better way. Eg,

RewriteCond %{REQUEST_URI} !^/\.well-known/
RewriteCond %{REQUEST_URI} ^/[^a-zA-Z0-9]
RewriteRules ^ - [F]

RewriteCond %{HTTP_REFERER} service.dropdowndeals.com [NC,OR]
RewriteCond %{REQUEST_URI} ^/(?:crossdomain|(?:phpmy)?admin|p2|ui|impl\.|editor|phpunit|header-rollup|system|bitcoin|rev/|script|public) [NC,OR]
...

RewriteCond %{THE_REQUEST} \?\ HTTP/|\/\*\ HTTP/|etc/passwd|%0A|%0D [NC]
RewriteRule ^ - [F]

[phranque]

i would simply insert the RewriteCond with the implied [AND] in the existing ruleset.
something like:

RewriteCond %{HTTP_REFERER} service.dropdowndeals.com [NC,OR]
RewriteCond %{REQUEST_URI} !^/\.well-known/
RewriteCond %{REQUEST_URI} ^/(?:[^a-z0-9]|crossdomain|(?:phpmy)?admin|p2|ui|impl\.|editor|phpunit|header-rollup|system|bitcoin|rev/|script|public) [NC,OR]
...

RewriteCond %{THE_REQUEST} \?\ HTTP/|\/\*\ HTTP/|etc/passwd|%0A|%0D [NC]
RewriteRule ^ - [F]

[csdude55]

The inability to group them in ( ) throws me off. How does it know what I mean, and not:

# parentheses obviously just for explanatory purposes
(
 RewriteCond %{HTTP_REFERER} service.dropdowndeals.com [NC,OR]
 RewriteCond %{REQUEST_URI} !^/\.well-known/
) [AND]
(
 RewriteCond %{REQUEST_URI} ^/(?:[^a-z0-9]|crossdomain|(?:phpmy)?admin|p2|ui|impl\.|editor|phpunit|header-rollup|system|bitcoin|rev/|script|public) [NC,OR]
 ...
 RewriteCond %{THE_REQUEST} \?\ HTTP/|\/\*\ HTTP/|etc/passwd|%0A|%0D [NC]
)

[phranque]

the conditions are evaluated serially

[csdude55]

Ohh, I see. I still wish Apache would somehow implement parentheses :-)

[dstiles]

I have a couple of checks for well-known. For Nominet, which checks regularly, I have:

BrowserMatch (nominet\.uk/privacy-notice) bot_is=wellknown:$0

The other well-known is for Let's Encrypt - note, IPs can vary so not included:

BrowserMatch "Let's Encrypt validation server" letsencryptB bot=letsencrypt
SetEnvIf REQUEST_URI \.well-known/acme-challenge/ letsencryptR requri=letsencrypt:$0
# Require env letsencrypt
<RequireAll>
 Require env letsencryptB
 Require env letsencryptR
</RequireAll>

and:

<if "! (%{HTTP_USER_AGENT} =~ m#letsencrypt.org#)">
 BrowserMatch [\"\'\$%&\*=@~#\{\}\[\]\<\>\?\|\\\!] useragent=symbol:$0
</if>

and then allow the bot as:

# allow acceptable bots
<if " ! %{HTTP_USER_AGENT} =~ m#((Apple|bing|Exa|Google|istella|Twitter)bot|(Mojeek|Seznam|Yandex)Bot|BingPreview|DuckDuck|facebook|Let's Encrypt|Qwantify|Vagabondo|Yeti)# && ! %{REQUEST_URI} =~ m#/robots\.txt#">
 BrowserMatch .{0,10}([Bb]ot|[Cc]rawl|rank|review|spider).{0,10} bot_is=bad_robot:$0
</if>

The above are all part of a more extensive file of tests included in each website.conf with:

 <Directory "/srv/website">
 DirectoryIndex index.php
 AllowOverride All
 Include /etc/apache2/use-setenv.conf
 </Directory>

[lucy24]

Huh. I would just have said (in htaccess or a <Directory> section)

RewriteCond %{REQUEST_URI} !\.well-known
RewriteRule ^\W - [F]

but perhaps I missed something?

The form \W does admit requests beginning in _ (lowline, counted as a word character); if that's an issue you would instead say
[\W_]
as a group.

Edit: If you also need to block unauthorized requests for .well-known, make that a separate rule, putting that exact URI in the pattern.