Forum Moderators: phranque
Status 550 (does not support the HTTP protocol)On all requests, or only HTTP/2 requests?
<?php
phpinfo();
?>
sudo a2dismod php7.3
sudo a2enconf php7.3-fpm
sudo a2enmod proxy_fcgi
sudo a2dismod mpm_prefork
sudo a2enmod mpm_event
sudo a2enmod ssl
sudo a2enmod http2<VirtualHost 185.35.148.127:443>
ServerAdmin alert@ssph.org.uk
ServerName www.example.co.uk
DocumentRoot /srv/mysite
Header edit Set-Cookie ^(.*)$ __Host-$1;HttpOnly;Secure;SameSite=Strict
<Directory "/">
AllowOverride None
Require all denied
</Directory>
<Directory "/srv/mysite">
DirectoryIndex index.php
AllowOverride All
Include /etc/apache2/use-setenv.conf
</Directory>
CustomLog ${APACHE_LOG_DIR}/mysite/access.log combined env=!dontlog
SSLEngine on
<FilesMatch "\.(php)$">
SSLOptions +StdEnvVars
</FilesMatch>
SetEnv nokeepalive ssl-unclean-shutdown
Include/etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/www.example.co.uk/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.example.co.uk/privkey.pem
</VirtualHost>
First hurdle: apache_getenv does not work, which throws a fatal error. I use this about three times on each page to manage bots/users/rubbish and direct accordingly to logs. Haven't really searched into this yet but it could be a stumbling block?This jumped out at me because I remember having to tweak parts of my logheaders code when the server was upgraded. What's the exact syntax of the lines that use this function? (Mine uses a combination of getenv() and $SERVER(blahblah), but I’ve got a vague notion apache_getenv was one of the things I could never get to work.)
apache_getenv(key) does not work on an php cgi installation, in this case rather use $_SERVER["REDIRECT_key"]
function apache_getenv($key) {
return(isset($_SERVER['REDIRECT_$key'])?$_SERVER['REDIRECT_URL']:"");
}
function apache_setenv($key,$val) {
$_SERVER['REDIRECT_$key']=$val; return(apache_getenv($key) );
}
function apache_getenv($key) { return(isset($_SERVER[$key])?$_SERVER[$key]:""); }
function apache_setenv($key,$val) { $_SERVER[$key]=$val; return(apache_getenv($key) ); }# letsencrypt
SetEnvIf Referer .well-known/acme-challenge/ letsencrypt bot=letsencryptr
SetEnvIf REQUEST_URI ^/\.well-known/acme-challenge/ letsencrypt bot=letsencryptu
BrowserMatch letsencrypt.org letsencrypt bot=letsencryptb
Require env letsencrypt# mojeek
<if "-R '5.102.173.64/28' ">
SetEnvIfExpr "%{REMOTE_ADDR} =~ /(.+)/" ips=mojeek:$0
BrowserMatch MojeekBot mojeek bot=mojeek
Require env mojeek
</if>BrowserMatchNoCase analy[sz]|CensysInspect|legs|search|seo bot_is=seo:$0
...
Require expr %{REQUEST_URI} =~ m#/robots\.txt#
<RequireAll>
Require method GET POST HEAD
<RequireNone>
....
Require env bot_is
....
</RequireNone>
</RequireAll>
<if " (%{HTTP_USER_AGENT} =~ m#(<|>|%0A|%0D|%27|%3C|%3E|%00)#i)">
SetEnvIfExpr "%{HTTP_USER_AGENT} =~ /(.+)/" useragent=inject:$0
Require all denied
</if>
Looking at apache's error.log shows a rejection of errordoc.php (sometimes!), which is where all the non-200 trickery happens, but no idea whyAre you getting the ludicrously unhelpful “client denied by server configuration”? That seems to be the one thing that’s unchanged between 2.2 and 2.4. Yes, thank you Apache, I realize the request must have run into some kind of rule, but WHAT rule?
The hidden folders are populated during a cert update and then emptied again (I guess a working folder for certbot).Aha, that explains it. Yes, my host used to use a different verification system for letsencrypt but I think they have now moved everything to the /.well-known/ method.
function apache_getenv($key,$walk_to_top=true) { return(isset($_SERVER[$key])?$_SERVER[$key]:""); }
function apache_setenv($key,$val,$walk_to_top=true) { $_SERVER[$key]=$val; return(apache_getenv($key,false) ); }<if " ! (%{ENV:bot} =~ m#^$# ) && ! (%{REQUEST_URI} =~ m#/robots\.txt#) && (%{HTTP_USER_AGENT} =~ m#.{0,10}([Bb]ot|[Cc]rawl|rank|review|spider).{0,10}#) ">
SetEnv bot_is=bad_robot:$0
Require all denied
</if>
SetEnvIfExpr "%{REMOTE_ADDR} =~ /(.+)/" bot=duck ips=duck:$0<if " ! (%{ENV:bot} == '') && (%{HTTP_USER_AGENT} =~ m#compatible;#i)">
SetEnvIfExpr "%{HTTP_USER_AGENT} =~ /(.+)/" badua=compatible
Require all denied
</if><if " ! (%{ENV:bot} =~ m#^$# ) && ! (%{REQUEST_URI} =~ m#/robots\.txt#) && (%{HTTP_USER_AGENT} =~ m#.{0,10}([Bb]ot|[Cc]rawl|rank|review|spider).{0,10}#) ">
SetEnv bot_is=bad_robot:$0
Require all denied
</if>
Setting browser, referer: https://www.example.co.uk/about.php
Setting browser, referer: https://www.example.co.uk/about.php
client denied by server configuration: /srv/example/, referer: https://www.example.co.uk/about.php
Setting browser, referer: https://www.example.co.uk/about.php
Setting browser, referer: https://www.example.co.uk/about.php
client denied by server configuration: /srv/example/errdoc.php, referer: https://www.example.co.uk/about.php
that gap seems to suggest that errordoc is processed as a new pageIn fact that is exactly what happens. First comes the external request for realpage.html, and then if the request is denied, there is a subsequent internal request for 403.html or whatever it may be. And this internal request is subject to exactly the same access-control rules as the preceding external request. That's why you need to make sure you have poked holes for errordoc.php in each and every place that is capable of issuing a 403, in the same way that you poke holes for robots.txt.
RewriteRule /errdoc.php /errdoc.php [L]
...with and without escaped dots.
I think it IS running errdoc as a separate process but surely apache KNOWS what it is doing?I'm sure you remember:
The error document is so fundamentalIt isn't. Required by the canons of ordinary human decency, yes. Built-in, no. The default is
Microsoft Internet Explorer (MSIE) will by default ignore server-generated error messages when they are "too small" and substitute its own "friendly" error messages. The size threshold varies depending on the type of error, but in general, if you make your error document greater than 512 bytes, then MSIE will show the server-generated error rather than masking it.Does MSIE really still do this? What a good thing it is on its way out; I can only hope Edge isn't similarly too smart for its own good.
SetEnvIf Request_URI /errdoc\.php exempt=errdoc
<if " %{ENV:exempt} == 'errdoc' ">
SetEnv test
</if><LocationMatch "^(.*\.php)$">
ProxyPass fcgi://127.0.0.1:9000/your/site/webroot
</LocationMatch>
<LocationMatch "^(.*\.php)$"><LocationMatch "\.php$">When you switched to http/2 did you have to modify your domains' vhosts?I'm on shared hosting, so the Server Fairies did it all for me ;) And, needless to say, didn't bother to tell us, so my first clue was when all my log processing scripts broke due to repeated occurrences of things like "HTTP/1\.[01]". My only contact with config files is in MAMP--most recently yesterday when I laboriously figured out that the reason one local site wasn't working as intended was that its directory name (on my HD, I'd never do it on a server) contains an apostrophe ' that I'd erroneously entered as ’ (curly apostrophe) in two different places. I use htaccess even on local sites, precisely because I want to replicate whatever I'm doing on the real sites.
if (!in_array($_SERVER["SERVER_PROTOCOL"], array('HTTP/0.9','HTTP/1.0','HTTP/1.1'))) $sc = 505;if (!in_array($_SERVER["SERVER_PROTOCOL"], array('HTTP/2.0','HTTP/1.0','HTTP/1.1'))) $sc = 505;
Now corrected and with 0.9 removed; I see no point in it.If ever there were an unequivocal 403 that would be it! Robots themselves must have abandoned it years ago; I don't find a single HTTP/0 in logs. I still see a few 1.0, but they're getting rarer--rare enough that I set an environmental variable bad_protocol and unset as needed.
