http > https fails when the directory is password-protected

Get a "Failed authorization" error

[MichaelBluejay]

My .htaccess redirects http to https requests like so:

RewriteCond %{HTTPS} !on
RewriteRule (.*) https://example.com/$1 [R=301,L]

That usually works fine, but if the request is for http://example.com/admin, then my browser loads the Failed Authorization page. (If I go to https://example.com/admin, then I get the password dialog as expected.)

Curiously, there is *no* request for /admin in my logfile.

Does anyone know how to get this to work?

[not2easy]

If the failed authorization page has an "Advanced" dialog you can store an "exception" to accept the example.com cert for use at example.com/admin page.

Why not just request "https://example.com/admin"? If /admin is a folder and not a page, it may need to have its own .htaccess file as rewrites aren't always inherited.

[justpassing]

Sounds like a matter of order of directives in the configuration file.

[lucy24]

400-class responses trump 300-class responses, regardless of which module issues them. That is, if mod_rewrite issues a 301 and then some other mod further along issues a 401 (is it 401? you know what I mean), the 301 will never be sent.

Curiously, there is *no* request for /admin in my logfile.

Is your /admin/ directory physically located inside the directory containing your other site files? If you request /admin/ with the correct protocol, but intentionally enter the wrong password, will that show up as a 400-whatever-it-is?

[Demaestro]

Agreed with Justpassing. It is almost for sure an order of operations issue.

[lucy24]

I don't see how order can possibly be relevant, since authorization is handled by a different module.

[MichaelBluejay]

Thank you for the replies. To answer questions:

Why not just request "https://example.com/admin"?

Because I can't control how my users will request pages from my sites. Many of them will type "example/protectedDirectory" into their browser's address bar, and, in Safari at least, that makes a request for http:// and not https://

If the failed authorization page has an "Advanced" dialog you can store an "exception" to accept the example.com cert for use at example.com/admin page.

It doesn't, but even if it did, my goal is to not show errors to my users. I'd like them to see the login dialog.

Is your /admin/ directory physically located inside the directory containing your other site files?

Yes. All the web files are in /home/username/example.com/. So, the admin folder is in /home/username/example.com/. Not, I have the same problem on another domain, where the domain itself is protected with htpasswd: /home/username2/example2.com/.

If you request /admin/ with the correct protocol, but intentionally enter the wrong password, will that show up as a 400-whatever-it-is?

Kind of. It appears to give me unlimited retries in the login dialog box, but when I click the Cancel button in the dialog box, I get an "Authorized Required" error page.

One workaround is to roll my own login page rather than using htpasswd, but I'm hoping to avoid that.

[lucy24]

Kind of.

Sorry, by “will that show up as” I meant, will the request appear in logs? You said at the outset that wrong-protocol requests for password-protected areas don't end up in logs at all. That's why I asked if they're in the same physical directory.

Did you originally find this issue by experimentation, or have bona fide users reported problems? It seems like the kind of thing that will disappear by itself in time, as more and more browsers default to https.

:: detour to experiment ::

The only analogy I could come up with on sites of my own was to request
http://www.example.com/stats/
where example.com is https, and /stats/ is password-protected with the ordinary htpasswd. (And I never use it, so no risk of interference from cached content.) In the browser I ended up at the URL:
https://www.example.com/failed_auth.html
with page text showing my 404 page. (I guess if had publicly accessible password-protected areas, it would be appropriate to make a failed_auth page.)

:: and now to logs ::

http access logs:
"GET /stats/ HTTP/1.1" 301
https access logs:
"GET /failed_auth.html HTTP/1.1" 404
https error logs:
File does not exist: /full/physical/file/path/example.com/failed_auth.html

Is this similar to what you're seeing? If I request a password-protected page with the wrong protocol, it never even asks for a password, but proceeds directly to “authorization failed”.

If I request the same page with the correct protocol, I get a dialog asking for username and password ... after which authorization fails anyway, because I never go there and have forgotten both the username and the password :) The last thing I see is a server message saying that “In addition, a 401 error was encountered while trying to” blahblah ErrorDocument. All this shows up in logs as a 404, not a 401. At no point do error logs record a 401 or Authorization Failed.

Huh.

[not2easy]

If users can access the /protectedDirectory at http, that indicates that your https: rewrite rules are not inherited in that directory. The default in many configurations (notably virtual hosts) is that the mod_rewrite rules do not inherit into each directory. Add a .htaccess file to /protectedDirectory or if there is already a .htaccess file in that directory, edit to add the rewrites for https adjusting both the target URL and the rewrite base to /protectedDirectory so your users can't get to "example.com/protectedDirectory" via http: even if it is typed in.

[MichaelBluejay]

Did you originally find this issue by experimentation, or have bona fide users reported problems?

Bona-fide users reported problems. But even if they hadn't, it's easy for me to reproduce 100% of the time, and I don't want to give users that experience. Heck, I don't want to give *myself* that experience every time I try to access a password-protected directory.

Is this similar to what you're seeing? If I request a password-protected page with the wrong protocol, it never even asks for a password, but proceeds directly to “authorization failed”.

Yes, exactly. I should have been clearer in my original post. The only difference is that there's no hit at all to my http access.log. I do get the request for /failed_auth.html in my https access log (which is a 404).

If users can access the /protectedDirectory at http...

They can't. As I said in my OP, " if the request is for http://example.com/admin, then my browser loads the Failed Authorization page."

Add a .htaccess file to /protectedDirectory or if there is already a .htaccess file in that directory...

Not surprisingly, it's there already. That's how the directory is protected.

edit to add the rewrites for https adjusting both the target URL and the rewrite base to /protectedDirectory so your users can't get to "example.com/protectedDirectory" via http: even if it is typed in.

Does not help. Same error. Adding "RewriteOptions inherit" also resulted in no change.

[not2easy]

Are you adding the https rewrite for the /protectedDirectory/ folder to include the directory name?

RewriteEngine On
RewriteCond %{HTTP_HOST} !^(example/.com)?$ [OR]
RewriteCond %{HTTPS} !on
RewriteRule (.*) https://example.com/protectedDirectory/$1 [R=301,L]

RewriteBase /protectedDirectory/

inside the /protectedDirectory/ folder? "OptionsInherit" may not work as expected when another .htaccess file is present.

[lucy24]

I think we've already established that /protectedDirectory/ can not be reached by http. And there would be no reason to duplicate the domain-name-canonicalization redirect unless the directory really is in a different physical location, or if it's already got its own RewriteRules. (The mere existence of another htaccess doesn't prevent RewriteRules from inheriting; that only happens if the second htaccess again invokes mod_rewrite.)

Now, I'm not sure why the site thinks there should be a failed_auth page, unless--heh, heh--we both coincidentally have the same host, and that's one of their defaults like missing.html and forbidden.html. But you may as well take advantage of it: create a page called failed_auth.html, and on this page put some of the information you'd like your human users to have if they land there by accident. You might even give a link to the protected page, using the full https to make sure they're requesting the right thing.

[MichaelBluejay]

Are you adding the https rewrite for the /protectedDirectory/ folder to include the directory name?

Yes, except you appear to have a forward slash where you meant a backslash in line 2. I used a backslash.

Now, I'm not sure why the site thinks there should be a failed_auth page, unless--heh, heh--we both coincidentally have the same host, and that's one of their defaults

We do have the same host. You're clever, I'll give you that!

But you may as well take advantage of it: create a page called failed_auth.html, and on this page put some of the information you'd like your human users to have if they land there by accident.

Also clever. But if there's more than one protected directory then this doesn't work so well, because I can't give a link to the directory the user was trying to get to, because I don't know which one it was. I can't grab that info from the referrer, because as you see from the logs, it's not recorded. If I don't ever have more than one protected directory, I can do you one better: I can just have failed_auth.html automatically redirect to https://example.com/protected_directory.

I could take it up with my webhost but they pride themselves on being unhelpful. So, it seems my options are:

(1) Live with it.
(2) Redirect the failed_auth.html page to the proper page.
(3) Write my own login script rather than relying on htpasswd
(4) Change hosts.

So, I think we're done. Thanks for your help, lucy24!