Authentication problems after rewrite

Problem combining mod_rewrite and mod_auth

[PatomaS]

Hi people

I'm working on a personal site made in PHP where all the requests are redirected to /index.php and from there the whole system triggers what is needed, showing a page, validating user input, etc. That is working with no problems so far.

The content is made of .xml files but the url's are .php, so for instance when you request http://example.com/news/upload/index.php, the content you are actually getting is from /content/languages/en/news/upload/index.xml.

Now, I want to protect one folder/directory in that site using .htaccess and .htpasswd but I'm having problems with that.

The problem in that the folder I want to be protected, doesn't seem to be aware of the presence of the .htaccess. If I write the authentication code in the .htaccess at root level there is also no effect.

The rules seem to be fine, I have tried protecting the main /index.php with a <Files> section in the .htaccess and that worked ok.

This is the relevant structure of the site

/
/languages/
/languages/en/
/languages/en/content/index.xml
/languages/en/content/news/index.xml
/languages/en/content/news/upload/index.xml

This is the code used for the redirections

RewriteCond %{REQUEST_URI} !^/index.php [NC]
RewriteCond %{REQUEST_URI} !/robots.txt [NC]
RewriteCond %{REQUEST_URI} !/*.css [NC]
RewriteCond %{REQUEST_URI} !/*.dtd [NC]
RewriteRule (.*) /index.php

This is the last thing I have tried for the authentication

<Files "/languages/en/content/news/upload/*.*">
AuthType Basic
AuthName "upload files"
AuthUserFile /.htpasswd
require valid-user
</Files>

Things I have tried

• <Files> section in the main .htaccess
  
• the authentication code withuout the <Files> in a .htaccess file in the specific folder
  
• Absolute path for AuthUserFile and file to protect inside the folder
  
• Relative path in the <Files> section: /news/upload/index.xml
  
• Relative path in the <Files> section: /news/upload/index.php
  
• ...
  

Any help will be most appreciated.

Thanks

[g1smd]

Literal periods in patterns should be escaped.

The pattern

!/*.

css negative matches

example.com/.css
example.com/Xcss
example.com/9css
example.com/;css
example.com////.css
example.com////Xcss
example.com////9css
example.com////;css
example.com/////////.css
example.com/////////Xcss
example.com/////////9css
example.com/////////;css
example.com.css
example.comXcss
example.com9css
example.com;css

Likewise for the

!/*.dtd

pattern.

"index.php" should not appear in the URLs.

I don't see how requests for images are handled. Does index.php serve those too?

The path in the htaccess file should start from server root, not site root:

AuthName "12346"
AuthType Basic
AuthUserFile /vhost/vhost4/e/x/example.com/www/test/.htpasswd
AuthGroupFile /dev/null
require user abcdef

[PatomaS]

Hi

Thanks for the quick response.

All the images and other files have rules, like:

RewriteCond %{REQUEST_URI} !/*.css [NC]
RewriteCond %{REQUEST_URI} !/*.dtd [NC]
RewriteCond %{REQUEST_URI} !/*.js [NC]
RewriteCond %{REQUEST_URI} !/*.jpg [NC]
RewriteCond %{REQUEST_URI} !/*.gif [NC]
RewriteCond %{REQUEST_URI} !/*.png [NC]
RewriteCond %{REQUEST_URI} !/*.pdf [NC]
RewriteCond %{REQUEST_URI} !/favicon.ico [NC]
RewriteRule (.*) /index.php

I'll make the adjustments for the dot.

About the index.php, usually the calls are for the directory itself, like example.com/news/, but I also consider calls for the index.php, like any other file in the folder.

About the path, As I mentioned, I already tried absolute path, May be I did something wrong, but I tried this:

<Files "/mnt/www/site/languages/en/content/news/upload/index.xml">
AuthType Basic
AuthName "upload"
AuthUserFile /mnt/www/site/.htpasswd
require valid-user
</Files>

<Files "/mnt/www/site/languages/en/content/news/upload/index.php">
AuthType Basic
AuthName "upload"
AuthUserFile /mnt/www/site/.htpasswd
require valid-user
</Files>

The <File> rule that mentions the .xml file, is the actual real path for the file, the <File> rule that mentions the .php file was a test since the other one didn't work.

I was wondering, since mod_rewrite is redirecting the call, may be, there is never a request for the file mentioned in the <Files> rule, at least that is what I think right now since nothing seems to be working. If that's true, then mod_rewrite works before mod_auth. If that's the case, can I change that order? or, can I add any kind of validation to the rewrite rule?

I feel completely lost here.

Thanks for the help :)

Bye

[lucy24]

If that's true, then mod_rewrite works before mod_auth.

In general it's perilous to make rules based on assumptions about which modules execute in what order. Unless of course it is your own server, so you don't have to guess. But it does seem safe to assume that the mod_auth package executes very late; in fact I tend to think of it as Core although technically it isn't.

All the images and other files have rules, like:

RewriteCond %{REQUEST_URI} !/*.css [NC]
RewriteCond %{REQUEST_URI} !/*.dtd [NC]
RewriteCond %{REQUEST_URI} !/*.js [NC]
RewriteCond %{REQUEST_URI} !/*.jpg [NC]
RewriteCond %{REQUEST_URI} !/*.gif [NC]
RewriteCond %{REQUEST_URI} !/*.png [NC]
RewriteCond %{REQUEST_URI} !/*.pdf [NC]
RewriteCond %{REQUEST_URI} !/favicon.ico [NC]
RewriteRule (.*) /index.php

In other words,

RewriteCond %{THE_REQUEST} !index\.php
RewriteRule (/|\.php)$ /index.php [L]

and that's all. Well, except for business with the query string; surely that goes in the rule somewhere?

You might think this format excludes requests for the front page-- null in htaccess --but these will be picked up on the second go-round, after mod_dir has done its stuff. In rare cases you have to make the Rule fancier. But you should never put something in a Condition that will go just as well in the body of the Rule.

One version: "If the requested file isn't a, b, c, d, e, f or g".

Other version: "If the requested file is h."

[PatomaS]

Hi

Thanks for the answer.

I'm already reading about what you mentioned, thanks for the suggestions.

About The order of the rules, I'm not considering any order of execution, to be honest, I don't know who comes first in execution time; what I wrote before was a thought I had considering the rules and the results. Like trying to figure out what is happening.

About the rule you wrote, that works as substitution/improvement of what I had before, but still no password is requested when you access the folder. Plus, if you request index.php you get a 404.

I suppose I can write something to fix the 404's, but first I'm more interested in the problem with the password.

Any ideas?

This is my actual .htaccess

RewriteEngine on
RewriteBase /


RewriteCond %{REQUEST_URI} !^/index.php [NC]
RewriteCond %{THE_REQUEST} !index\.php
RewriteRule (/|\.php)$ /index.php [L]


<Files "/mnt/www/site/languages/en/news/upload/*.*">
AuthType Basic
AuthName "subir archivos"
AuthUserFile /mnt/www/site/.htpasswd
require valid-user
</Files>

Again, thanks for all the help.

Bye

[g1smd]

Literal periods in patterns should be escaped.

[lucy24]

RewriteCond %{REQUEST_URI} !^/index.php [NC]
RewriteCond %{THE_REQUEST} !index\.php

I can't think of any situation where you would need both these lines.

Matter of fact, I can't think of any situation where the REQUEST_URI element (not to be confused with REQUEST_FILENAME, which can change) is not the same as the URI given in the body of the rule.

[PatomaS]

I forgot to mention how the situation is right now.

With the above .htaccess, I can load all the pages correctly, but no password is requested where it should be.

If I remove the <Files> section and just write the rules in a .htaccess file in the /mnt/www/site/languages/en/news/upload/, also nothing happens.

If I remove the rewrite rules in the main .htaccess and remove the .htaccess file in the internal folder and load the files directly, no password is requested but I can see the .xml file in the browser.

If I have the rewrite rules in the main .htaccess, with no <Files> section and the authentication lines in the internal .htaccess file, no password is requested when accessing the file with the right url (http://example.com/news/upload/).

If I have the rewrite rules in the main .htaccess, with no <Files> section and the authentication lines in the internal .htaccess file, password is requested when accessing the file with the wrong url (http://example.com/languages/en/content/news/upload/).

So as far as I can tell, still seems to me that mod_rewrite is interfering with the Auth process, or in my ignorance of the process, I just can't see what is happening.

Again, thanks for all the helpand time dedicated.

Bye

[PatomaS]

Hi

@lucy24
I added again the

RewriteCond %{REQUEST_URI} !^/index.php [NC]

because When I used only

RewriteCond %{THE_REQUEST} !index\.php

, I got errors.

access.log

127.0.0.1 - - [04/Mar/2013:09:45:52 +0800] "GET / HTTP/1.1" 500 346

error.log

[Mon Mar 04 09:45:52 2013] [error] [client 127.0.0.1] Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.

If you could elaborate a bit more your last comment, may be I can explain why I used REQUEST_URI how I did, or improve my knowledge and fix the errors. :)

Thanks.

[PatomaS]

Hi

@g1smd
I suppose you are talking about the <Files> rule I wrote up. Yes, I forgot again, sorry.

I thought that Files accepted only a real path and only ? and * as special chars, I forgot that is also a pattern matching.

Still, the same situation.

Thanks :)

[lucy24]

Escape literal periods in all mod_rewrite patterns (not targets), but not in Files envelopes. Only in FilesMatch-- or anything else that takes a Regular Expression.

I added again the RewriteCond %{REQUEST_URI} !^/index.php [NC] because When I used only RewriteCond %{THE_REQUEST} !index\.php, I got errors.

Oops, sorry, now I see how it works. The rule itself is supposed to apply to all requests for pages except internal requests for index.php -- that's why you need the REQUEST_URI line.

But there should be an earlier redirect (RewriteRule with [R=301] flag) that intercepts all requests for "index.php" by name and redirects to directory alone. So by the time you reach the simple rewrites, there won't be any %{THE_REQUEST} to worry about.

I thought that Files accepted only a real path

So did I, until I added a FilesMatch envelope to my own htaccess and found that it only worked as intended if I included requests for nonexistent files. I think it may be more accurate to say that Files and FilesMatch apply to both real files (with implied real path) and to requests.

I am now going to read your second-to-last post very slowly and carefully, because I think you've covered all possible options.

---

There's one issue that jumps out right away.

<Files "/mnt/www/site/languages/en/news/upload/*.*">

So you're not really matching files at all. You're matching everything in a particular directory. This makes me wonder if a <Files> or <FilesMatch> envelope is even the right way to go. Did you try putting the ruleset inside the {blahblah}/en/news/upload/ directory instead?

Answer: Yes, you did.

If I have the rewrite rules in the main .htaccess, with no <Files> section and the authentication lines in the internal .htaccess file, password is requested when accessing the file with the wrong url (http://example.com/languages/en/content/news/upload/).

and

... no password is requested when accessing the file with the right url (http://example.com/news/upload/)

:: racking brains ::

D'oh! You need another RewriteRule. You would have needed it anyway; it's just that you would not have noticed if it weren't for the authentication issue.

Whenever you have the pattern
{user request for A}
=
{content served from B}
you need two RewriteRules. You've got one of them: the part that looks at the request and serves up the content. You need the other part: the part that grabs any request for the "real" filename and forcibly redirects them to the URL you want people to use.

This is a redirect, so it will come earlier in mod_rewrite. Make a rule that takes explicit requests for
/content/languages/en/(\w+/\w+/).*
and redirects to
http://www.example.com/$1

The .* is to get rid of the "index.php" element, if any, because you don't want that either. This bit may need further adjusting depending on your site structure: do the pseudo-directories contain names other than index.php? I'm saying "pseudo-directories" because here we are talking about URLs, not physical locations.

There are also two entirely different extensions. If I've got it right, .php is used only in URLs and .xml is used only in the "real" content. This, in turn, means that your Files envelope-- if any-- needs only ".xml" and nothing else. You don't need to say anything about .php, because those requests were redirected before they got anywhere near the directory where the content lives.

And this, in turn, means that your RewriteRules fall into two groups. In the first group are rules involving \.xml$ --with RewriteCond looking at {THE_REQUEST} --leading to redirects. In the second group are the rules involving (/|\.php)$ --no Conditions required-- leading to rewrites.

[PatomaS]

Hi

I have to say that I'm not getting your answer completely, or so I think, may be I didn't explained my self properly, so I'll try now. With the code I have before, without the auth.

Every time a request for a folder or file was made, the rewrite rules in the main .htaccess told Apache to serve /index.php (except for images, scripts, styles and a few more).

/index.php did many things, like set the php environment, validate information and so on; but also, one of the things it did, was to take the url requested, look for the right file in the folder structure, build the answer and serve it to the client.

For instance:
Client requests:

• http://www.example.com/news/
  

.htaccess process:

• change /news to /index.php
  

/index.php goes to work:

• read from $_SERVER and realize that /news was requested (plus a bit more information)
  
• go to /languages/en/news/, load index.xml from that folder
  
• process it, including generate more content from linked php files, DB, etc.
  
• apply xsl rules
  
• verify XHTML validity
  
• serve the result document to the client with the url requested
  

That part was working fine.

Now I wanted to add the authentication in one folder and I didn't get that to work.

Thinking a little bit about why it doesn't work, I realize that the url requested doesn't represent the folders structure, neither the rewrite does, so I suppose that if mod_auth has to 'see' a literal request for the specific file/folder, that is not happening.

So, considering your answer about the two rules, and combining it with what I was thinking, should I make a rule that transforms http://example.com/news/ into /mnt/www/site/languages/en/news/index.xml to activate mod_auth and then another one that transforms /mnt/www/site/languages/en/news/index.xml into /index.php so the system works?. Is that even possible? How would mod_auth know that is it's time to work?

If not, should I then make a rule to transform http://example.com/news/ into /mnt/www/site/languages/en/news/index.xml to activate mod_auth and then another one to transform http://example.com/news/ into /index.php so the php system works. Again the same questions as before.

I think that I'm looking at this from the wrong perspective, I'm asking something too complicated or ...

Again, thanks for all the help and time. :)

Bye

[lucy24]

Looking again at your initial post I am now seriously confused.

First paragraph:

I'm working on a personal site made in PHP where all the requests are redirected to /index.php and from there the whole system triggers what is needed, showing a page, validating user input, etc. That is working with no problems so far.

... and I'm following the description with no problems so far ;)

But then second paragraph:

The content is made of .xml files but the url's are .php, so for instance when you request http://example.com/news/upload/index.php, the content you are actually getting is from /content/languages/en/news/upload/index.xml

So which is it? :( Are the "real" pages .php or .xml? Or, to put it differently, what does "request ... index.php" mean? I could see a user requesting php pages and getting served xml-- or requesting extensionless and getting served php, or any other permutation you could make up. But the user would never request index.anything.

[g1smd]

In your case, it is the root /index.php script that makes an internal request to the deep folder to get a file which is then used to build the content for the page that is returned.

Since there isn't a direct HTTP request to the deep internal folder, you can't use HTaccess and HTpasswd to protect those page requests where the data for the page is in the deep folder.

You can use htaccess and htpasswd to stop people directly requesting the files in the deep folder, and nothing more.

You'll need to build the authentication for the URLs you don't want accessed, as part of the PHP script.In this case, htaccess and htpasswd can't help you.

[PatomaS]

hi

@lucy24
Yes, the real files are .xml files, and all the content is inside the /languages/en/... folders.

My site was all .php before, so bookmarks and some links could already have the .php included, so all the pages are accessible with .php extension in the url.

The site has a /index.php which is the one doing all the work, well, not all, it's the 'brains', the actual working files, also .php, are in a central or 'kernel' folder.

@g1smd
Yes, so it seems. I was hoping there was some way to intercept the call for the actual file, the .xml and use the auth, and after that, when the control returns to the server, do the rest of the transformation/rewriting. But I suppose it's not possible.

In any case, thank you so much for all the time and effort. :)

Bye