multiple password requests

[tthdoc]

A little background. I have a domain example.com. The host also allows multiple addon domains which are in folders. Wanted to password protect example.com, but this would have caused all subfolders (hence the addon domains) to be protected as well. So an htaccess file was placed in the root directory, and a folder called example was created. So now if you go to www.example.com you are redirected to the folder example. We are using wordpress and there are links to wmv files in example/videos folder. When you click on the link, it asks again for the username and password. It does this each time you click on a video link. This only occurs with IE not FF. I asked the host support and they replied:

"IE may not be saving password information once you have entered in, but this is a browser setting and not anything we can change on our server."

I am hoping they are wrong and there is something I can do on my end to fix it. Here is the htaccess file in the root directory that they had me use. (I left the commented lines that was in their original script)

-----------------------
# Do not change this line.
RewriteEngine on

# Change yourdomain.com to be your main domain.
# RewriteCond %{HTTP_HOST} ^(www.)?yourmaindomain.com$
RewriteCond %{HTTP_HOST} ^(www.)?example.com$

# Change 'subfolder' to be the folder you will use for your main domain.
# RewriteCond %{REQUEST_URI} !^/subfolder/
RewriteCond %{REQUEST_URI} !^/example/

# Don't change this line.
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d

# Change 'subfolder' to be the folder you will use for your main domain.
# RewriteRule ^(.*)$ /subfolder/$1
RewriteRule ^(.*)$ /example/$1

# Change yourdomain.com to be your main domain again.
# Change 'subfolder' to be the folder you will use for your main domain
# followed by / then the main file for your site, index.php, index.html, etc.
# RewriteCond %{HTTP_HOST} ^(www.)?yourmaindomain.com$
# RewriteRule ^(/)?$ subfolder/index.php [L]

RewriteCond %{HTTP_HOST} ^(www.)?example.com$
RewriteRule ^(/)?$ example/index.php [L]
----------------------

Is there anything wrong or that I can change in this file to correct it. I have tried changing the links to http://www.example.com/videos/video.wmv and http://example.com/videos/video.wmv and it still asks for the username and password each time you click on the link.

I hope there is a resolution to this..

Thanks

Doc

[jdMorgan]

Where is the code you posted above located? -- Is it in example.com/.htaccess? If so, what is in example.com/example/.htaccess -- the folder for your main/wordpress site?

The critical item --aside from the errors and redundancies in your cut-and-paste code-- is that the main site's authorization code must not be located in the example.com/.htaccess, or it will apply to all subdirectories.

Jim

[tthdoc]

Ok, my knowledge of apache is limited (but learning) so I will answer as best I can.

First - the code listed above is in example.com/.htaccess. As I understand, it is needed there to redirect requests for example.com to go to example.com/example.

Second - this is the code from the .htaccess in the example.com/example folder:

-----------------
AuthType Basic
AuthName "Restricted"
AuthUserFile "/home/-edited-/.htpasswds/public_html/example/passwd"
require valid-user
# BEGIN WordPress

# END WordPress
----------------------

Third - if I understand correctly, you said "the main site's authorization code must not be located in the example.com/.htaccess", and I believe it is not (it is in the example.com/example directory, correct?)

Last, you said "aside from the errors and redundancies in your cut-and-past code". What are they? Is there something I should change.

Thanks in advance for all the help....

Doc

[g1smd]

I don't know where to start with this.

You'll need [L] on the end of every rule.

You have a load of conditions that are separated out. Are they supposed to have a rule after each, or are they really all together and preceding one rule?

I find it easier on the eye to group all lines forming one rule so that they are together.

Your condition on the final rule creates duplicate content. Drop that condition, and instead insert a redirect to the canonical form before that rewrite.

When you mention "change" I don't understand if you are connecting a URL request to an internal filepath, or externally redirecting a URL to a new URL. The notes are not clear.

[tthdoc]

Ok, first, I changed the file to eliminate all the comments from the host file I used. Here is the cleaned up version (including adding [L} at the end of the rule that was missing one). I tested it again and the problem still occurs. Also, this file was supplied by the host, so I cannot answer the technical questions about the method.

-------------
RewriteEngine on

RewriteCond %{HTTP_HOST} ^(www.)?example.com$
RewriteCond %{REQUEST_URI} !^/example/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d

RewriteRule ^(.*)$ /example/$1 [L]

RewriteCond %{HTTP_HOST} ^(www.)?example.com$
RewriteRule ^(/)?$ example/index.php [L]

---------------------
As I said, I am not well versed in this, so I am not sure what you mean by:

"Your condition on the final rule creates duplicate content. Drop that condition, and instead insert a redirect to the canonical form before that rewrite."

As to "change", I meant was there anything wrong with the code that should be changed (like adding [L] at the end of each rule).

As to what I am doing, perhaps this would be clearer:

Domain is example.com.

There are addon domains so in example.com there was the main site files, and then folders for each addon domain, for example addon1 and addon2.

If I password protected example.com, then addon1.com and addon2.com would all be password protected as well.

So to get around this, everything was moved out of example.com and put in a folder named example. So in the public_html directory you now see folders example, addon1, addon2, and the .htaccess file you see above. Now I can protect the example folder so when you go to example.com, it redirects to example.com/example and it is password protected. BUT, there is a folder called videos in the example folder (example.com/example/videos) with a bunch of wmv files. Once logged into example.com with a username and password, you should not have to enter it again each time you click on one of the wmv file links.

Does this help clarify what I am trying to accomplish and the problem with it asking over and over for a password?

Thanks,

Doc

[jdMorgan]

OK, that sounds like most of the code is where it should be, except for the rewrite to WordPress. We need to simplify this and get rid of the redundancies to get anywhere, though.

-----

In example.com/.htaccess:

# Enable rewriting engine
RewriteEngine on
#
# Externally redirect to canonicalize the main domain
RewriteCond %{HTTP_HOST} !^(www\.example\.com)?$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]
#
# Internally rewrite requests for the main domain to
# the main domain's subdirectory if not already done
RewriteCond %{HTTP_HOST} ^(www\.example\.com)?$
RewriteCond $1 !^example/
RewriteRule (.*) /example/$1 [L]

(Delete or comment-out your existing rules (those that you've posted, anyway), as they're no longer needed)

-----

In example.com/example/.htaccess:

# Set up authorization
AuthType Basic
AuthName "Restricted"
AuthUserFile "/home/-edited-/.htpasswds/public_html/example/passwd"
require valid-user
#
# Enable rewriting engine
RewriteEngine on
#
# BEGIN WordPress
# If not already rewritten to /example/index.php
RewriteCond $1 !^index\.php$
# And not an image, CSS, JavaScript, text file, xml, or rdf file
RewriteCond $1 !\.(gif¦jpe?g¦png¦css¦js¦txt¦xml¦rdf)$ [NC]
# And if requested URL does not resolve to an existing file
RewriteCond %{REQUEST_FILENAME} !-f
# And if requested URL does not resolve to an existing directory
RewriteCond %{REQUEST_FILENAME} !-d
# Then internally rewrite the requested URL to the WordPress index.php file
RewriteRule (.*) /example/index.php [L] 
# END WordPress 

-----

The first RewriteCond in the /example.com/example/.htaccess code prevents an 'infinite' rewriting loop, while the second one --which is optional, but recommended-- prevents unnecessarily calling the filesystem to check for file- and directory-exists on every single request, which is a CPU-intensive time-waster. The more commonly-requested filetypes you can exclude in the list the better, but there is no real need to exhaustively list all of them -- The purpose is simply to save time for the most common request cases. You also don't want to be rewriting requests for robots.txt, sitemap.xml, w3c/p3p.xml, or labels.rdf to WordPress, regardless of whether you use them or if they exist; If you don't use them, then you probably want requests for them to return a 404-Not Found, and if you do use them, then you certainly don't want to rewrite requests for them to WordPress but you also probably want a 404 if they go accidentally/unexpectedly-missing.

I've added a redirect to the example.com/.htaccess to canonicalize your main domain name. You should pick either "example.com" or "www.example.com" as the preferred domain name, and link only to that domain from within your site. This prevents duplicate content and also removes the requirement to continually check for both hostnames in the .htaccess code. I show "www.example.com", but if you pick "example.com" then change all the hostname references in the code above in a consistent manner.

With an existing site, you generally want to use the currently most-linked-to/highest-ranking domain as the canonical domain, although sometimes "branding" considerations can override this if you're willing to "take the hit" on ranking for a few weeks/months in order to re-establish your "brand."

Note that in both rules in example.com/.htaccess, the hostname pattern is enclosed in "()?", making a request with a blank hostname behave as if it were a request for the correct hostname. This is to support HTTP/1.0 requests, where the client will not supply a hostname. While this support is not needed on name-based virtual servers (which are inaccessible to HTTP/1.0), it is important on IP-based virtual servers, Virtual Private Servers, and dedicated servers. It is cheap insurance against real (though now very rare) HTTP/1.0 requests, and against spoofed HTTP/1.0 requests or badly-coded 'bots, which could cause a self-inflicted denial of service attack as the server continually tries to redirect a blank-hostname HTTP/1.0 request to the correct domain, and then redirects again because the post-redirect client request won't contain a hostname, either.

Important: Replace the broken pipe "¦" characters in the regex patterns with solid pipe characters before use; Posting on this forum modifies the pipe characters.

Once you get this part working, you should consider adding at least two more redirects. The first should redirect any direct client requests for "example.com/example/index.php back to "example.com/" to correct inadvertent exposure of the WordPress PHP file's existence, and the second should redirect any direct client requests for "example.com/example/<anything-else> back to "example.com/<anything-else>" to similarly correct any inadvertent 'exposures' of the main domain subdirectory's existence.

An aside: For those who think that all mod_rewrite issues have to do with the syntax of the code itself, the side topics covered here should be an eye-opener... :)

Jim

[tthdoc]

Did not work, to verify, I cut and paste exactly what I have so you can let me know if I missed something.

In example/.htaccess:

###original code###
#RewriteEngine on
#
#RewriteCond %{HTTP_HOST} ^(www.)?example.com$
#RewriteCond %{REQUEST_URI} !^/example/
#RewriteCond %{REQUEST_FILENAME} !-f
#RewriteCond %{REQUEST_FILENAME} !-d
#
#RewriteRule ^(.*)$ /example/$1 [L]
#
#RewriteCond %{HTTP_HOST} ^(www.)?example.com$
#RewriteRule ^(/)?$ example/index.php [L]

RewriteEngine on
RewriteCond %{HTTP_HOST} !^(www\.example\.com?$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]

RewriteCond %{HTTP_HOST} ^(www\.example\.com?$
RewriteCond $1 !^example/
RewriteRule (.*) /example/$1 [L]

In example.com/example/.htaccess:

AuthType Basic
AuthName "Restricted"
AuthUserFile "/home/-edited-/.htpasswds/public_html/example/passwd"
require valid-user
# BEGIN WordPress
RewriteEngine on
RewriteCond $1 !^index\.php$
RewriteCond $1 !\.(gif¦jpe?g¦png¦css¦js¦txt¦xml¦rdf)$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) /example/index.php [L]
# END WordPress

When I go to example.com OR www.example.com, I get the following error:

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, [e-mail] and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.

I changed the example.com/.htaccess back to what it was and it worked again. I did not change the example.com/example/.htaccess back to the original, but it still is asking for the password each time I try to access the wmv files.

I did a search and found an old post that sounded similar to what is going on with my site. Perhaps you could look at it and see if that could be a clue. It was [webmasterworld.com...]

Thanks again.
Doc

[jdMorgan]

More information about this error may be available in the server error log.

So what was in your server error log?

I note that you dropped a closing parenthese from one of the RewriteCond patterns. So you need to review the code very carefully and correct this and any other errors.

Also, as I stated previously:

Important: Replace the broken pipe "¦" characters in the regex patterns with solid pipe characters before use; Posting on this forum modifies the pipe characters.

Be aware that you must get every single character correct in mod_rewrite code; It is completely and utterly unforgiving of errors.

Jim

[g1smd]

There's more than one set of unmatched brackets in the code.

[tthdoc]

Fixed one problem, now another.

Here is the .htaccess in the main directory:

RewriteEngine on
RewriteCond %{HTTP_HOST} !^(www\.example\.com)?$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]

RewriteCond %{HTTP_HOST} ^(www\.example\.com)?$
RewriteCond $1 !^example/
RewriteRule (.*) /example/$1 [L]

Apparently it is working since when I type in example.com I get the password box to enter username and password. However, when I enter it, there is a long delay, then I get a page cannot be displayed error page.

Here is the code in the example folder .htaccess:

AuthType Basic
AuthName "Restricted"
AuthUserFile "/home/-edited-/.htpasswds/public_html/example/passwd"
require valid-user
# BEGIN WordPress
RewriteEngine on
RewriteCond $1 !^index\.php$
RewriteCond $1 !\.(gif¦jpe?g¦png¦css¦js¦txt¦xml¦rdf)$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) /example/index.php [L]
# END WordPress

And yes, I replaced all the broken pipe characters.

After correcting the missing brackets, and now getting the new error, here is the server log:

[Wed Dec 31 14:13:10 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet1397747540.html
[Wed Dec 31 14:13:10 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet782564883.htm
[Wed Dec 31 14:13:10 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet1853552170.jsp
[Wed Dec 31 14:13:13 2008] [error] [client 216.35.7.110] Client sent malformed Host header [Wed Dec 31 14:13:15 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet1809113412.asp
[Wed Dec 31 14:13:15 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet2124464979.shtm
[Wed Dec 31 14:13:20 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet221837392.shtml
[Wed Dec 31 14:13:21 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet233337780.php
[Wed Dec 31 14:13:21 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/login.html
[Wed Dec 31 14:13:24 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/images
[Wed Dec 31 14:13:24 2008] [error] [client 216.35.7.110] Invalid URI in request GET local.html HTTP/1.1
[Wed Dec 31 14:13:26 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet1218672934.php3
[Wed Dec 31 14:13:26 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet764145362.php4
[Wed Dec 31 14:13:26 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/niet385358845.cfm
[Wed Dec 31 14:13:27 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/brightmail
[Wed Dec 31 14:13:29 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/sabin
[Wed Dec 31 14:13:29 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/html
[Wed Dec 31 14:13:34 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/Admin
[Wed Dec 31 14:13:34 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/system
[Wed Dec 31 14:13:34 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/version.txt
[Wed Dec 31 14:13:39 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/SProtectLinux
[Wed Dec 31 14:13:39 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/content
[Wed Dec 31 14:13:41 2008] [error] [client 216.35.7.110] Invalid URI in request GNUTELLA CONNECT/0.4
[Wed Dec 31 14:13:46 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/perl
[Wed Dec 31 14:13:51 2008] [error] [client 216.35.7.110] Invalid URI in request GET GET /servlet/admin?category=server&method=listAll&Authorization=Digest+username%3D%22admin%22%2C+response%3D%22ae9f86d6beaa3f9ecb9a5b7e072a4138%22%2C+nonce%3D%222b089ba7985a883ab2eddcd3539a6c94%22%2C+realm%3D%22adminRealm%22%2C+uri%3D%22%2Fservlet%2Fadmin%22&service= HTTP/1.0
[Wed Dec 31 14:13:53 2008] [error] [client 216.35.7.110] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /top.html, referer: [192.168.0.1...]
[Wed Dec 31 14:13:53 2008] [error] [client 216.35.7.110] File does not exist: /usr/local/apache/htdocs/idm

Thanks,

Doc

[tthdoc]

Any other suggestions? I tried all kinds of things, but nothing fixes it. I am not sure I am clear on exactly what the rewrite does, so let me know if this is correct. You type in example.com. The .htaccess file redirects you to example.com/example, but "rewrites" the URL that is shown to example.com, so you do not realize you are redirected. Now, since the example.com/example directory is protected, you are asked for the username/password and let in.

Now, as far as the issue at hand, this is where I get lost. In the code in wordpress, there is a link. Here is the code:

<a href="http://www.example.com/videos/moviefile.wmv">Movie</a>

When you click on the link, it asks again for the password and username. So it seems that for some reason, IE does not want to retain the password. I have tried various changes. If the link is as above, it asks for the username/password when you click the link, and then again when the media player comes up (twice). If I put <a href target="_blank" in first, it only asks for the password when the media player comes up. I even tried changing to:

<a ref="http://www.example.com/example/videos/moviefile.wmv">Movie</a>

and it still asks for the username/password.

Is it something in the nature of the rewrite that does not retain the password info, and is there someway in the mod-rewrite that it can be forced to be retained? (but why does FF not have a problem, and only in IE?)

Please correct me if anything I said above is incorrect (so I can better understand how the rewrite is working), and hopefully offer a solution.

Thanks

Doc

[g1smd]

*** I am not sure I am clear on exactly what the rewrite does, so let me know if this is correct. You type in example.com. The .htaccess file redirects you to example.com/example, but "rewrites" the URL that is shown to example.com, so you do not realize you are redirected. Now, since the example.com/example directory is protected, you are asked for the username/password and let in. ***

That's a convoluted description, and not the reality.

The reality is far simpler.

A redirect takes your URL request and tells the browser to go and fetch a new URL in a new HTTP transaction. The way the server does that is to return a HTTP status of 301, and the new location, in the HTTP header.

A rewrite takes your URL request and silently "translates" that request to go fetch the content from within the server at an internal filepath location that is different to the one suggested by the original URL request. That is, you ask for example.com/foo and instead of the content coming from /somepath/yoursite/htdocs/foo in the server filesystem, the content comes from some other place such as /somepath/yoursite/htdocs/somescript instead. When the content is returned, it comes with the HTTP status of "200 OK".

[jdMorgan]

Instead of debugging several problems at once, I'd suggest you comment out the login stuff, and test the redirect and rewrite rules without that additional complication. Looking at your error log with no commentary about what those requests mean or where they come from doesn't tell us much, except that it looks like the rewrites are not being applied.

Then report what URL you tested, what the expected result was, what the actual result was, and show the error log entry (if any). Please omit any entries not caused by your testing, and remember to flush your browser cache completely before requesting a URL from your server (you may just want to disable the cache temporarily, either by setting its size to zero, or by setting the 'keep things in cache' time to zero days).

Also, look at those filepaths in the error log; Are they correct/ -- As in, do those locations match the location you'd use in FTP to find the files or the script that's supposed to produce the content for those URLs?

What is with those "malformed Host headers" and "Invalid URI in request" messages? If those were just caused by typos, please let us know -- or don't post such typo-induced log entries here, as they'll only confuse the issue.

Please remember that we don't know anything about your site, and can't tell what is 'normal or expected' from what is 'obviously wrong' to you. A bit of organization in the testing and error reporting will speed up the process quite a bit.

Thanks,
Jim