Apache mod_rewrite and FrontPage extensions

A solution for the widespread incompatibility problem

[Merlin2000]

I have searched around quite awhile for a fix with the error received when using Apache's RewriteEngine and FrontPage Server Extensions.

It seems that whenever you have

Rewrite Engine On

in your .htaccess, and FPSE are enabled, a 403 error is produced after you submit a form (the form never gets submitted).

The only place I have found that gives a working solution, is found here: [msfrontpage.net...]

However that solution doesn't seem to work, and many articles on the site are outdated. (The author dicusses background fixes for NS 3/4 in one of his posts :P )

[Merlin2000]

Is there something I could help clear up about my problem or maybe someone can tell me where else I can post my question?

[jdMorgan]

1) I think it's just that not many people who hang out back here use FP extensions. There are now better ways to get the functionality, without all the hassles (like you're experiencing).

2) This forum is slow on weekends.

Jim

[Merlin2000]

Thanks for the reply Jim. I am not sure where else I can post this to get some more help.

I understand where you are coming from regarding FP- but I can make a fully functioning FP form is about 10 seconds. If you have any suggestions for something that can be done in say a few minutes, I am open to suggestions. All I use FP for are the forms as they take so little time to create something very detailed. Plus, I don't have to update any software or worry about holes.

Hopefully someone will have some info about this later on in the week.

[jdMorgan]

Any other FrontPage Extensions users in the house?
<bump>

Jim

[Merlin2000]

Would this be how I exclude the FP folder from the rewrite?

#first, correct the www if necessary 
RewriteCond %{HTTP_HOST} .
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{HTTP_HOST} !^www\.my-domain-name-here\.com [NC]
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301]
 
#then, rewrite secure folder 1 
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} !^/helpdesk/admin/parser.php [NC]
RewriteCond %{REQUEST_URI} ^/helpdesk/ [NC]
RewriteCond %{SERVER_PORT} !^443$
RewriteRule ^(.*)$ https://www.example.com/$1 [R=301,L]
 
#then, rewrite secure folder 2 
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/modernbill/ [NC]
RewriteCond %{SERVER_PORT} !^443$
RewriteRule ^(.*)$ https://www.example.com/$1 [R=301,L] 
#then, rewrite secure folder 3 
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/Secure-Order/ [NC]
RewriteCond %{SERVER_PORT} !^443$
RewriteRule ^(.*)$ https://www.example.com/$1 [R=301,L] 
 
#then, rewrite slashes for secure folder 1
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/helpdesk [NC]
RewriteCond %{REQUEST_URI} !^/helpdesk/ [NC]
RewriteRule ^(.*)$ https://www.example.com/$1/ [R=301,L]
 
#then, rewrite slashes for secure folder 2
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/modernbill [NC]
RewriteCond %{REQUEST_URI} !^/modernbill/ [NC]
RewriteRule ^(.*)$ https://www.example.com/$1/ [R=301,L]
 
#then, rewrite slashes for secure folder 3
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/Secure-Folder [NC]
RewriteCond %{REQUEST_URI} !^/Secure-Folder/ [NC]
RewriteRule ^(.*)$ https://www.example.com/$1/ [R=301,L]

[edited by: jdMorgan at 12:20 am (utc) on May 24, 2005]
[edit reason] Examplified. [/edit]

[jdMorgan]

Assuming you mean, "Is this how I exclue requests for /_vti_bin from being rewritten?" then yes, that should work.

Is this code for use in .htaccess? I'm asking because there are several major code-speed-up opportunities here.

Jim

[chopin2256]

Hi! :)

I think my problem is similar, maybe slightly different. I don't use FP forms, however I do use frontpage to manage and design my site. Everytime I change the htaccess from the www root directory, I cannot sign into Frontpage. This is the code I want in the htaccess file:
_______________________________________________________
RewriteEngine on
RewriteCond %{HTTP_HOST} ^example\.com [NC]
RewriteRule ^(.*)$ [example.com...] [L,R=301]
_______________________________________________________

However with this piece of code, I can't sign in FP :(

Is there a fix for this?

[edited by: jdMorgan at 11:53 pm (utc) on May 23, 2005]
[edit reason] Examplified. [/edit]

[jdMorgan]

chopin2256,

Have you reviewed the code above to exclude /_vti_bin? Also, everybody, let's please use example.com as the domain name... Some other example names resolve to actual sites, and you really don't want to go there. Thanks.

Jim

[Merlin2000]

Is this code for use in .htaccess? I'm asking because there are several major code-speed-up opportunities here.

- Jim, that is part of the code being used, just in case you need the other part, I will post it below:

#turn on the engine 
RewriteEngine On
 
#first, correct the www if necessary 
RewriteCond %{HTTP_HOST} .
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{HTTP_HOST} !^www\.example\.com [NC]
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301]
 
#then, rewrite secure folder 1 
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} !^/helpdesk/admin/parser.php [NC]
RewriteCond %{REQUEST_URI} ^/helpdesk/ [NC]
RewriteCond %{SERVER_PORT} !^443$
RewriteRule ^(.*)$ https://www.example.com/$1 [R=301,L]
 
#then, rewrite secure folder 2 
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/modernbill/ [NC]
RewriteCond %{SERVER_PORT} !^443$
RewriteRule ^(.*)$ https://www.example.com/$1 [R=301,L] 
#then, rewrite secure folder 3 
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/Secure-Order/ [NC]
RewriteCond %{SERVER_PORT} !^443$
RewriteRule ^(.*)$ https://www.example.com/$1 [R=301,L] 
 
#then, rewrite slashes for secure folder 1
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/helpdesk [NC]
RewriteCond %{REQUEST_URI} !^/helpdesk/ [NC]
RewriteRule ^(.*)$ https://www.example.com/$1/ [R=301,L]
 
#then, rewrite slashes for secure folder 2
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/modernbill [NC]
RewriteCond %{REQUEST_URI} !^/modernbill/ [NC]
RewriteRule ^(.*)$ https://www.example.com/$1/ [R=301,L]
#then, rewrite slashes for secure folder 3
RewriteCond %{REQUEST_URI} !^/_vti_bin [NC]
RewriteCond %{REQUEST_URI} ^/Secure-Folder [NC]
RewriteCond %{REQUEST_URI} !^/Secure-Folder/ [NC]
RewriteRule ^(.*)$ https://www.example.com/$1/ [R=301,L]
# -FrontPage-
IndexIgnore .htaccess */.?* *~ *# */HEADER* */README* */_vti*
<Limit GET POST>
order deny,allow
deny from all
allow from all
</Limit>
<Limit PUT DELETE>
order deny,allow
deny from all
</Limit>
AuthName www.example.com
AuthUserFile /home/username/public_html/_vti_pvt/service.pwd
AuthGroupFile /home/username/public_html/_vti_pvt/service.grp

This code still produces the error as well.

[edited by: jdMorgan at 12:22 am (utc) on May 24, 2005]
[edit reason] De-linked. [/edit]

[jdMorgan]

Note the path to the password files at the end of your code. These paths will be rewritten, and may break FP. You might want to exclude "/_vti_" instead of the more-specific "/_vti_bin" you are excluding now.

If it's not clear, I am guessing in this thread. I don't use FP extensions myself. At the point where the guessing becomes useless, please let me know.

Jim

[Merlin2000]

I believe this is what you were suggesting:

#turn on the engine 
RewriteEngine On
 
#first, correct the www if necessary 
RewriteCond %{HTTP_HOST} .
RewriteCond %{REQUEST_URI}!^/_vti_ [NC]
RewriteCond %{HTTP_HOST}!^www\.example\.com [NC]
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301]
 
#then, rewrite secure folder 1 
RewriteCond %{REQUEST_URI}!^/_vti_ [NC]
RewriteCond %{REQUEST_URI}!^/helpdesk/admin/parser.php [NC]
RewriteCond %{REQUEST_URI} ^/helpdesk/ [NC]
RewriteCond %{SERVER_PORT}!^443$
RewriteRule ^(.*)$ https://www.example.com/$1 [R=301,L]
 
#then, rewrite secure folder 2 
RewriteCond %{REQUEST_URI}!^/_vti_ [NC]
RewriteCond %{REQUEST_URI} ^/modernbill/ [NC]
RewriteCond %{SERVER_PORT}!^443$
RewriteRule ^(.*)$ https://www.example.com/$1 [R=301,L] 
#then, rewrite secure folder 3 
RewriteCond %{REQUEST_URI}!^/_vti_ [NC]
RewriteCond %{REQUEST_URI} ^/Secure-Order/ [NC]
RewriteCond %{SERVER_PORT}!^443$
RewriteRule ^(.*)$ https://www.example.com/$1 [R=301,L] 
 
#then, rewrite slashes for secure folder 1
RewriteCond %{REQUEST_URI}!^/_vti_ [NC]
RewriteCond %{REQUEST_URI} ^/helpdesk [NC]
RewriteCond %{REQUEST_URI}!^/helpdesk/ [NC]
RewriteRule ^(.*)$ https://www.example.com/$1/ [R=301,L]
 
#then, rewrite slashes for secure folder 2
RewriteCond %{REQUEST_URI}!^/_vti_ [NC]
RewriteCond %{REQUEST_URI} ^/modernbill [NC]
RewriteCond %{REQUEST_URI}!^/modernbill/ [NC]
RewriteRule ^(.*)$ https://www.example.com/$1/ [R=301,L]
#then, rewrite slashes for secure folder 3
RewriteCond %{REQUEST_URI}!^/_vti_ [NC]
RewriteCond %{REQUEST_URI} ^/Secure-Folder [NC]
RewriteCond %{REQUEST_URI}!^/Secure-Folder/ [NC]
RewriteRule ^(.*)$ https://www.example.com/$1/ [R=301,L]
# -FrontPage-
IndexIgnore .htaccess */.?* *~ *# */HEADER* */README* */_vti*
<Limit GET POST>
order deny,allow
deny from all
allow from all
</Limit>
<Limit PUT DELETE>
order deny,allow
deny from all
</Limit>
AuthName www.example.com
AuthUserFile /home/username/public_html/_vti_pvt/service.pwd
AuthGroupFile /home/username/public_html/_vti_pvt/service.grp

It still shoots back a 403: http://www[.]example.com/_vti_bin/shtml.exe/form.htm

[edited by: engine at 10:43 am (utc) on Aug. 26, 2005]
[edit reason] delinked [/edit]

[chopin2256]

Have you reviewed the code above to exclude /_vti_bin?

I tried this code:

RewriteEngine on
RewriteCond %{REQUEST_URI}!^/_vti_bin [NC]
RewriteCond %{HTTP_HOST} ^example\.com [NC]
RewriteRule ^(.*)$ http://www.example.com/$1 [L,R=301]

But still no luck, I can't sign into frontpage.

Any suggestions?

[edited by: engine at 11:13 am (utc) on Aug. 26, 2005]
[edit reason] delinked [/edit]

[Merlin2000]

chopin2256,

- The problem you have is the EXACT same one I have. So let's just chill for a bit while the experts lend a hand.

I have posted my question on about 4 other boards, none have any advice at all :(

[jdMorgan]

Are there any other directories or subdirectories that might need to be excluded? Taking chopin's simpler code as an example, it should work if everything that Frontpage needs starts with "/_vti_". So, look for anything else that Frontpage might not allow you to rewrite.

[added] I'd also encourage you two to compare notes. [/added]

Jim

[chopin2256]

Are there any other directories or subdirectories that might need to be excluded?

I tried this, but this didnt even do what I wanted it to do in the first place..but is this even on the right track?:

RewriteEngine on
RewriteCond %{REQUEST_URI}!^/_vti_bin [NC]
RewriteCond %{REQUEST_URI}!^/_borders [NC]
RewriteCond %{REQUEST_URI}!^/_fpclass [NC]
RewriteCond %{REQUEST_URI}!^/_private [NC]
RewriteCond %{REQUEST_URI}!^/_themes [NC]
RewriteCond %{REQUEST_URI}!^/_vti_cnf [NC]
RewriteCond %{REQUEST_URI}!^/_vti_log [NC]
RewriteCond %{REQUEST_URI}!^/_vti_pvt [NC]
RewriteCond %{REQUEST_URI}!^/_vti_txt [NC]
RewriteCond %{REQUEST_URI}!^/_vti_inf.html [NC]
RewriteCond %{HTTP_HOST} ^example\.com [NC]
RewriteRule ^(.*)$ http://www.example.com/$1 [L,R=301]

I am assuming that these are some of the frontpage directories. This code does not work.

[edited by: engine at 11:14 am (utc) on Aug. 26, 2005]
[edit reason] delinked [/edit]

[jdMorgan]

Please be specific. What do you mean "it did not work"? Did it cause a 500 Server Error? If so, what's in your server error log file? If not, then how did the results differ from your expectations?

You don't have to list each and every directory and file... For example, the second RewriteCond below takes care of anything that starts with /_vti_ and so covers lots of directories and files with only one line of code. Also, you should not need the [NC] flag unless you have some incorrect-case links on your site. The idea is to make the code as lean and mean as possible, so you don't slow down your server under heavy load.

RewriteEngine on
RewriteCond %{HTTP_HOST} ^example\.com [NC]
RewriteCond %{REQUEST_URI} !^/_vti_
RewriteCond %{REQUEST_URI} !^/_borders
RewriteCond %{REQUEST_URI} !^/_fpclass
RewriteCond %{REQUEST_URI} !^/_private
RewriteCond %{REQUEST_URI} !^/_themes
RewriteRule ^(.*)$ http://www.example.com/$1 [L,R=301]

Note also that the space between "}" and "!" is required, and I changed the order of the code so that normal requests (for "www") will run faster; they will fail to match on the first RewriteCond, and therefore "skip" the rule faster.

This is a very strange problem. It's a wonder that there are not more discussions of solutions for it on the Web.

Jim

[chopin2256]

The code performs the desired task by changing the non-www to the www, which is what I want.

However, I cant login to Frontpage with the optimized code..unless of course i take the code out. Yes this is so weird, this is why I never even used hotlink protection, because its inconvenient to disable the code every time I want to use frontpage to design my site. If we figure this out, I can also use hotlink finally :) But this is really important, because I need the non-www to be redirected big time. I may have lost my serps in Google due to this! And I like frontpage!

Merlin...did I miss any subcategories that Frontpage uses? I'm not too good with this htaccess stuff...I actually just learned what it did today.

[Merlin2000]

Jim, sounds like it would be better to test using more simple code, like what chopin2256 is using.

Here is a test that I believe will make a huge difference. A blank .htaccess file with RewriteRules On and nothing else listed, will produce a 403 error on forms, search, various other FPSE components, as well not allow the publish using FrontPage.

I am going to setup a test account on my server, just to make sure everything is correctly configured. Be back in 10 minutes to report my findings. Also Jim, if you have any ideas how to make my .htaccess code perform faster, that would be great :)

[Merlin2000]

The Form and the Search pages both use FPSE and produce the same error.

Here is the simple .htaccess code I am using:

RewriteEngine on
RewriteCond %{HTTP_HOST} ^example\.com [NC]
RewriteCond %{REQUEST_URI}!^/_vti_
RewriteCond %{REQUEST_URI}!^/_borders
RewriteCond %{REQUEST_URI}!^/_fpclass
RewriteCond %{REQUEST_URI}!^/_private
RewriteCond %{REQUEST_URI}!^/_themes
RewriteCond %{REQUEST_URI}!^/testing
RewriteRule ^(.*)$ http://example.com/$1 [L,R=301] 
# -FrontPage-
IndexIgnore .htaccess */.?* *~ *# */HEADER* */README* */_vti*
<Limit GET POST>
order deny,allow
deny from all
allow from all
</Limit>
<Limit PUT DELETE>
order deny,allow
deny from all
</Limit>
AuthName wwWebmasterWorldd.info
AuthUserFile /home/mbwdinf/public_html/_vti_pvt/service.pwd
AuthGroupFile /home/mbwdinf/public_html/_vti_pvt/service.grp

I tried the code in public_html as well as public_html/testing, and then tried it with both and it did not work....

This really is a pain in the ass, hopefully someone will have a clue about this. :)

[edited by: jdMorgan at 3:22 am (utc) on Aug. 26, 2005]
[edit reason] Removed unlinked URL per TOS. [/edit]

[jdMorgan]

The code in msg31 redirects example.com to itself, so that's a problem.
Also, I am not sure that the "allow deny" stuff is completly correct.

However, if all it takes to break FrontPage is to add "RewriteEngine on" to a blank .htaccess, these two points are moot. Obviously, it is not the RewriteRules which break FrontPage extensions, but rather the invokation of mod_rewrite. As a result, no number of exclusions to the RewriteRules, is likely to help.

The key to solving this problem is just that - to avoid complicating the problem until the basic cause is found. After that, we can discuss optimizing the code and other side issues. But for now, the central mystery is, "Why does enabling the RewriteEngine cause a problem?"

The only thing that comes to mind is an inheritance issue. And the only thing I can think of to try is to put something like:

RewriteEngine off

into a minimal .htaccess file (leave the FP stuff in, but remove all RewriteRules). See if that works.

If so, try:

RewriteEngine on
RewriteEngine off

And if that works, then put one simple RewriteRule in between the 'on' and 'off', such as,

RewriteEngine on
RewriteRule ^foo\.html$ /any_static_file_that_exists.html [L]
RewriteEngine off

And so on, proceeding stepwise through the various levels of 'activating' mod_rewrite to see where the problem comes in. If a failure is encountered, then check the server error log file if available, and examine the raw access log too, in case there's anything interesting or strange in there.

The 'RewriteEngine off' directive is not normally needed in .htaccess. But it *may* affect the way that code in .htaccess files below the current directory behaves if 'RewriteOptions Inherit' has been set.

Again, sorry I can't be of more help -- These are just guesses at a way to proceed. The more closely the problem can be defined, the more likely that a solution may be found.

Jim

[Merlin2000]

This code lets FPSE run correctly, but the RewriteRules do nothing at all:

RewriteEngine On
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST}!^www\.example\.com [NC]
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301] 
RewriteEngine Off
# -FrontPage-
IndexIgnore .htaccess */.?* *~ *# */HEADER* */README* */_vti*
<Limit GET POST>
order deny,allow
deny from all
allow from all
</Limit>
<Limit PUT DELETE>
order deny,allow
deny from all
</Limit>
AuthName wwWebmasterWorldd.info
AuthUserFile /home/mbwdinf/public_html/_vti_pvt/service.pwd
AuthGroupFile /home/mbwdinf/public_html/_vti_pvt/service.grp

[edited by: engine at 11:15 am (utc) on Aug. 26, 2005]
[edit reason] delinked [/edit]

[wilderness]

Here's a decent FP Forum.
Somebody there may be able to help.
[frontpagewebmaster.com...]

[Merlin2000]

Already did, suprised I didn't receive a single response!

[frontpagewebmaster.com...]

[Merlin2000]

I found a rather outdated post from someone that lives pretty close to me: [archives.neohapsis.com...]

I gave him a call and left a message, hopefully he will respond. His suggestion of using the Options SymLinksIfOwnerMatch did not work, I also tried Options +SymLinksIfOwnerMatch which also did not work. The end result was exactly the same :/

I'll keep looking and updating this thread. We have tried many things so far, and documented all of it. Hopefullt someone will come along with some more suggestions.

---------
Chris finally replied saying he didn't quite remember everything he did, but played a bit with the httpd.conf for Apache, and said something he did got it to work.

[Merlin2000]

Btw Jim, do you have a suggestion for cleaning up my code a bit? You meantioned something earlier that suggests there might be quicker/better ways of doing what I want.

[jdMorgan]

Yes, I really want to avoid diversions from getting an answer to the problem that you and chopin2256 are experiencing. This comes up from time to time, and I'd like to see an answer.

However, once you nail down the filenames that need to be excluded from all rewrites, it's a simple matter to put a rule like

RewriteRule ^_vti_bin - [L]

ahead of the other rules. This will stop mod_rewrite processing without changing the URL, thereby skipping all subsequent rules. In cases where it is required that only some rules be skipped, you can use the [S=nn] flag to skip the next nn rules. This avoids having to have the same RewriteCond repeated in each rule.

Again, this is a minor point and I'd like to stay on-topic. Perhaps you could get your 'contact' to send you a copy of his httpd.conf file with all 'private stuff' removed for dissection and analysis...

Jim

[chopin2256]

I am just so "shocked" that no one else has this problem. No one uses Frontpage anymore? It just seems that it would be a more common problem but I guess not. I even posted a question regarding the same exact incident here [webmasterworld.com...] and got no answer. I have to let in the leechers because of this problem. Merlin, lets work together to try to figure this out, (and Jim of course) because it looks like both you and me are the only ones on the internet that have this problem :P

[jdMorgan]

Yes, by all means, "try stuff." Especially if you have the option to do the rewrites in httpd.conf instead of .htaccess.

Jim

[Merlin2000]

Jim

Something like this is what I would need, right?

<VirtualHost *>
ServerName example.com
..the rest
Redirect 301 / http://www.example.com/
</VirtualHost>

If the above didn't loop, it would redirect only example.com and not example.com/another-folder

If there is code that I can use to add a www before a example.com, as well as a way to make certain folders change to https, then I would love to use that! I just didn't think you could do this in the apache config file.

I am just so "shocked" that no one else has this problem. No one uses Frontpage anymore? It just seems that it would be a more common problem but I guess not.

- There are a lot of people, anyone who uses FP when they modify their .htaccess, should experience this problem. Most of the time these people are newbs that are told not to use .htaccess, and use something in FP to do what is needed.

Merlin, lets work together to try to figure this out,

- Before I came here, I tried just about everything I could- I rarely have a FP related question that I cannot figure out. I think the only person who can answer this is a .htaccess/FP guru. You have to be able to code in .htaccess, as well as understand how FPSE works.

[edited by: engine at 11:17 am (utc) on Aug. 26, 2005]