homepage Welcome to WebmasterWorld Guest from 54.237.184.242
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
Problems with rewrite in .htaccess
Redirection is to an unknown path
LeeU




msg:4625741
 1:37 am on Nov 25, 2013 (gmt 0)

I have set-up an .htaccess file (listed below) for a newly designed website (the domain has been up for years). One of the errors I am receiving is when a visitor goes to the URL

http://example.com/Newsletters/5765-2005.pdf

they should be directed to

http://example.com/newsletters/5765-2005.pdf

(small "n" in "newsletters"). But they somehow are redirected to

http://example.com/articles/letters/5765-2005.pdf

The sub-directory "letters" does not exist, nor is it listed in the .htaccess file, unless it is shortening the "newsletters" directory and placing the "articles" directory in front of it (which I don't know how it would be doing that).

I have checked the page and it says it's a 301 permanent move and gives the correct directory where it has moved, i.e.,

http://example.com/newsletters/5765-2005.pdf

Any one have any ideas of what is going on?

.htaccess file


AddHandler application/x-httpd-php54s .php
AddHandler server-parsed .html
AddHandler server-parsed .htm

Options +SymLinksIfOwnerMatch
RewriteEngine on
RewriteBase /
ServerSignature Off
Options ExecCGI Includes IncludesNOEXEC SymLinksIfOwnerMatch -Indexes

# backward compatibility ruleset for rewriting document.htm to document.php
RewriteRule ^(.*)\.htm$ $1 [C,E=WasHTML:yes]
RewriteCond %{REQUEST_FILENAME}.php -f
RewriteRule ^(.*)$ $1.php [S=1,R]
RewriteCond %{ENV:WasHTML} ^yes$
RewriteRule ^(.*)$ $1.htm

# Blocks access from libwww-perl user-agents and URLS which include the command "=http:"
RewriteCond %{HTTP_USER_AGENT} libwww [NC,OR]
RewriteCond %{QUERY_STRING} ^(.*)=http [NC]
RewriteRule ^(.*)$ - [F,L]

# Redirect www to non-www
RewriteCond %{HTTP_HOST} ^www.example.com$ [NC]
RewriteRule ^(.*)$ http://example.com/$1 [R=301,L]

# Prevent hotlinking of images
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?example\.org/.*$ [NC]
RewriteRule \.(gif|jpg|swf|flv|png)$ /feed/ [R=302,L]

# Used for the errorpage.php file
ErrorDocument 400 /errorpage.php
ErrorDocument 401 /errorpage.php
ErrorDocument 403 /errorpage.php
ErrorDocument 404 /errorpage.php
ErrorDocument 500 /errorpage.php

# Caching
# ExpiresActive On
# ExpiresDefault A0
<ifModule mod_headers.c>
# 1 YEAR
<filesMatch ".(ico|pdf|flv)$">
Header set Cache-Control "max-age=29030400, public"
</filesMatch>
# 1 WEEK
<filesMatch ".(jpg|jpeg|png|gif|swf|css|js)$">
Header set Cache-Control "max-age=604800, public"
</filesMatch>
# 2 DAYS
<filesMatch ".(xml|txt)$">
Header set Cache-Control "max-age=172800, proxy-revalidate"
</filesMatch>
# 1 MIN
<filesMatch ".(html|htm|php)$">
Header set Cache-Control "max-age=60, private, proxy-revalidate"
</filesMatch>
</ifModule>
<ifModule mod_expires.c>
# any Expires Directives go here
</ifModule>
# End Caching

# compress text, html, javascript, css, xml:
FilterDeclare COMPRESS
FilterProvider COMPRESS DEFLATE resp=Content-Type $text/html
FilterProvider COMPRESS DEFLATE resp=Content-Type $text/css
FilterProvider COMPRESS DEFLATE resp=Content-Type $text/javascript
FilterChain COMPRESS
FilterProtocol COMPRESS DEFLATE change=yes;byteranges=no
# <IfModule mod_deflate.c>
# AddOutputFilterByType DEFLATE text/plain
# AddOutputFilterByType DEFLATE text/html
# AddOutputFilterByType DEFLATE text/xml
# AddOutputFilterByType DEFLATE text/css
# AddOutputFilterByType DEFLATE application/xml
# AddOutputFilterByType DEFLATE application/xhtml+xml
# AddOutputFilterByType DEFLATE application/rss+xml
# AddOutputFilterByType DEFLATE application/javascript
# AddOutputFilterByType DEFLATE application/x-javascript
# </IfModule>

# <Files 403.shtml>
# order allow,deny
# allow from all
# </Files>

<Files php.ini>
Order allow,deny
Deny from all
</Files>

RewriteRule ^Articles/?(.*)$ http://example.com/articles/$1 [R=301,L]
RewriteRule ^Events/?(.*)$ http://example.com/events/$1 [R=301,L]
RewriteRule ^News/?(.*)$ http://example.com/articles/$1 [R=301,L]
RewriteRule ^Newsletters/?(.*)$ http://example.com/newsletters/$1 [R=301,L]

 

lucy24




msg:4625743
 2:56 am on Nov 25, 2013 (gmt 0)

Options +SymLinksIfOwnerMatch
<snip>
Options ExecCGI Includes IncludesNOEXEC SymLinksIfOwnerMatch -Indexes

Stop right there.

Edit:
Oh, all right, I continued looking.
RewriteRule ^News/?(.*)$ http://example.com/articles/$1 [R=301,L]
RewriteRule ^Newsletters/?(.*)$ http://example.com/newsletters/$1 [R=301,L]

As cosgan dot de would say:
/images/smilie/froehlich/a065.gif

Further edit:
I understand the intention, but it can't be done that way. The directory slash is only optional if there is nothing after it. So the pattern is
blahblah(/.*)?$
without capture, or
blahblah(?:/(.*))?$
with capture. This only works in RegEx engines that don't object to potentially undefined back-references. mod_rewrite is fine with it. Using ?: in the outer group lets you stick with $1 in the target; otherwise it would become $2.

LeeU




msg:4625744
 3:20 am on Nov 25, 2013 (gmt 0)

If I change the line

RewriteRule ^Newsletters/?(.*)$ http://example.com/newsletters/$1 [R=301,L]

to

RewriteRule ^Newsletters(/.*)?$ http://example.com/newsletters/$1 [R=301,L]

I still get the error.

I just saw your addition. I will try that.

LeeU




msg:4625745
 3:25 am on Nov 25, 2013 (gmt 0)

Nope, still doesn't work with:

RewriteRule ^Newsletters(?:/(.*))?$ http://example.com/newsletters/$1 [R=301,L]
LeeU




msg:4625746
 3:28 am on Nov 25, 2013 (gmt 0)

Interestingly, the others work. Only the that doesn't is for the newsletters directory. Would it matter if it directed to PDF files to the directory? (Just trying to make some sense.)

lucy24




msg:4625753
 4:38 am on Nov 25, 2013 (gmt 0)

The problem is not in Newsletters. The problem is in the immediately preceding line, News.

aakk9999




msg:4625801
 1:51 pm on Nov 25, 2013 (gmt 0)

The sub-directory "letters" does not exist, nor is it listed in the .htaccess file,

Remove the word "News" from the word "Newsletters" and you get "letters".

The problem is not in Newsletters. The problem is in the immediately preceding line, News.

Exactly!

@LeeU, swap the order of rules to list your Newsletters redirect before your News redirect.

LeeU




msg:4625807
 2:26 pm on Nov 25, 2013 (gmt 0)

That's it! Thanks, @aakk9999! It works just fine now.

For my own information, why did it change it the way it did, in the order I had in the original listing?

aakk9999




msg:4625833
 4:00 pm on Nov 25, 2013 (gmt 0)

Because you need to put rules in the order of more specific to less specific. Your URL with "Newsletters" was executing "News" rule, because "News" is subset of "Newsletters"

^News/?(.*)$ says "Anything starting with "News" or "News/"

The (.*) after the News in URLs with "Newsletters" then captured the pattern "letters" and this is waht got inserted after the /articles/ (referred to by $1).

When you swapped the lines around, URLs with "Newsletters" matched the ^Newsletters rule first and this one got executed.

LeeU




msg:4625871
 5:32 pm on Nov 25, 2013 (gmt 0)

Ah, that makes sense now. Thanks again.

(Now, how do I marked this "solved"?)

lucy24




msg:4625952
 1:35 am on Nov 26, 2013 (gmt 0)

No such button :)

But that was the point of fine-tuning to
(?:/(.*))?$
With this configuration, the rules will work regardless of order, even if one pattern name is contained within another. It also eliminates a problem in the original form, where a filename might be captured with or without leading directory slash.

aakk9999




msg:4625959
 2:57 am on Nov 26, 2013 (gmt 0)

So changing the News rule would allow any order of rules and it is probably a safer solution to use:

RewriteRule ^News(?:/(.*))?$ http://example.com/articles/$1 [R=301,L]

Thank you Lucy for explaining :)

LeeU




msg:4626080
 3:16 pm on Nov 26, 2013 (gmt 0)

Ah, now I get it. Yes, thanks for the explanation. It makes sense now. And it works just fine (someday I'll learn regular expression properly).

Thanks to both of you!

g1smd




msg:4626484
 2:06 am on Nov 28, 2013 (gmt 0)

You're not quite done. There's a number of other issues to fix up. These include:
- rewriterules that block access or prevent hot-linking should be listed first
- jpg|jpeg| simplifies to jpe?g|
- [F,L] flag should be [F]
- redirects should be listed before rewrites
- the www to non-www redirect should be the last of the redirects
- the RegEx pattern ^www.example.com$ [NC] should be just !^(example\.com)?$
- in any rewrite with rule target beginning $1, the rule target must be prefixed with a slash
- missing escaping in multiple RegEx patterns.

You might also usefully list the "deny from" rules at the very top of the file.

LeeU




msg:4627311
 10:34 pm on Dec 2, 2013 (gmt 0)

Thanks, g1smd. I cleaned it up with those tips. It seems there are many articles about the .htaccess file but I have never seen one that explains the order things should be done (although it always seemed like there should be one).

LeeU




msg:4627533
 5:23 pm on Dec 3, 2013 (gmt 0)

Okay, well after doing a site search on Google, I have found I have problems with another domain ("example.com") and one of its sub-domains ("example2.com"). I have listed below excerpts from the .htaccess files (you can see that I have attempted to fix this problem before but, evidently, not very effectively).

One problem I can see is the rewrite to the "blosxom.cgi" file. I need to differentiate between example.com and example2.com domain/sub-domain.

I would appreciate some more help, if possible.

MAIN DOMAIN - excerpted portion


RewriteEngine on

Redirect 301 /emet/weekly.xml http://example2.com/
Redirect 301 /rss-info.htm http://example2.com/rss-info.php
Redirect 301 /articles//going/feedback.php http://www.example.com/going/feedback.php
Redirect 301 /cgi-bin/2me_4see_comment.cgi http://www.example.com/contact.php
Redirect 301 /emet/analysis/arab_claim_to_palestine.htm http://example2.com/analysis/arab_claim_to_palestine.php
Redirect 301 /articles/important-jeru.php http://www.example.com/articles/what-is-so-important-about-jerusalem.php
Redirect 301 /articles/important-jeru.htm http://www.example.com/articles/what-is-so-important-about-jerusalem.php
Redirect 301 /articles/histjeru.php http://www.example.com/articles/a-biblical-history-of-jerusalem.php
Redirect 301 /articles/histjeru.htm http://www.example.com/articles/a-biblical-history-of-jerusalem.php
Redirect 301 /articles/jerucov.php http://www.example.com/articles/jerusalem-a-separate-covenant.php
Redirect 301 /articles/jerucov.htm http://www.example.com/articles/jerusalem-a-separate-covenant.php
Redirect 301 /articles/seventhmonth.php http://www.example.com/articles/index.php
Redirect 301 /articles/touch_not_mine_anointed.php http://www.example.com/articles/index.php
Redirect 301 /articles/camel-needle.php http://www.example.com/articles/index.php
Redirect 301 /articles/camel-needle.htm http://www.example.com/articles/index.php
Redirect 301 /bookReviews/review-paul_the_jewish_theologian.php http://www.example.com/books.php
Redirect 301 /articles/spiritual-authority-what-it-is-and-what-it-isnt.php http://www.example.com/articles/index.php
Redirect 301 /articles/godsplan.php http://www.example.com/articles/index.php
Redirect 301 /articles/blessings.php http://www.example.com/articles/index.php
Redirect 301 /articles/fathers_love.php http://www.example.com/articles/index.php
# Redirect 301 /book_reviews/ http://www.example.com/books.php
Redirect 301 /articles/harry_potter.php http://www.example.com/articles/index.php
Redirect 301 /articles/tradmen.php http://www.example.com/articles/index.php
Redirect 301 /articles/pastors.php http://www.example.com/articles/index.php
Redirect 301 /articles/covering.php http://www.example.com/articles/index.php
Redirect 301 /articles/bride_ready.php http://www.example.com/articles/index.php

RewriteRule ^weblog/?(.*)$ /cgi-bin/blosxom.cgi/$1

RewriteRule ^(.+)\.htm$ http://www.example.com/$1.php [R=301,L]

RewriteCond %{HTTP_HOST} ^example\.org [OR]
RewriteCond %{HTTP_HOST} ^69\.89\.23\.241
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]

RewriteRule ^views/?(.*)$ http://example2.com/views/$1 [R=301,L]
RewriteRule ^analysis/?(.*)$ http://example2.com/analysis/$1 [R=301,L]
RewriteRule ^documents/?(.*)$ http://example2.com/documents/$1 [R=301,L]
RewriteRule ^palestinian-myths/?(.*)$ http://example2.com/palestinian-myths/$1 [R=301,L]
RewriteRule ^media/?(.*)$ http://example2.com/media/$1 [R=301,L]


==========================

SUB-DOMAIN - excerpted portion


RewriteEngine On

Redirect 301 /articles/ http://www.example.org/articles/histjeru.php
Redirect 301 /analysis/gaza_and_katrina.php http://example2.com/analysis/gaza-and-katrina.php
Redirect 301 /analysis/palestine-history.pdf http://example2.com/analysis/palestine_history.pdf
Redirect 301 /documents/unsc_resolution_338.php http://example2.com/documents/unsc-resolution-338.php
Redirect 301 /analysis/the_myth_of_occupied_territories.php http://example2.com/analysis/the-myth-of-occupied-territories.php
Redirect 301 /analysis/palestine_history.php http://example2.com/analysis/palestine-history.php
Redirect 301 /gaza-assistance.php http://example2.com/gaza-commentaries-booklets-pictures.php
Redirect 301 /analysis/the_israeli_humanitarian_lifeline_to_gaza.php http://example2.com/analysis/the-israeli-humanitarian-lifeline-to-gaza.php
Redirect 301 /analysis/jewish_blood_who_is_really_behind_the_gaza_flotilla.php http://example2.com/analysis/jewish-blood-who-is-really-behind-the-gaza-flotilla.php
Redirect 301 /analysis/is_israel_bound_by_international_law_to_supply_utilities_goods_and_services_to_gaza.php http://example2.com/analysis/is-israel-bound-by-international-law-to-supply-utilities-goods-and-services-to-gaza.php
Redirect 301 /contact.php http://example2.com/contact/contact.php
Redirect 301 /Blog.cgi/index.rss http://example2.com/cgi-bin/blosxom.cgi/index.rss
Redirect 301 /blog.cgi/index.rss http://example2.com/cgi-bin/blosxom.cgi/index.rss
Redirect 301 /weblog/why-must-an-arab-palestinian-state-be-jew-free.html) http://www.example2.com/weblog/why-must-an-arab-palestinian-state-be-jew-free.html
Redirect 301 /current.htm http://example2.com/
Redirect 301 /current.php http://example2.com/
Redirect 301 /weekly.xml http://example2.com/

RewriteRule ^weblog/?(.*)$ /cgi-bin/blosxom.cgi/$1

RewriteCond %{HTTP_HOST} ^www.example2.com$ [NC]
RewriteRule ^(.*)$ http://example2.com/$1 [R=301,L]

RewriteCond %{HTTP_HOST} example\.org$
RewriteRule ^(.*)$ "http\:\/\/example2\.org\/" [R=301,L]

RewriteRule ^(.+)\.htm$ http://example2.com/$1.php [R=301,L]

RedirectMatch 301 ^/palestinian_myths/$ http://example2.com/palestinian-myths/

RewriteRule ^archives/?(.*)$ http://example2.com/cgi-bin/mail.cgi/archive/emet
RewriteRule ^palestinian_myths/(.*)$ http://example2.com/palestinian-myths/$1 [R=301,L]


Thanks in advance for any help you can give.

lucy24




msg:4627583
 7:30 pm on Dec 3, 2013 (gmt 0)

GET RID OF all rules involving mod_alias (Redirect by that name). You may be able to combine mod_alias and mod_rewrite in the config file, but don't try it in htaccess.

Open a copy of your htaccess file in a text editor that does Regular Expressions and run the following global replaces:

# change . to \.
^(Redirect \d\d\d \S+?[^\\])\.
TO
$1\\.
# now change Redirect to Rewrite
^Redirect(?:Match)? 301 /(.+)
TO
RewriteRule $1 [R=301,L]
# and (if necessary)
^Redirect(?:Match)? 410 /(.+)
TO
RewriteRule $1 - [G]

Also get rid of all escaped : colons and / directory slashes AND all quotation marks in targets.

LeeU




msg:4627629
 10:07 pm on Dec 3, 2013 (gmt 0)

Thanks! I'll give it a try and get back to you.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved