Welcome to WebmasterWorld Guest from 54.162.168.187

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

Problems with rewrite in .htaccess

Redirection is to an unknown path

     
1:37 am on Nov 25, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


I have set-up an .htaccess file (listed below) for a newly designed website (the domain has been up for years). One of the errors I am receiving is when a visitor goes to the URL

http://example.com/Newsletters/5765-2005.pdf


they should be directed to

http://example.com/newsletters/5765-2005.pdf


(small "n" in "newsletters"). But they somehow are redirected to

http://example.com/articles/letters/5765-2005.pdf


The sub-directory "letters" does not exist, nor is it listed in the .htaccess file, unless it is shortening the "newsletters" directory and placing the "articles" directory in front of it (which I don't know how it would be doing that).

I have checked the page and it says it's a 301 permanent move and gives the correct directory where it has moved, i.e.,

http://example.com/newsletters/5765-2005.pdf


Any one have any ideas of what is going on?

.htaccess file


AddHandler application/x-httpd-php54s .php
AddHandler server-parsed .html
AddHandler server-parsed .htm

Options +SymLinksIfOwnerMatch
RewriteEngine on
RewriteBase /
ServerSignature Off
Options ExecCGI Includes IncludesNOEXEC SymLinksIfOwnerMatch -Indexes

# backward compatibility ruleset for rewriting document.htm to document.php
RewriteRule ^(.*)\.htm$ $1 [C,E=WasHTML:yes]
RewriteCond %{REQUEST_FILENAME}.php -f
RewriteRule ^(.*)$ $1.php [S=1,R]
RewriteCond %{ENV:WasHTML} ^yes$
RewriteRule ^(.*)$ $1.htm

# Blocks access from libwww-perl user-agents and URLS which include the command "=http:"
RewriteCond %{HTTP_USER_AGENT} libwww [NC,OR]
RewriteCond %{QUERY_STRING} ^(.*)=http [NC]
RewriteRule ^(.*)$ - [F,L]

# Redirect www to non-www
RewriteCond %{HTTP_HOST} ^www.example.com$ [NC]
RewriteRule ^(.*)$ http://example.com/$1 [R=301,L]

# Prevent hotlinking of images
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?example\.org/.*$ [NC]
RewriteRule \.(gif|jpg|swf|flv|png)$ /feed/ [R=302,L]

# Used for the errorpage.php file
ErrorDocument 400 /errorpage.php
ErrorDocument 401 /errorpage.php
ErrorDocument 403 /errorpage.php
ErrorDocument 404 /errorpage.php
ErrorDocument 500 /errorpage.php

# Caching
# ExpiresActive On
# ExpiresDefault A0
<ifModule mod_headers.c>
# 1 YEAR
<filesMatch ".(ico|pdf|flv)$">
Header set Cache-Control "max-age=29030400, public"
</filesMatch>
# 1 WEEK
<filesMatch ".(jpg|jpeg|png|gif|swf|css|js)$">
Header set Cache-Control "max-age=604800, public"
</filesMatch>
# 2 DAYS
<filesMatch ".(xml|txt)$">
Header set Cache-Control "max-age=172800, proxy-revalidate"
</filesMatch>
# 1 MIN
<filesMatch ".(html|htm|php)$">
Header set Cache-Control "max-age=60, private, proxy-revalidate"
</filesMatch>
</ifModule>
<ifModule mod_expires.c>
# any Expires Directives go here
</ifModule>
# End Caching

# compress text, html, javascript, css, xml:
FilterDeclare COMPRESS
FilterProvider COMPRESS DEFLATE resp=Content-Type $text/html
FilterProvider COMPRESS DEFLATE resp=Content-Type $text/css
FilterProvider COMPRESS DEFLATE resp=Content-Type $text/javascript
FilterChain COMPRESS
FilterProtocol COMPRESS DEFLATE change=yes;byteranges=no
# <IfModule mod_deflate.c>
# AddOutputFilterByType DEFLATE text/plain
# AddOutputFilterByType DEFLATE text/html
# AddOutputFilterByType DEFLATE text/xml
# AddOutputFilterByType DEFLATE text/css
# AddOutputFilterByType DEFLATE application/xml
# AddOutputFilterByType DEFLATE application/xhtml+xml
# AddOutputFilterByType DEFLATE application/rss+xml
# AddOutputFilterByType DEFLATE application/javascript
# AddOutputFilterByType DEFLATE application/x-javascript
# </IfModule>

# <Files 403.shtml>
# order allow,deny
# allow from all
# </Files>

<Files php.ini>
Order allow,deny
Deny from all
</Files>

RewriteRule ^Articles/?(.*)$ http://example.com/articles/$1 [R=301,L]
RewriteRule ^Events/?(.*)$ http://example.com/events/$1 [R=301,L]
RewriteRule ^News/?(.*)$ http://example.com/articles/$1 [R=301,L]
RewriteRule ^Newsletters/?(.*)$ http://example.com/newsletters/$1 [R=301,L]
2:56 am on Nov 25, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13681
votes: 446


Options +SymLinksIfOwnerMatch
<snip>
Options ExecCGI Includes IncludesNOEXEC SymLinksIfOwnerMatch -Indexes

Stop right there.

Edit:
Oh, all right, I continued looking.
RewriteRule ^News/?(.*)$ http://example.com/articles/$1 [R=301,L]
RewriteRule ^Newsletters/?(.*)$ http://example.com/newsletters/$1 [R=301,L]

As cosgan dot de would say:
/images/smilie/froehlich/a065.gif

Further edit:
I understand the intention, but it can't be done that way. The directory slash is only optional if there is nothing after it. So the pattern is
blahblah(/.*)?$

without capture, or
blahblah(?:/(.*))?$

with capture. This only works in RegEx engines that don't object to potentially undefined back-references. mod_rewrite is fine with it. Using ?: in the outer group lets you stick with $1 in the target; otherwise it would become $2.
3:20 am on Nov 25, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


If I change the line

RewriteRule ^Newsletters/?(.*)$ http://example.com/newsletters/$1 [R=301,L]


to

RewriteRule ^Newsletters(/.*)?$ http://example.com/newsletters/$1 [R=301,L]


I still get the error.

I just saw your addition. I will try that.
3:25 am on Nov 25, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


Nope, still doesn't work with:

RewriteRule ^Newsletters(?:/(.*))?$ http://example.com/newsletters/$1 [R=301,L]
3:28 am on Nov 25, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


Interestingly, the others work. Only the that doesn't is for the newsletters directory. Would it matter if it directed to PDF files to the directory? (Just trying to make some sense.)
4:38 am on Nov 25, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13681
votes: 446


The problem is not in Newsletters. The problem is in the immediately preceding line, News.
1:51 pm on Nov 25, 2013 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Apr 30, 2008
posts:2630
votes: 191


The sub-directory "letters" does not exist, nor is it listed in the .htaccess file,

Remove the word "News" from the word "Newsletters" and you get "letters".

The problem is not in Newsletters. The problem is in the immediately preceding line, News.

Exactly!

@LeeU, swap the order of rules to list your Newsletters redirect before your News redirect.
2:26 pm on Nov 25, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


That's it! Thanks, @aakk9999! It works just fine now.

For my own information, why did it change it the way it did, in the order I had in the original listing?
4:00 pm on Nov 25, 2013 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Apr 30, 2008
posts:2630
votes: 191


Because you need to put rules in the order of more specific to less specific. Your URL with "Newsletters" was executing "News" rule, because "News" is subset of "Newsletters"

^News/?(.*)$ says "Anything starting with "News" or "News/"

The (.*) after the News in URLs with "Newsletters" then captured the pattern "letters" and this is waht got inserted after the /articles/ (referred to by $1).

When you swapped the lines around, URLs with "Newsletters" matched the ^Newsletters rule first and this one got executed.
5:32 pm on Nov 25, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


Ah, that makes sense now. Thanks again.

(Now, how do I marked this "solved"?)
1:35 am on Nov 26, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13681
votes: 446


No such button :)

But that was the point of fine-tuning to
(?:/(.*))?$
With this configuration, the rules will work regardless of order, even if one pattern name is contained within another. It also eliminates a problem in the original form, where a filename might be captured with or without leading directory slash.
2:57 am on Nov 26, 2013 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Apr 30, 2008
posts:2630
votes: 191


So changing the News rule would allow any order of rules and it is probably a safer solution to use:

RewriteRule ^News(?:/(.*))?$ http://example.com/articles/$1 [R=301,L]

Thank you Lucy for explaining :)
3:16 pm on Nov 26, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


Ah, now I get it. Yes, thanks for the explanation. It makes sense now. And it works just fine (someday I'll learn regular expression properly).

Thanks to both of you!
2:06 am on Nov 28, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


You're not quite done. There's a number of other issues to fix up. These include:
- rewriterules that block access or prevent hot-linking should be listed first
- jpg|jpeg| simplifies to jpe?g|
- [F,L] flag should be [F]
- redirects should be listed before rewrites
- the www to non-www redirect should be the last of the redirects
- the RegEx pattern ^www.example.com$ [NC] should be just !^(example\.com)?$
- in any rewrite with rule target beginning $1, the rule target must be prefixed with a slash
- missing escaping in multiple RegEx patterns.

You might also usefully list the "deny from" rules at the very top of the file.
10:34 pm on Dec 2, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


Thanks, g1smd. I cleaned it up with those tips. It seems there are many articles about the .htaccess file but I have never seen one that explains the order things should be done (although it always seemed like there should be one).
5:23 pm on Dec 3, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


Okay, well after doing a site search on Google, I have found I have problems with another domain ("example.com") and one of its sub-domains ("example2.com"). I have listed below excerpts from the .htaccess files (you can see that I have attempted to fix this problem before but, evidently, not very effectively).

One problem I can see is the rewrite to the "blosxom.cgi" file. I need to differentiate between example.com and example2.com domain/sub-domain.

I would appreciate some more help, if possible.

MAIN DOMAIN - excerpted portion


RewriteEngine on

Redirect 301 /emet/weekly.xml http://example2.com/
Redirect 301 /rss-info.htm http://example2.com/rss-info.php
Redirect 301 /articles//going/feedback.php http://www.example.com/going/feedback.php
Redirect 301 /cgi-bin/2me_4see_comment.cgi http://www.example.com/contact.php
Redirect 301 /emet/analysis/arab_claim_to_palestine.htm http://example2.com/analysis/arab_claim_to_palestine.php
Redirect 301 /articles/important-jeru.php http://www.example.com/articles/what-is-so-important-about-jerusalem.php
Redirect 301 /articles/important-jeru.htm http://www.example.com/articles/what-is-so-important-about-jerusalem.php
Redirect 301 /articles/histjeru.php http://www.example.com/articles/a-biblical-history-of-jerusalem.php
Redirect 301 /articles/histjeru.htm http://www.example.com/articles/a-biblical-history-of-jerusalem.php
Redirect 301 /articles/jerucov.php http://www.example.com/articles/jerusalem-a-separate-covenant.php
Redirect 301 /articles/jerucov.htm http://www.example.com/articles/jerusalem-a-separate-covenant.php
Redirect 301 /articles/seventhmonth.php http://www.example.com/articles/index.php
Redirect 301 /articles/touch_not_mine_anointed.php http://www.example.com/articles/index.php
Redirect 301 /articles/camel-needle.php http://www.example.com/articles/index.php
Redirect 301 /articles/camel-needle.htm http://www.example.com/articles/index.php
Redirect 301 /bookReviews/review-paul_the_jewish_theologian.php http://www.example.com/books.php
Redirect 301 /articles/spiritual-authority-what-it-is-and-what-it-isnt.php http://www.example.com/articles/index.php
Redirect 301 /articles/godsplan.php http://www.example.com/articles/index.php
Redirect 301 /articles/blessings.php http://www.example.com/articles/index.php
Redirect 301 /articles/fathers_love.php http://www.example.com/articles/index.php
# Redirect 301 /book_reviews/ http://www.example.com/books.php
Redirect 301 /articles/harry_potter.php http://www.example.com/articles/index.php
Redirect 301 /articles/tradmen.php http://www.example.com/articles/index.php
Redirect 301 /articles/pastors.php http://www.example.com/articles/index.php
Redirect 301 /articles/covering.php http://www.example.com/articles/index.php
Redirect 301 /articles/bride_ready.php http://www.example.com/articles/index.php

RewriteRule ^weblog/?(.*)$ /cgi-bin/blosxom.cgi/$1

RewriteRule ^(.+)\.htm$ http://www.example.com/$1.php [R=301,L]

RewriteCond %{HTTP_HOST} ^example\.org [OR]
RewriteCond %{HTTP_HOST} ^69\.89\.23\.241
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]

RewriteRule ^views/?(.*)$ http://example2.com/views/$1 [R=301,L]
RewriteRule ^analysis/?(.*)$ http://example2.com/analysis/$1 [R=301,L]
RewriteRule ^documents/?(.*)$ http://example2.com/documents/$1 [R=301,L]
RewriteRule ^palestinian-myths/?(.*)$ http://example2.com/palestinian-myths/$1 [R=301,L]
RewriteRule ^media/?(.*)$ http://example2.com/media/$1 [R=301,L]


==========================

SUB-DOMAIN - excerpted portion


RewriteEngine On

Redirect 301 /articles/ http://www.example.org/articles/histjeru.php
Redirect 301 /analysis/gaza_and_katrina.php http://example2.com/analysis/gaza-and-katrina.php
Redirect 301 /analysis/palestine-history.pdf http://example2.com/analysis/palestine_history.pdf
Redirect 301 /documents/unsc_resolution_338.php http://example2.com/documents/unsc-resolution-338.php
Redirect 301 /analysis/the_myth_of_occupied_territories.php http://example2.com/analysis/the-myth-of-occupied-territories.php
Redirect 301 /analysis/palestine_history.php http://example2.com/analysis/palestine-history.php
Redirect 301 /gaza-assistance.php http://example2.com/gaza-commentaries-booklets-pictures.php
Redirect 301 /analysis/the_israeli_humanitarian_lifeline_to_gaza.php http://example2.com/analysis/the-israeli-humanitarian-lifeline-to-gaza.php
Redirect 301 /analysis/jewish_blood_who_is_really_behind_the_gaza_flotilla.php http://example2.com/analysis/jewish-blood-who-is-really-behind-the-gaza-flotilla.php
Redirect 301 /analysis/is_israel_bound_by_international_law_to_supply_utilities_goods_and_services_to_gaza.php http://example2.com/analysis/is-israel-bound-by-international-law-to-supply-utilities-goods-and-services-to-gaza.php
Redirect 301 /contact.php http://example2.com/contact/contact.php
Redirect 301 /Blog.cgi/index.rss http://example2.com/cgi-bin/blosxom.cgi/index.rss
Redirect 301 /blog.cgi/index.rss http://example2.com/cgi-bin/blosxom.cgi/index.rss
Redirect 301 /weblog/why-must-an-arab-palestinian-state-be-jew-free.html) http://www.example2.com/weblog/why-must-an-arab-palestinian-state-be-jew-free.html
Redirect 301 /current.htm http://example2.com/
Redirect 301 /current.php http://example2.com/
Redirect 301 /weekly.xml http://example2.com/

RewriteRule ^weblog/?(.*)$ /cgi-bin/blosxom.cgi/$1

RewriteCond %{HTTP_HOST} ^www.example2.com$ [NC]
RewriteRule ^(.*)$ http://example2.com/$1 [R=301,L]

RewriteCond %{HTTP_HOST} example\.org$
RewriteRule ^(.*)$ "http\:\/\/example2\.org\/" [R=301,L]

RewriteRule ^(.+)\.htm$ http://example2.com/$1.php [R=301,L]

RedirectMatch 301 ^/palestinian_myths/$ http://example2.com/palestinian-myths/

RewriteRule ^archives/?(.*)$ http://example2.com/cgi-bin/mail.cgi/archive/emet
RewriteRule ^palestinian_myths/(.*)$ http://example2.com/palestinian-myths/$1 [R=301,L]


Thanks in advance for any help you can give.
7:30 pm on Dec 3, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13681
votes: 446


GET RID OF all rules involving mod_alias (Redirect by that name). You may be able to combine mod_alias and mod_rewrite in the config file, but don't try it in htaccess.

Open a copy of your htaccess file in a text editor that does Regular Expressions and run the following global replaces:

# change . to \.
^(Redirect \d\d\d \S+?[^\\])\.
TO
$1\\.
# now change Redirect to Rewrite
^Redirect(?:Match)? 301 /(.+)
TO
RewriteRule $1 [R=301,L]
# and (if necessary)
^Redirect(?:Match)? 410 /(.+)
TO
RewriteRule $1 - [G]

Also get rid of all escaped : colons and / directory slashes AND all quotation marks in targets.
10:07 pm on Dec 3, 2013 (gmt 0)

New User

10+ Year Member

joined:Jan 4, 2005
posts: 15
votes: 0


Thanks! I'll give it a try and get back to you.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members