301 Redirect question

Not efficient. Too many redirects?

[WebWalla]

I have a domain, example.com, with subdomains for different languages, de.example.com, es.example.com, etc.
These language sites are also available as sub-directories: example.com/de/, example.com/es/
In order not avoid duplicate content, I did a 301 redirect:-

RedirectMatch 301 ^/de/(.*)$ http://de.example.com/$1
RedirectMatch 301 ^/es/(.*)$ http://es.example.com/$1

I also did a 301 to redirect requests for example.com to www.example.com. So far, so good.

But now I have bought a better domain name (newdomain.com) for this site and I am doing a 301 of everything to the new domain name - redirecting normal requests, redirecting requests without the "www" and making sure the duplicate content problem with sub-directories and sub-domians is avoided. This is what I have at the moment:-

Options +FollowSymLinks 
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteOptions inherit
RewriteCond %{HTTP_HOST} example.com$ [NC]
RewriteRule ^(.*) http://www.newdomain.com/$1 [R=301,L]
RewriteCond %{HTTP_HOST} ^newdomain\.com 
RewriteRule ^(.*)$ http://www.newdomain.com/$1 [R=permanent,L] 
RedirectMatch 301 ^/es/(.*)$ http://es.newdomain.com/$1
RedirectMatch 301 ^/pt/(.*)$ http://pt.newdomain.com/$1
RedirectMatch 301 ^/de/(.*)$ http://de.newdomain.com/$1
RedirectMatch 301 ^/it/(.*)$ http://it.newdomain.com/$1
RedirectMatch 301 ^/fr/(.*)$ http://fr.newdomain.com/$1
RedirectMatch 301 ^/nl/(.*)$ http://nl.newdomain.com/$1
RedirectMatch 301 ^/jp/(.*)$ http://jp.newdomain.com/$1
</IfModule>

Everything is working, but when I make a request for de.example.com, I get two 301 redirects, the first one to www.newdomain.com/de/ and the second one to de.newdomain.com. This seems to me excessive.

Is there any way of configuring my .htaccess to deal with all situations more efficiently and have just 1 redirect? Is there anything else wrong with my code?

Thanks a lot!

[g1smd]

There will be a way to do this in one step, but first you need to take a look at your usage of both RedirectMatch and RewriteRule within the same file.

Rules are processed in per-module order. So, it may be that all the RedirectMatch stuff is being processed *before* the RewriteRule stuff. Should you ever move to a different server, the order might change to be the reverse. That would cause odd effects. The order of processing is dependent on the order that the modules are loaded into memory in the server.

It therefore follows, that should you happen to use RewriteRule for some of your redirects and/or rewrites, that you should use it for *all* of them.

You also need to optimise the per-module order of processing. Aim it so that stuff affecting only a small number of files (very specific names) is listed first, and more general stuff (like non-www to www) is listed last. This avoids a redirection chain.

Note also that your use of [NC] is sporadic, and you use '301' in one rule and 'permanent' in another. Use the same conventions for all rules to avoid mis-reading your code.

[WebWalla]

Thanks for your feedback. This is my second attempt using only RewriteRule, but it still gives me too many redirects for certain requests.

Options +FollowSymLinks 
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteOptions inherit
RewriteCond %{HTTP_HOST} ^(www\.)?newdomain\.com
RewriteRule ^/es/?(.*) http://es.newdomain.com/$1 [R=301,L]
RewriteRule ^/pt/?(.*) http://pt.newdomain.com/$1 [R=301,L]
RewriteRule ^/de/?(.*) http://de.newdomain.com/$1 [R=301,L]
RewriteRule ^/it/?(.*) http://it.newdomain.com/$1 [R=301,L]
RewriteRule ^/fr/?(.*) http://fr.newdomain.com/$1 [R=301,L]
RewriteRule ^/nl/?(.*) http://nl.newdomain.com/$1 [R=301,L]
RewriteRule ^/jp/?(.*) http://jp.newdomain.com/$1 [R=301,L]
RewriteCond %{HTTP_HOST} example.com$ [NC]
RewriteRule ^(.*) http://www.newdomain.com/$1 [R=301,L]
RewriteCond %{HTTP_HOST} ^newdomain\.com 
RewriteRule ^(.*)$ http://www.newdomain.com/$1 [R=301,L] 
</IfModule>

I'm afraid I don't know where to go from here. Can you give me a clue as to how to do this?

Thanks.

[jdMorgan]

On the first rule, add an [OR] flag to the first RewriteCond, and add a second RewriteCond:

RewriteCond %{HTTP_HOST} ^(www\.)?newdomain\.com [OR]
RewriteCond %{HTTP_HOST} ^(www\.)?example\.com

In this way, requests for more-specific URL-path requests to either domain are handled before less-specific requests, and, for example, requests for "example.com/es/" will be redirected to "es.newdomain" without having to first go through the second-to-last-rule.

Simply take each of your problematic URLs, and "think through" the code asking, "how will this specific URL" be affected by my rules?" The trouble should be easy to find if you do that.

And of course, be sure to completely flush your browser cache after changing any code on the server in order to avoid seeing stale browser-cached responses.

Jim

[edited by: jdMorgan at 2:53 pm (utc) on April 20, 2009]

[jdMorgan]

Note: Your second to last rule has several anchoring and character-escaping problems. Make it look exactly like the last rule except for the start anchor and the hostname in the RewriteCond.

Alternately, simply merge the two rules by adding an [OR] flag to the first RewriteCond and deleting the first RewriteRule directive of the two.

RewriteCond %{HTTP_HOST} example\.com [NC,OR]
RewriteCond %{HTTP_HOST} ^newdomain\.com [NC]
RewriteRule ^(.*)$ http://www.newdomain.com/$1 [R=301,L] 

Jim

[WebWalla]

Jim,

Thanks a lot for your help - followed your advice and it's now working perfectly. I hadn't realised that the [OR] possibility existed.

Good advice also about flushing the cache - I was in fact already doing this after being caught out a couple of times!

[g1smd]

Some lines have [NC] and others do not.

You might want to check out which lines need that adding.

[jdMorgan]

You can also eliminate the <IfModule> container. It is not needed unless you wish this code to fail silently on any server that does not have mod_rewrite installed -- and it's not likely you would want that.

Jim

[WebWalla]

Hmmm, you're planting doubts in my mind now, but I think I've got it right. Final version? ...

Options +FollowSymLinks 
RewriteEngine on
RewriteOptions inherit
RewriteCond %{HTTP_HOST} ^(www\.)?newdomain\.com [OR] 
RewriteCond %{HTTP_HOST} ^(www\.)?example\.com 
RewriteRule ^/es/?(.*) http://es.newdomain.com/$1 [R=301,L]
RewriteRule ^/pt/?(.*) http://pt.newdomain.com/$1 [R=301,L]
RewriteRule ^/de/?(.*) http://de.newdomain.com/$1 [R=301,L]
RewriteRule ^/it/?(.*) http://it.newdomain.com/$1 [R=301,L]
RewriteRule ^/fr/?(.*) http://fr.newdomain.com/$1 [R=301,L]
RewriteRule ^/nl/?(.*) http://nl.newdomain.com/$1 [R=301,L]
RewriteRule ^/jp/?(.*) http://jp.newdomain.com/$1 [R=301,L]
RewriteCond %{HTTP_HOST} example\.com [NC,OR] 
RewriteCond %{HTTP_HOST} ^newdomain\.com [NC] 
RewriteRule ^(.*)$ http://www.newdomain.com/$1 [R=301,L]

[g1smd]

The first two Conditions affect only the single /es/ Rule.

Should they not be duplicated for each of the other following rules?

[jdMorgan]

Since the assumption here is that the URLs handled by the first set of rules are "wrong" and the disposition is to redirect them, then it's probably a good idea to accept casing errors on the "language" subdirectory part of the requested URL-path. So, I'd recommend adding the NC flag to these rules, making the flag value "[NC,R=301,L]".

One more thing: The first two RewriteConds in your code apply only to the first RewriteRule in your code, but I suspect you don't need them at all. You can confirm by testing any/all of the rules past the first one - the one that handles "/es/xyz" URLs.

If you do have a problem with the additional language rules, then you'll either need to replicate the first two RewriteConds for each language rule, or make a "skip" rule at the top to skip over all of the language redirects if the requested hostname is NOT newdomain.com and NOT example.com.

Jim

[edited by: jdMorgan at 9:13 pm (utc) on April 20, 2009]

[WebWalla]

Indeed, I hadn't tested enough. It's now:

RewriteCond %{HTTP_HOST} ^(www\.)?newdomain\.com [OR] 
RewriteCond %{HTTP_HOST} ^(www\.)?example\.com 
RewriteRule ^/es/?(.*) http://es.newdomain.com/$1 [NC,R=301,L]
RewriteCond %{HTTP_HOST} ^(www\.)?newdomain\.com [OR] 
RewriteCond %{HTTP_HOST} ^(www\.)?example\.com 
RewriteRule ^/pt/?(.*) http://pt.newdomain.com/$1 [NC,R=301,L]
...
... etc.

[g1smd]

I think you now see that the complete solution is a lot more complex than first imagined, and that if you tackle each part in turn you can break it down into manageable chunks.

One last thing is to add a blank line after each RewriteRule line in the code, and to add a

# comment

after the blank line, with the comment describing the code that follows the comment.

[jdMorgan]

A skip rule would be more efficient:

Options +FollowSymLinks
RewriteEngine on
RewriteOptions inherit
#
# Externally redirect subfolder requests to subdomains
# Skip next seven rules if NOT newdomain.com and NOT example.com
RewriteCond %{HTTP_HOST} !^(www\.)?newdomain\.com
RewriteCond %{HTTP_HOST} !^(www\.)?example\.com 
RewriteRule . - [S=7]
RewriteRule ^/es/?(.*) http://es.newdomain.com/$1 [R=301,L]
RewriteRule ^/pt/?(.*) http://pt.newdomain.com/$1 [R=301,L]
RewriteRule ^/de/?(.*) http://de.newdomain.com/$1 [R=301,L]
RewriteRule ^/it/?(.*) http://it.newdomain.com/$1 [R=301,L]
RewriteRule ^/fr/?(.*) http://fr.newdomain.com/$1 [R=301,L]
RewriteRule ^/nl/?(.*) http://nl.newdomain.com/$1 [R=301,L]
RewriteRule ^/jp/?(.*) http://jp.newdomain.com/$1 [R=301,L]
#
# Externally redirect requests for non-canonical hostnames to www.newdomain.com
RewriteCond %{HTTP_HOST} example\.com [NC,OR]
RewriteCond %{HTTP_HOST} ^newdomain\.com [NC]
RewriteRule ^(.*)$ http://www.newdomain.com/$1 [R=301,L]

Jim

[WebWalla]

Jim,

I don't know why, but now nothing concerning the subdomains is working. I've tried following the advice in this thread step by step and also simply copying your last example directly, but now only simple redirects work, not the subdomains.
For example, a request for example.com/es now gives:
example.com/es -> www.newdomain.com/es -> newdomain.com/es/ -> www.newdomain.com/es/

This is the current code:

AddHandler application/x-httpd-php .htm .html
# 
Options +FollowSymLinks 
RewriteEngine on 
RewriteOptions inherit 
# 
# Externally redirect subfolder requests to subdomains 
# Skip next seven rules if NOT newdomain.com and NOT example.com 
RewriteCond %{HTTP_HOST} !^(www\.)?newdomain\.com 
RewriteCond %{HTTP_HOST} !^(www\.)?example\.com 
RewriteRule . - [S=7] 
RewriteRule ^/es/?(.*) http://es.newdomain.com/$1 [R=301,L] 
RewriteRule ^/pt/?(.*) http://pt.newdomain.com/$1 [R=301,L] 
RewriteRule ^/de/?(.*) http://de.newdomain.com/$1 [R=301,L] 
RewriteRule ^/it/?(.*) http://it.newdomain.com/$1 [R=301,L] 
RewriteRule ^/fr/?(.*) http://fr.newdomain.com/$1 [R=301,L] 
RewriteRule ^/nl/?(.*) http://nl.newdomain.com/$1 [R=301,L] 
RewriteRule ^/jp/?(.*) http://jp.newdomain.com/$1 [R=301,L] 
# 
# Externally redirect requests for non-canonical hostnames to www.newdomain.com 
RewriteCond %{HTTP_HOST} example\.com [NC,OR] 
RewriteCond %{HTTP_HOST} ^newdomain\.com [NC] 
RewriteRule ^(.*)$ http://www.newdomain.com/$1 [R=301,L]

Note: I'm using a server header check tool to test this, as well as my own browser.

[g1smd]

The problem is that for requests without the trailing slash, the server has to issue a redirect to add the slash back on. When it does that, it uses the entry set for CanonicalName as the hostname for the target URL of that redirect. In this case, it appears that you have set CanonicalName to be just example.com when it should actually be set to be www.example.com instead. That's why the domain name currently reverts back to non-www at step three.

I don't know whether it is best to alter the CanonicalName entry to be www.example.com or to set CanonicalName to OFF. There's arguments either way.

[WebWalla]

I've changed the subdomain RewriteRules a little. This now actually works except for the fact that es.example.com is redirected twice:
es.example.com -> www.newdomain.com/es/ -> es.newdomain.com/

Is this due to the CNAME configuration you mention or is there a way round it in .htaccess?

Requests for example.com/es and example.com/es/ are both redirected correctly to es.newdomain.com/

Current code:

AddHandler application/x-httpd-php .htm .html
#
Options +FollowSymLinks
RewriteEngine on
RewriteOptions inherit
#
# Externally redirect subfolder requests to subdomains
# Skip next seven rules if NOT newdomain.com and NOT example.com
RewriteCond %{HTTP_HOST} !^(www\.)?newdomain\.com
RewriteCond %{HTTP_HOST} !^(www\.)?example\.com
RewriteRule . - [S=7]
RewriteRule ^es(.*) http://es.newdomain.com$1 [R=301,L]
RewriteRule ^pt(.*) http://pt.newdomain.com$1 [R=301,L]
RewriteRule ^de(.*) http://de.newdomain.com$1 [R=301,L]
RewriteRule ^it(.*) http://it.newdomain.com$1 [R=301,L]
RewriteRule ^fr(.*) http://fr.newdomain.com$1 [R=301,L]
RewriteRule ^nl(.*) http://nl.newdomain.com$1 [R=301,L]
RewriteRule ^jp(.*) http://jp.newdomain.com$1 [R=301,L]
#
# Externally redirect requests for non-canonical hostnames to www.newdomain.com
RewriteCond %{HTTP_HOST} example\.com [NC,OR]
RewriteCond %{HTTP_HOST} ^newdomain\.com [NC]
RewriteRule ^(.*)$ http://www.newdomain.com/$1 [R=301,L]

[jdMorgan]

Is this the case? es.example.com -> www.newdomain.com/es/ -> es.newdomain.com/
or is it this? es.example.com/es/ -> www.newdomain.com/es/ -> es.newdomain.com/

I am asking because you have no rule shown here that rewrites cc.example.com to example.com/cc/, although you do have redirects that go the opposite way. Note that in this file es.example.com would be redirected to www.newdomain.com, and the "es" would be lost.

As such, it looks like the problem is outside of this code and outside of this .htaccess file. The only way to fix the problem is to go find that code and fix it there.

Jim

[WebWalla]

It is this, as stated:
es.example.com -> www.newdomain.com/es/ -> es.newdomain.com/

The first redirect is shown also by Brent's Server Header Checker. The second one is shown by other tools.

I believe it is a problem with how subdomains are set up by my host. I've logged a ticket with them.
Thanks for your help with the htacess code.