.htaccess / Redirect exclusions - where am I going wrong?

[Mikroz]

Hi all,

I've been battling my .htaccess for 3 days now and am still stuck.

Here's hoping one of you can assist...

# Engine
RewriteEngine on
Options +FollowSymLinks
RewriteBase /

# Hotlinking
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{REQUEST_FILENAME} -f
RewriteCond %{REQUEST_FILENAME} \.(gif|jpe?g?|png|bmp|ico|swf)$ [NC]
RewriteCond %{HTTP_REFERER} !^https?://([^.]+\.)?example\. [NC] [OR]
RewriteCond %{HTTP_REFERER} !^https?://([^.]+\.)?example2\. [NC]
RewriteRule \.(gif|jpe?g?|png|bmp|ico|swf)$ - [F,NC,L]

########## Remove duplicate trailing slashes if exist
##########RewriteRule (.*)/$ $1 [L,R=301]
# Add trailing slash
##########RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule .*[^/]$ /$0/ [L,R=301]

#HTTPS to HTTP
#RewriteCond %{HTTPS} on [OR]
#RewriteCond %{SERVER_PORT} 443
#RewriteRule ^/?$ http://%{SERVER_NAME}/ [R=301,L]

# No 'www'
#RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
#RewriteRule ^(.*)$ http://%1/$1 [R=301,L]

# Root redirect exclusions
# Domain exclusions
##########RewriteCond %{REQUEST_URI} !^/(example2).*$ [OR]
##########RewriteCond %{HTTP_HOST} !^(www\.)?(example2)\.?(co\.za)
##########RewriteCond %{HTTP_HOST} ^(www\.)?example2.co.za$
RewriteRule .* - [S=3]

# Page exclusions
RewriteCond $1 !^samplepage\.php$
RewriteCond $1 !^robots\.txt$

# Root redirect
RewriteRule (.*) http://mrally.example.com/$1 [R=301,L]

OK, I realize that's a bit much to swallow in one go, so here's a breakdown of the problem...

I have two subdomains ('blog' and 'mrally'), one addon domain (example2.co.za) and some specific files in the root.

When someone types in any one of the following:

example.com/<addon or subdomain folder>
example.com/<addon or subdomain folder>/
<addon>.example.com
<addon>.example.com/
example2.co.za

...they need to go to either the applicable domain (example2.co.za) or be redirected to the right subdomain.

If I type in example.com/<specific excluded page> then I obviously need that page.

Lastly, anything not conforming to this gets redirected to mrally.example.com

I'm not sure if my code is the most efficient or not but I am mostly there.

The only pain at the moment is the addon domain. It keeps redirecting to the mrally subdomain.

Where I marked lines with ######### above is what I have tried to cure my .htaccess woes with.

Lastly, I have discovered that if I enter <addon>/<page> then it loads, so it's only the root that is failing.

What am I missing?

Thanks, Nic

PS: My apologies for the rather lengthy post. :)

[edited by: jdMorgan at 3:12 pm (utc) on May 31, 2010]
[edit reason] example.com [/edit]

[Mikroz]

OK, going through the archive here I've edited a few things above based on my newfound knowledge...

I've changed the domain exclusion to this:

RewriteCond %{HTTP_HOST} !example2 [NC]

Then I put this into the example2 folder:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

# Subdomain folder redirects
RewriteCond %{HTTP_HOST} !^example2\.example\.com
RewriteRule (.*) http://example2.co.za/$1 [R=301,L]

</IfModule>

Now...

example.com/example2
example.com/example2/
example2.co.za

...all redirect to example2.example.com

#*$!?

[edited by: jdMorgan at 3:14 pm (utc) on May 31, 2010]
[edit reason] example.com [/edit]

[Mikroz]

3 days and I finally sorted it. @#$%!

The thing is though...I have absolutely nFi what the problem or the solution was!? *lol*

Here are the code snippets that now I have in place:

ROOT

# Root redirect exclusions
# Domain exclusions
RewriteCond %{HTTP_HOST} ^(www\.)?example2.co.za
RewriteRule .* - [S=3]
# Page exclusions
RewriteCond $1 !^sample\.php$
RewriteCond $1 !^robots\.txt$
# Root redirect
RewriteRule (.*) http://mrally.example.com/$1 [R=301,L]

example2 subdomain folder

# Engine
RewriteEngine on
RewriteBase /

RewriteCond %{HTTP_HOST} !www\.example2\.co\.za
RewriteRule (.*) "http\:\/\/www\.example2\.co\.za\/" [R=301,L]

1. Exclude the addon domain 'example2' from the three following rules.
2. Exclude the sample.php and robots.txt files from the redirect.
3. Redirect all other traffic to (www.)example.com to the 'mrally' subdomain.
4. Redirect visitors to the 'example2' subdomain and folder to www.example2.co.za

*pats self on back* :D

[edited by: jdMorgan at 3:14 pm (utc) on May 31, 2010]
[edit reason] example.com [/edit]

[jdMorgan]

What three rules are you skipping with that first [S=3] rule? Be very sure that there are three following rules, and that you want to skip all of them if the requested hostname is "example2.co.za" or "www.example2.com.za"... Otherwise, correct the skip count or delete the first rule and move the exclusion into the individual rules using a negative-match RewriteCond.

Further tweaks and clean-ups you might consider...

Root .htaccess:

# Root redirect exclusions
# Skip next three rules for "example2.co.za" and "www.example.2.co.za" requests
RewriteCond %{HTTP_HOST} ^(www\.)?example2.co.za
RewriteRule ^ - [S=3]
#
# Exclude sample.php and robots.txt from redirect to mrally.example.com
RewriteCond $1 !^(sample\.php|robots.txt)$
# Externally redirect all other requests for hostnames other than "example2.co.za"
# and "www.example2.co.za" to "mrally.example.com"
RewriteRule ^(.*)$ http://mrally.example.com/$1 [R=301,L]

example2 subdomain .htaccess:

# Enable mod_rewrite rewriting engine
RewriteEngine on
#
# Set RewriteBase back to default from previously-set
# non-default setting ([b]is this really required?[/b])
RewriteBase /
#
# Redirect all non-blank non-canonical hostname requests to canonical hostname
RewriteCond %{HTTP_HOST} !^(www\.example2\.co\.za)?$
RewriteRule ^(.*)$ http://www.example2.co.za/ [R=301,L]

Jim

[Mikroz]

Hi Jim,

Many thanks for your reply.

I found that my original redirect was causing the addon domain to redirect too.

I then did some Googling and came across the skip command.

I added the addon domain as a condition (excuse my terminology) and instructed it to skip the next three commands - skip page1, page2 and the general redirect.

Am I right in saying I now need to skip only two lines?

RewriteCond $1 !^(sample\.php|robots.txt)$
RewriteRule ^(.*)$ http://mrally.example.com/$1 [R=301,L]

I have used your updated code (thank you very much) and it is working. I will change the skip to 2.

I'm rather new to this stuff but I'm picking it up fairly fast. :D I'll have to read up a bit more to understand your tweaks.

Cheers, Nic

[edited by: jdMorgan at 5:04 pm (utc) on May 31, 2010]
[edit reason] de-linked [/edit]

[jdMorgan]

The skip count does not apply to "lines," it applies to RewriteRules -- no matter how many "lines" each of those rules comprises.

If there is only one rewrieRule that needs to be skipped, then my above comment about replacing the skip rule with a negative-match RewriteCond is applicable:

# Root redirect exclusions
# Skip next three rules for "example2.co.za" and "www.example.2.co.za" requests
RewriteCond %{HTTP_HOST} ^(www\.)?example2\.co\.za
[b]RewriteRule ^ - [S=1][/b]
#
# Exclude sample.php and robots.txt from redirect to mrally.example.com
RewriteCond $1 !^(sample\.php|robots\.txt)$
# Externally redirect all other requests for hostnames other than "example2.co.za"
# and "www.example2.co.za" to "mrally.example.com"
RewriteRule ^(.*)$ http://mrally.example.com/$1 [R=301,L]

would be more-efficiently replaced by

# Exclude hostnames "example2.co.za" and "www.example2.co.za" and blank
RewriteCond %{HTTP_HOST} !^((www\.)?example2\.co\.za\.?(:[0-9]+)?)?$
# Exclude sample.php and robots.txt
RewriteCond $1 !^(sample\.php|robots\.txt)$
# Externally redirect all other requests to "mrally.example.com"
RewriteRule ^(.*)$ http://mrally.example.com/$1 [R=301,L]

It's also not clear why you're excluding robots.txt from this redirect; Each domain and/or subdomain can and should have its own robots.txt and sitemap.xml files... located in whatever directory constitutes its 'Web root' directory.

Jim

[Mikroz]

Hi Jim,

Thanks for the explanation. Consider me educated. :)

That's very neat code indeed.

It's got me inspired to see how else I can upgrade my file.

I wasn't aware that each subdomain and addon used their own robots.txt file - I will make a point of creating new ones asap.

I haven't really worked with sitemaps as yet and will do some Googling.

The robots.txt exclusion above is for the root.

I will spec' new ones for the 'mrally' website and other 'sites.

Cheers, Nic

[jdMorgan]

I'd encourage you to "go to the source" for this stuff, instead of doing searches and following the results to 'general' Web sites or forums.

Basically, the Web is full of information, but a lot of it is wrong -- in its details or in whole. As contradictory as it might sound given the context of this post, I would suggest that you *not* base your server configuration files (e.g. .htaccess files) on code found in forums. :)

Instead, look for the Apache mod_rewrite documentation at apache.org, and look to the "Standard for Robot Exclusion" document for info on robots.txt. The regular-expressions tutorial cited in our Apache Forum Charter is useful for regex clarification.

All of these documents should be read very carefully. They mean exactly what they say, and they do not mean what they do not say. Like those in many answers posted here, the words and terms in these documents are chosen very deliberately. Many times, you'll find that terms commonly used in an apparently-interchangeable way do not in fact mean the same thing, so details matter. (Examples we see here all the time are confusion between URLs and filepaths, between client redirects and server-internal rewrites, between client-side functions and server-side functions, etc.)

Jim

[Mikroz]

Crikey. My head's already overloading just from your description. Haha.

Thanks Jim. Duly noted.

I've been looking at the Apache 'site the last few days and it's pretty intense to say the least.

One day at a time. ;)

Cheers, Nic