Forum Moderators: phranque

Message Too Old, No Replies

rewriting/forwarding wildcard subdomains to same subdomain on diff dom

htaccess vs httpd.conf vs DNS?

         

Bike

11:44 am on Sep 6, 2007 (gmt 0)

10+ Year Member



Hi there, after years of lurking and finding answers here, I finally have a problem that I could not find an answer to, so I hope you can help.

I have set up the multiuser version of Wordpress. It offers visitors to set up a blog with url: username.domain1.com
This is being set-up using a virtual host addition (wildcards). An non existing address is being captured by the altered .htaccess file, redirecting it to a signup page:


RewriteEngine On
RewriteBase BASE/

#uploaded files
RewriteRule ^(.*/)?files/$ index.php [L]
RewriteRule ^(.*/)?files/(.*) wp-content/blogs.php?file=$2 [L]

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule . - [L]
RewriteRule ^([_0-9a-zA-Z-]+/)?(wp-.*) $2 [L]
RewriteRule ^([_0-9a-zA-Z-]+/)?(.*\.php)$ $2 [L]
RewriteRule . index.php [L]

Sor far so good, it works.

As my domain name is spelled differently in UK vs US english (****travellers, with 1 or 2 'l's) I have all related domains and alternative TLD's. Currently these are parked on top of domain.com using Cpanel. I am on a VPS and can change DNS, httpd.conf etc.

My questions is:
- how can I use a rewrite rule so that user1.domain1.com rewrites to user1.domain2.com or (or user1.domain1.net etc). So the wildcards subdomains (usernames) must resolve for every domain and rewrite to the same subdomain on domain2.com/domain1.net.

Subquestions:
- Do I need to set up a <virtualhost> for every domain in httpd.conf?
Or is it maybe possible to add serveraliases to one virtualhost?

- What should be added/changed to the above .htaccess?
- do I need to change anything to the DNS zones, or was parking enough?
- How/where can I best add a non-www rewrite, taking the above into consideration, while preventing any loops?

There are so many different combinations possible, so all help is welcome!

Thanks in advance!

[edited by: jdMorgan at 3:11 pm (utc) on Sep. 6, 2007]
[edit reason] Obscured domain name per TOS. [/edit]

jdMorgan

3:36 pm on Sep 6, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



My questions is:
- how can I use a rewrite rule so that user1.domain1.com rewrites to user1.domain2.com or (or user1.domain1.net etc). So the wildcards subdomains (usernames) must resolve for every domain and rewrite to the same subdomain on domain2.com/domain1.net.

Subquestions:
- Do I need to set up a <virtualhost> for every domain in httpd.conf?
Or is it maybe possible to add serveraliases to one virtualhost?

You can do it either way. It's probably be easiest to define the main domain, and then define all the alternate domains as ServerAliases, using a wild-card for the subdomains.

- What should be added/changed to the above .htaccess?
- do I need to change anything to the DNS zones, or was parking enough?
- How/where can I best add a non-www rewrite, taking the above into consideration, while preventing any loops?

It would be simplest to dump the "parking" complication/silliness, and set up DNS A-records to point all of your domains --again, with wild-card subdomains as needed-- to the IP address of your server. I hold "parking" in disdain because it is over-hyped, often "mis-implemeted" (causing ranking problems), and it is a non-technical, non-specific term which has no specific meaning. In most cases, it causes more trouble than it is worth, and is useful mostly for domain speculators who don't want to pay to host all of the domains in their large collections.

Before addressing any coding details, we need to establish the requirements better: You've used the word 'rewrite' many times, but I suspect you mean 'redirect' in many/most cases. In another case, you used 'redirect' when I think you meant 'rewrite'.

So the simplest question to ask is, "If a user types in user1.domain1.com, and you 'modify' that to user1.domain2.com, which URL do you want the user to see in his/her browser address bar?

It might also prove useful to very briefly explain why you have multiple domains, what your goal was in acquiring them, and what you wish to accomplish by merging them all into 'domain2.com'. This with an eye to getting a good solution to the right problem, and not a good solution to the wrong problem.

Of particular concern is this part of this requirement:

- how can I use a rewrite rule so that user1.domain1.com rewrites to user1.domain2.com or (or user1.domain1.net etc). So the wildcards subdomains (usernames) must resolve for every domain and rewrite to the same subdomain on domain2.com/domain1.net.

How do you decide which to 'rewrite' to, domain2.com or domain1.net? ...Or perhaps I mis-understood the meaning and intent of the "domain2.com/domain1.net" construct in that sentence.

Jim

Bike

4:13 pm on Sep 6, 2007 (gmt 0)

10+ Year Member



Hello Jim, thanks for your quick reply, much appreciated.

I can see your note about the parking, frankly I do not know exactly what it does, so it is difficult to change where needed. If I "unpark" and then manually add them back to the <virtual host>, should I then add an entry to the main domain's DNS records (CNAME?) or add new DNS records for all domains?


So the simplest question to ask is, "If a user types in user1.domain1.com, and you 'modify' that to user1.domain2.com, which URL do you want the user to see in his/her browser address bar?

It might also prove useful to very briefly explain why you have multiple domains, what your goal was in acquiring them, and what you wish to accomplish by merging them all into 'domain2.com'. This with an eye to getting a good solution to the right problem, and not a good solution to the wrong problem.

Sorry for the mix between rewrite/redirect. Let me explain the multiple domains first. Basically it is caused by different spelling: ****traveller is English, ****traveler is US English. To avoid losing visitors, I want both to work, though I will stick to one spelling.

As everybody can sign up for a blog (=subdomain) I do not want blog1.domain1.com to co-exist with blog1.domain2.com, that will be confusing and will cause duplicate content I think.
(I have .org as it actually might be an Org soon as it is non-commercial, to promote ecological travel.)

Actually I have not decided yet if everything should go to ****travellers or to ****travelers, but that is something I can switch back later. For now, to answer your first question:
"If a user types in user1.domain1.com, and you 'modify' that to user1.domain2.com, which URL do you want the user to see in his/her browser address bar?"

I want him/her to see user1.domain2.com as that will keep things consistent and will prevent duplicate content (correct me if I am wrong here, please)

Of particular concern is this part of this requirement:
- how can I use a rewrite rule so that user1.domain1.com rewrites to user1.domain2.com or (or user1.domain1.net etc). So the wildcards subdomains (usernames) must resolve for every domain and rewrite to the same subdomain on domain2.com/domain1.net.

How do you decide which to 'rewrite' to, domain2.com or domain1.net? ...Or perhaps I mis-understood the meaning and intent of the "domain2.com/domain1.net" construct in that sentence.


Sorry, non-native english-speaking sentence constructions are showing :)
Let me try to rephrase that, basically it means the same as mentioned before: how can I make sure that *.domain1.com or *.domain1.net redirect to *.domain2.com, where * can be anything, but must stay the same when applied to *.domain2.com.

Thanks in advance for your time

jdMorgan

5:42 pm on Sep 6, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I can see your note about the parking, frankly I do not know exactly what it does, so it is difficult to change where needed. If I "unpark" and then manually add them back to the <virtual host>, should I then add an entry to the main domain's DNS records (CNAME?) or add new DNS records for all domains?

I don't know what it does on your server either, since different parking services and different control panels do it differently. Again, it has no precise technical meaning.

Use DNS A records, and point them straight to the IP address of your server.

I want him/her to see user1.domain2.com as that will keep things consistent and will prevent duplicate content (correct me if I am wrong here, please)

OK, so that's an external 301 redirect then -- which updates the browser address bar, and tells the search engines to use the new address only.

In order to implement the many and somewhat-overlapping requirements for the redirect, you'll need to use mod_rewrite RewriteConds -- mod_alias isn't sufficient for this job.

Something like this in httpd.conf, then::


Options +FollowSymLinks
RewriteEngine on
#
# IF requested hostname does not end in "domain2.com" or "domain2.com:<port_number>"
RewriteCond %{HTTP_HOST} !\.domain2\.com(:[0-9]+)?$
# AND [b]([/b]
# IF requested hostname starts with <subdomain>.domain1.<anything> or www.<subdomain>.domain1.<anything>
RewriteCond %{HTTP_HOST} ^(www\.)?([^.]+)\.domain1\. [OR]
# OR IF requested hostname starts with <subdomain>.domain2.<anything> or www.<subdomain>.domain2.<anything>
RewriteCond %{HTTP_HOST} ^(www\.)?([^.]+)\.domain2\. [OR]
# OR IF requested hostname starts with <subdomain>.domain3.<anything> or www.<subdomain>.domain3.<anything> [b])[/b]
RewriteCond %{HTTP_HOST} ^(www\.)?([^.]+)\.domain3\.
# THEN redirect the request to <subdomain>.domain2.com/<requested_page>
RewriteRule ^/(.*) http://%2.domain2.com/$1 [R=301,L]

Note that $1 refers to the requested local URL-path matched by the RewriteRule, while %2 refers to the subdomain matched in the last-matched RewriteCond -- The numbers in $1, %1, %2, etc. correspond to the ordinal number of each of the pairs of parentheses in each pattern.

As for "travellers" versus "travelers", pick the spelling that the majority of your current visitors --or the majority of your potential future visitors-- will be using. If most of your market is in the UK, then use the UK spelling conventions.

Jim

and1c

11:31 pm on Sep 6, 2007 (gmt 0)

10+ Year Member



As usual, Jims reply is spot on!

I'm curious though, if you use a permanent 301 redirect from travelers.domain to travellers.domain (or vice versa depending which is your majority market) you will save yourself a lot of hassle and unnecessary complexity....

And don't under estimate your users, if they are finding you from the search engines the spelling shouldnt be a big issue provided you cover both spellings as keywords and in your content and SEO provisions...... and if it is an international domain (for instance US customers wouldnt pick a .co.uk domain in many cases unless it was the only place for what they wanted)...

If it is from type in traffic, it will of course be an issue but if you own both domains a permanent redirect will fix it and not cause any google penalties..

Bike

1:54 am on Sep 7, 2007 (gmt 0)

10+ Year Member



Hi And1c, thanks for commenting. I am not quite sure what you mean with adding extra complexity, is there an easier way to include the wildcard subdomains in a 301 forward?

Hi Jim, thanks again for the quick reply.

Thanks for the example code, I have added it inside my <virtual host> of the main (target) domain. (=domain2.com), have

restarted apache and cleared my browser cache.
I have unparked everything and have added DNS zones for domain1.com
& domain2.org.

The DNS is not working as I want (subdomains do not work for any other domains besides main), but as the "www" subdomains seem to work, I think I could test your code for the redirect already:

- (www.)domain2.com is still working as normal (which is good :)
- www.domain1.com is now correctly resolving to http://domain2.com and
- www.domain2.org is still resolving to http://domain2.com/wp-signup.php?new=domain2

However
- "domain1.com" (no "www" or other subdomain) is still resolving to http://domain2.com/wp-signup.php?new=domain1
and
- "domain2.org" (no "www" or other subdomain) is also still resolving to http://domain2.com/wp-signup.php?new=domain2

It seems to me that:
- In your code there is no RewriteCond for domains without subdomains, therefore it is passed through and caught by the sign-up script?

This is how my <virtual host> looks like (taking out user, admin etc):


<VirtualHost 12.345.67.89>
ServerName domain2.com
ServerAlias domain2.com *.domain2.com
ServerAlias domain2.org *.domain2.org
ServerAlias domain1.com *.domain2.com

Options +FollowSymLinks
RewriteEngine on
# IF requested hostname does not end in "domain2.com" or "domain2.com:<port_number>"
RewriteCond %{HTTP_HOST}!\.domain2\.com(:[0-9]+)?$
# AND (
# IF requested hostname starts with <subdomain>.domain1.<anything> or www.<subdomain>.domain1.<anything>
RewriteCond %{HTTP_HOST} ^(www\.)?([^.]+)\.domain1\.

# OR IF requested hostname starts with <subdomain>.domain2.<anything> or www.<subdomain>.domain2.<anything>
RewriteCond %{HTTP_HOST} ^(www\.)?([^.]+)\.domain2\. [OR]

# THEN redirect the request to <subdomain>.domain2.com/<requested_page>
RewriteRule ^/(.*) http://%2.domain2.com/$1 [R=301,L]

</VirtualHost>

The 2nd part of the problem (actually first part), is likely the DNS or this httpd.conf <virtual host>. How do I get wildcard subdomains also working for domain1.com & domain2.org?
I am using WHM, but maybe this also creates silly things (similar to the Cpanel parking) when editing DNS zones and I should edit DNS files directly to add the

By the way, I assume it is needed to restart BIND after every DNS change, is that correct?

Thanks for your help

[edited by: jdMorgan at 8:29 pm (utc) on Sep. 7, 2007]
[edit reason] De-linked URLs. [/edit]

jdMorgan

2:52 am on Sep 7, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It seems to me that:
- In your code there is no RewriteCond for domains without subdomains, therefore it is passed through and caught by the sign-up script?

Correct, there's no rule for requests without subdomains -- I missed that request, buried as it was among all the subdomain questions... :)

If you want to redirect them all to www.domain2.com, then add this rule below the first:


# Externally redirect non-www domain requests to www.domain2.com
RewriteCond %{HTTP_HOST} ^domain1\. [OR]
RewriteCond %{HTTP_HOST} ^domain2\.
RewriteRule ^/(.*)$ http://www.domain2.com/$1 [R=301,L]

Note that the last RewriteCond above must not have an [OR] on it. (Your modified code also contained this error -- Watch the details, they're critical.)

Also, the previous-discussed ruleset needs one additional RewriteCond to avoid looping:


Options +FollowSymLinks
RewriteEngine on
# IF requested hostname is not exactly "www.domain2.com"
RewriteCond %{HTTP_HOST} !^www\.domain2\.com$
# AND IF requested hostname does not end in "domain2.com" or "domain2.com:<port_number>"
RewriteCond %{HTTP_HOST} !\.domain2\.com(:[0-9]+)?$
# AND (
# IF requested hostname starts with <subdomain>.domain1.<anything> or www.<subdomain>.domain1.<anything>
RewriteCond %{HTTP_HOST} ^(www\.)?([^.]+)\.domain1\. [OR]
# OR IF requested hostname starts with <subdomain>.domain2.<anything> or www.<subdomain>.domain2.<anything> )
RewriteCond %{HTTP_HOST} ^(www\.)?([^.]+)\.domain2\.
# THEN redirect the request to <subdomain>.domain2.com/<requested_page>
RewriteRule ^/(.*)$ http://%2.domain2.com/$1 [R=301,L]

The 2nd part of the problem (actually first part), is likely the DNS or this httpd.conf <virtual host>. How do I get wildcard subdomains also working for domain1.com & domain2.org?

Your third ServerAlias has a copy-and-paste error/typo, so wildcard subdomains are not defined at all for domain1.com

Not sure about BIND, but go ahead and restart it. If you are testing "through the Web" then it may take 24 hours for the DNS changes to propagate. If you're testing locally, flush your machine's DNS cache -- Use "ipconfig /flushdns" from a Windows Cmd prompt, for example.

Also, be careful with those A records: Just forgetting the period at the end of the domain name will cause it not to work.

Jim

Bike

8:18 am on Sep 7, 2007 (gmt 0)

10+ Year Member



thanks again Jim, you and your solutions are very helpful.


If you want to redirect them all to www.domain2.com, then add this rule below the first:

# Externally redirect non-www domain requests to www.domain2.com
RewriteCond %{HTTP_HOST} ^domain1\. [OR]
RewriteCond %{HTTP_HOST} ^domain2\.
RewriteRule ^/(.*)$ http://www.domain2.com/$1 [R=301,L]

Actually I want everything rewritten to the non-www version. I already had some rule for that in .htaccess, but it created a loop. Also when simply taking away "www." in the above RewriteRule, it looped as Firefox was kind enough to point out.

I solved it by simply rewriting it back:


# Externally redirect non-www domain requests to www.domain2.com
RewriteCond %{HTTP_HOST} ^domain1\. [OR]
RewriteCond %{HTTP_HOST} ^domain2\.
RewriteRule ^/(.*)$ http://www.domain2.com/$1
# redirect back to non-www
RewriteCond ^/(.*)$ http://www.domain2.com/$1
RewriteRule ^/(.*)$ http://domain2.com/$1 [R=301,L]

That worked to my surprise :) but I am sure there is a more elegant solution.

[edit] It works only partially: domain1.com/domain2.org (without "www") do resolve to domain2.com as planned. But www.domain1.com/ or www.domain2.com keep their "www" which is not my intention.[/edit]

Note that the last RewriteCond above must not have an [OR] on it. (Your modified code also contained this error -- Watch the details, they're critical.)

Totally correct. Actually I had taken out the wrong [OR]!

The only thing remaining now is getting the wildcard subdomains also working for domain1.com & domain2.org. The typo above was due to replacing with dummy domains, in my files it was ok, but no subdomains resolve except for the main domai2n.com.

Also, be careful with those A records: Just forgetting the period at the end of the domain name will cause it not to work.

I checked named.conf and all zones are in there. I checked the domain2.com.db vs domain1.com.db and domain2.org.db and they are all the same. So only thing I can think of is that the subdomains are defined correctly for domain2.com as a result of the "servername" in httpd.conf?
As far as I know you can only define one servername per <virtualhost> (?), so would I need to copy the entire <virtualhost> with all rules as above per additional domain and just slightly alter it? Or won't that work anyway is there a more sensible way :)

Thanks in advance,
Harry

[edited by: Bike at 8:57 am (utc) on Sep. 7, 2007]

[edited by: jdMorgan at 8:26 pm (utc) on Sep. 7, 2007]
[edit reason] No URLs, please. [/edit]

Bike

9:26 am on Sep 7, 2007 (gmt 0)

10+ Year Member



As I cannot edit the above anymore: when testing I noticed that some links inside the site do not work as expected, (likely because of my added line ;)

http://www.domain2.com/?attachment_id=xx
Works ok, but
http://domain2.com/?attachment_id=xx
Looks ok (url looks fine), but just shows the homepage. (It also does not show a 404 page when xx does not exist.)

By the way, it has nothing to do with the "?attachment etc" page name, the problem is the same for all subpages.

With domain1.com & domain2.org it also does not work, none of the rules catches it and therefore it gets caught by the .htaccess and redirects to the sign-up page (http://domain2.com/wp-signup.php?new=domain1). Is this a result of my line/edit or one of the previous RewriteConds and what should it be to rewrite to non-www with all pages resolving correctly? I assumed a $1 too much in the RewriteCond, but removing it does not make a difference?

[edit] I replaced the last rule with "RedirectMatch 301 (.*) http://domain2.com$1", but it had the same result. When totally deleting my extra line, I still have the above problem, so it has to be a result of one of the other RewriteCond's?[/edit]

Thanks,
Harry

[edited by: Bike at 10:06 am (utc) on Sep. 7, 2007]

[edited by: jdMorgan at 8:27 pm (utc) on Sep. 7, 2007]
[edit reason] examplified & de-linked URL. [/edit]

jdMorgan

8:33 pm on Sep 7, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Actually I want everything rewritten to the non-www version.

That changes the structure of the code, and can't be solved by "writing it back." I'll try to address this soon -- very busy day today, I'm afraid. But it's not terribly complex or daunting, it's just a matter of starting with correct requirements and going back through the code exercise and modifying a few things. It just requires a bit of concentration that is escaping me at the moment with the phone ringing, etc... :)

Jim

jdMorgan

9:39 pm on Sep 7, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You should not define a ServerAlias with the same name as the ServerName, so the first ServeAlias line needs to omit domain2.com. Also, you can combine all ServerAliases on one line.

Having defined these alternate domain names as ServerAliases of the main domain, you should delete (or comment out) any other VirtualHost containers having these ServerAliased names as their ServerName. It goes without saying that if you delete them, you should make a back-up copy of the file and keep it until you are satisfied that the new configuration works properly -- Like a week or more.

To handle the non-www canonical domain requirement needed a complete rewrite, and there was some 'creeping elegance' in there that needed to be removed. So this may work better:


<VirtualHost 123.45.67.89>
ServerName domain2.com
ServerAlias *.domain2.com domain2.org *.domain2.org domain1.com *.domain1.com
#
Options +FollowSymLinks
RewriteEngine on
#
# IF requested subdomain is not "www"
RewriteCond %{HTTP_HOST} !^www\.
# AND
# ( IF requested domain is domain2.org
RewriteCond %{HTTP_HOST} ^([^.]+)\.domain2\.org [OR]
# OR IF requested domain is domain1.com )
RewriteCond %{HTTP_HOST} ^([^.]+)\.domain1\.com
# THEN redirect the request to <subdomain>.domain2.com/<requested_page>
RewriteRule ^/(.*)$ http://%1.domain2.com/$1 [R=301,L]
#
# IF requested subdomain is www (followed by any domain name)
RewriteCond %{HTTP_HOST} ^www\. [OR]
# OR IF requested domain is domain2.org or domain2.org<port number>
RewriteCond %{HTTP_HOST} ^domain2\.org(:[0-9]+)?$ [OR]
# OR IF requested domain is domain1.com or domain1.com<port number>
RewriteCond %{HTTP_HOST} ^domain1\.com(:[0-9]+)?$
# THEN externally redirect to domain2.com/<requested_page>
RewriteRule ^/(.*)$ http://domain2.com/$1 [R=301,L]

Jim

[edited by: jdMorgan at 10:11 pm (utc) on Sep. 7, 2007]

Bike

5:56 pm on Sep 9, 2007 (gmt 0)

10+ Year Member



Hi Jim,

thanks very much for your code. I have implemented and after a few days of DNS updates it does seem to work exactly as planned.

I also found out about why my wildcard subdomains were not working, maybe useful for others to know:
Even though I can change the domain.db files either directly or through the WHM DNS tools, adding A name records were not getting through. Apparently this was because I am not running my own DNS servers, but I am using my host's (as they know better how to run them!).

Just a quick note to support and it was fixed on their servers in 5 minutes. I am pretty sure I should not say their names, just let me state that when looking for a VPS host, response time and quality is extremely important. Once you know this, you will likely end up with my host :)

Thanks again for your time and efforts,
With best regards,

Bike