301 redirect in htaccess question

Will this coding work for my situation?

[grandma genie]

Hi,
I have just learned about the duplicate content issue with Google and have one website with three names. Here is the htaccess code I want to try:

RewriteEngine On
RewriteCond %{HTTP_HOST} ^websiteA.com
RewriteCond %{HTTP_HOST} ^websiteB.com
RewriteCond %{HTTP_HOST} ^www.websiteB.com
RewriteRule (.*) [websiteA.com...] [R=301,L]

I want all the search results from websiteA.com, websiteB.com and www.websiteB.com redirected to www.websiteA.com. Will this htaccess coding work? I haven't tried it yet. I want this to effect all pages for this site, which are considerable.
Thanks for your help.

[jdMorgan]

RewriteEngine on
#
RewriteCond %{HTTP_HOST} ^websiteA\.com [NC,OR]
RewriteCond %{HTTP_HOST} ^(www\.)?websiteB\.com [NC]
RewriteRule ^(.*)$ http://www.websiteA.com/$1 [R=301,L]

would be more robust and slightly more compact.

Jim

[g1smd]

You'll still be left with a Duplicate Content issue if someone decides to link to or request

www.websiteA.com:80

though. Trailing port number is catered for for all other URL versions, except this one.

See also current discussion: [webmasterworld.com...]

[grandma genie]

My web host says this: I don't believe this is considered duplicate traffic by google as it is incredibly commonplace to have www and rootdomain.com have the same content. I would recommend not using the rewrite engine variable in the htaccess.
Does Webmasterworld agree with that accessment? I've lost so much traffic in the last year and can't explain it. I assume it is because no one is finding my site due to Google's indexing practices. My site's key words do much better in the other search engines: Yahoo, Bing, Ask, etc.. Google is the most popular search engine and most of my pages don't show up in Google until page 5 or worse. So the problem seems to be Google. That is why I am concerned about the duplicate content issue.

[g1smd]

Your webhost, like many others, has no clue.

Check the (more than 500) previous WebmasterWorld threads discussing Duplicate Content issues.

[jdMorgan]

You may "rely on the kindness of strangers" -- the search engine programmers to de-duplicate your URLs -- if you like... and IF that de-duplication code is working this month, and IF they get around to processing your URLs (it is after all a back-end processing function, with limited capacity), and IF they pick the "right domain" -- i.e. they pick www or no-www, and you get no choice in the matter.

Carpe diem. An ounce of prevention is worth a pound of cure... :)

To handle the case g1smd mentioned requires an additional RewriteCond:

RewriteEngine on
#
# Externally redirect certain non-canonical hostnames to canonical hostname:
# non-www websiteA hostname
RewriteCond %{HTTP_HOST} ^websiteA\.com [NC,OR]
# FQDN-format www.websiteA.com or with appended port number
RewriteCond %{HTTP_HOST} ^www\.websiteA\.com(\.|\.?:[0-9]+)$ [NC,OR]
# www- or non-www websiteB.com
RewriteCond %{HTTP_HOST} ^(www\.)?websiteB\.com [NC]
RewriteRule ^(.*)$ http://www.websiteA.com/$1 [R=301,L]

Another approach to this, if you have no subdomains other than "www" and do not plan to add more, is:

RewriteEngine on
#
# Externally redirect all non-blank, non-canonical hostname requests to canonical hostname
RewriteCond %{HTTP_HOST} !^(www\.websiteA\.com)?$
RewriteRule ^(.*)$ http://www.websiteA.com/$1 [R=301,L]

Jim

[g1smd]

And the "requesting port number" issue isn't just theoretical. In recent days, in another WebmasterWorld thread, someone reported a nasty bot doing just that.

[jdMorgan]

Yes, and we took some heat as I recall, recommending that that potential problem be addressed... several years ago.

If you don't take steps prevent something, it will happen eventually -- whether by accident or malice.

Jim

[grandma genie]

Thank you for all your advice. I am not a pro when it comes to code, so am too chicken to try to fix this using htaccess. I have a shopping cart on my site with https, so that might require some other things, which I am also too chicken to try. I wish the www was as nice as it used to be 10 years ago. Too many nasties now trying to make a buck at others' expense. If I have questions about strange listings in my server logs, where should that be posted and can I post the listings as is, or do they need editing? Such as:
89.238.192.238 - - [01/Aug/2010:06:32:26 -0400] "CONNECT 64.12.202.22:443 HTTP/1.0" 405 324 "-" "-" Does anyone know what this means?

[jdMorgan]

To preserve the http or https protocol of the request through a domain redirect, just add a rewritecond to check and 'copy' the http/https selection:

RewriteEngine on
#
# Externally redirect certain non-canonical hostnames to canonical hostname, preserving http/https protocol
# non-www websiteA hostname
RewriteCond %{HTTP_HOST} ^websiteA\.com [NC,OR]
# FQDN-format www.websiteA.com or with appended port number
RewriteCond %{HTTP_HOST} ^www\.websiteA\.com(\.|\.?:[0-9]+)$ [NC,OR]
# www- or non-www websiteB.com
RewriteCond %{HTTP_HOST} ^(www\.)?websiteB\.com [NC]
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^(.*)$ http%2://www.websiteA.com/$1 [R=301,L]
#
# -or-
#
RewriteEngine on
#
# Externally redirect all non-blank, non-canonical hostname requests to canonical hostname
RewriteCond %{HTTP_HOST} !^(www\.websiteA\.com)?$
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^(.*)$ http%2://www.websiteA.com/$1 [R=301,L]

Install the code, then test it with a server headers checker like the Live HTTP Headers add-on for Firefox. If it doesn't work, restore your previous .htaccess file from backup and no harm done.

From the HTTP/1.1 specification [w3.org]:

9.9 CONNECT

This specification reserves the method name CONNECT for use with a proxy that can dynamically switch to being a tunnel (e.g. SSL tunneling).

If your server is not configured as a forward proxy, then this kind of request would not be expected. But neither would it be supported, so it would not be a problem -- basically, if your server's not a proxy, this request won't *do* anything.

Jim

[grandma genie]

Hi JD,
I have one more snag before I try this. I just discovered the host I am using, before I moved to a dedicated IP, had the site on a server with this designation: xyz.com/~adminxx/
I thought when I got the dedicated IP the old URL would be removed, but I find it is still there and acting like another copy of my site. Very upsetting. I need to have that redirected too. How do I add that one to the redirect?
Grandma_genie

[jdMorgan]

Add a new rule at the top:

RewriteCond %{HTTP_HOST} ^(www\.)?xyz\.com(\.|\.?:[0-9]+)$ [NC]
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^~adminxx/(.*)$ http%2://www.websiteA.com/$1 [R=301,L]

Jim

[grandma genie]

Hi Jim,
Thank you for all your help. Before I try this, I should explain how my site is set up. There have been quite a few changes since I began 10 years ago. At first the site did not use an SSL certificate. It was all html and used PayPal buttons for checking out. When I moved I had to change the name of the site for business reasons. Then I had a developer add a "real" shopping cart to the site using osCommerce. So, in order for customers to buy anything, they clicked on a link that took them to the osCommerce section of the site. That is when all the xyz.com/~adminxx stuff happened. Then I went to a dedicated IP to get rid of the ~adminxx and used the websiteA.com without the www. So today customers are finding the site with all these various designations and when they want to buy something they click on a link that takes them to the websiteA.com/osc/index.php section. The secure SSL part happens when they click on the checkout button. Then the URL changes to the https with the www.websiteA.com. (I am going to assume the host made things complicated by not being consistent with the canonical and noncanonical designations.) So, that is why there is a www.websiteA.com, websiteA.com, www.websiteB.com, websiteB.com and the xyz.com/~adminxx which is on a secure server (https).

So, the way I have the whole thing set up for htaccess would be (website A being the current business name, website B being the previous business name):

RewriteEngine on
#
# Externally redirect certain non-canonical hostnames to canonical hostname, preserving http/https protocol
# non-www websiteA hostname
RewriteCond %{HTTP_HOST} ^websiteA\.com [NC,OR]
# FQDN-format www.websiteA.com or with appended port number
RewriteCond %{HTTP_HOST} ^www\.websiteA\.com(\.|\.?:[0-9]+)$ [NC,OR]
# www- or non-www websiteB.com
RewriteCond %{HTTP_HOST} ^(www\.)?websiteB\.com [NC]
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^(.*)$ http%2://www.websiteA.com/$1 [R=301,L]

RewriteCond %{HTTP_HOST} ^(www\.)?xyz\.com(\.|\.?:[0-9]+)$ [NC]
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^~adminxx/(.*)$ http%2://www.websiteA.com/$1 [R=301,L]

Is that right?
Grandma_genie

[jdMorgan]

You've said what you've got now, but not what you want.

Also, I specifically said to add the new rule "at the top" and such things are *very* important...

RewriteEngine on
#
# Externally redirect old mod_userdir-format requests to canonical hostname
RewriteCond %{HTTP_HOST} ^(www\.)?xyz\.com(\.|\.?:[0-9]+)$ [NC]
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^~adminxx/(.*)$ http%2://www.websiteA.com/$1 [R=301,L]
#
# Externally redirect certain non-canonical hostnames to canonical hostname, preserving http/https protocol
# non-www websiteA hostname
RewriteCond %{HTTP_HOST} ^websiteA\.com [NC,OR]
# FQDN-format www.websiteA.com or with appended port number
RewriteCond %{HTTP_HOST} ^www\.websiteA\.com(\.|\.?:[0-9]+)$ [NC,OR]
# www- or non-www websiteB.com
RewriteCond %{HTTP_HOST} ^(www\.)?websiteB\.com [NC]
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^(.*)$ http%2://www.websiteA.com/$1 [R=301,L]

This code forces the canonical domain name "www.domainA.com" no matter whether http or https is used. All links on your site should be changed to point to www.domainA.com, leaving the http/https as it is currently.

You may also wish to consider adding further rules to force the use of http *unless* the request is made to the secure cart section of the site, and when accessing that secure cart section, to force the use of https.

Defining the "secure cart section of the site" in terms of the requested URLs is actually the hard part, although one must remember to allow images, css, and JavaScripts shared between secure and non-secure pages to be fetched using either http or https to prevent "mixed secure/insecure content" warnings in the browser...

In this case, it might be as simple as forcing https for all *page* requests in the /osc/ subdirectory.

Add after the now-first rule above:

# Externally redirect to force SSL/https on secure shopping cart pages
RewriteCond %{SERVER_PORT} !=443
RewriteCond $1 !\.(gif|jpe?g|png|ico|css|js)$
RewriteRule ^osc/(.*)$ https://www.domainA.com/osc/$1 [R=301,L]
#
# Externally redirect to force http on all pages except secure shopping cart pages
RewriteCond %{SERVER_PORT} =443
RewriteCond $1 !^osc/
RewriteCond $1 !^(robots\.txt|sitemap\.xml)$
RewriteCond $1 !\.(gif|jpe?g|png|ico|css|js)$
RewriteRule ^(.*)$ http://www.domainA.com/$1 [R=301,L]

The 'lists' of filetypes shared between SSL and non-SSL pages might need to be expanded -- I don't know enough about your site to be sure. You will know immediately if the browser starts throwing "Mixed secure/insecure content" warnings when you surf your secure cart pages while testing.

Jim

[grandma genie]

Hi Jim,
I just tried to add the code to htaccess and this is not working. Can I give you the actual link to look at? _ Jeannie

RewriteEngine on
#
# Externally redirect old mod_userdir-format requests to canonical hostname
RewriteCond %{HTTP_HOST} ^(www\.)?xyz\.com(\.|\.?:[0-9]+)$ [NC]
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^~adminxx/(.*)$ http%2://www.websiteA.com/$1 [R=301,L]

[jdMorgan]

Hmmm... Nasty problem, that...

Try this:

# Externally redirect old mod_userdir-format requests to canonical hostname
RewriteCond %{THE_REQUEST} ^[A-Z]+\ /~adminxx/[^\ ]*\ HTTP/
RewriteCond %{HTTP_HOST} ^(www\.)?xyz\.com(\.|\.?:[0-9]+)$ [NC]
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^(.*)$ http%2://www.websiteA.com/$1 [R=301,L]

Jim

[grandma genie]

Hi Jim

I asked my host to make the changes for me and here is what he said:

... as we've discussed before, this is a PHP issue that I can have my developer make work, yet a 301 redirect won't function right as it is looking at the same physical location on the server and would go into a loop. I would need my developer to edit the index page and put some code at the top that will look to see if the request is coming from secure.hostinghere.com and if so redirect to www.websiteA.com.

I believe I estimated about 30 minutes to accomplish this. Please advise if you would like me to proceed...."

So I assume it can't be fixed in htaccess. I asked him to take care of it.

I appreciate all your help.

Jeannie

[jdMorgan]

yet a 301 redirect won't function right as it is looking at the same physical location on the server and would go into a loop.

The code I posted addresses the problem your host mentioned. It does so by examining THE_REQUEST.

If you have not tested it, please do so, as this code may be useful to solve another member's problem in a concurrent thread.

Jim

[grandma genie]

I tried it, but it still does not work. I have some other stuff in the htaccess file. It looks like this:

php_flag display_errors 1

order allow,deny
deny from 69.175.79.162
deny from 70.16.145.82
deny from 78.96.13
deny from 78.97.7
deny from 78.137.163
deny from 78.170.247
deny from 78.170.244
deny from 79.98.31.40
deny from 79.115.201
allow from all

ErrorDocument 404 /notfound.html

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?websiteA.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER} !^https://(www\.)?websiteA.com(/)?.*$ [NC]
RewriteRule \.(jpe?g|gif|bmp|png|ico)$ images/hotlinked.gif [L,NC]

I put the code you suggested at the bottom of all this. Is that the correct place to put it?

Jeannie

[grandma genie]

Oh, help! Here is the code my host put in the index.html page to cause the old url to redirect to the new:
<?php
if($_SERVER["HTTP_HOST"] == "secure2.hostinghere.com")
header('Location: http://websiteA.com/');
?>

He charged me $40 to do that and it is not doing what I wanted done. He also changed the index.html to index.php making my ability to edit that page even more difficult.

So, here is my question. it seems the site secure2.hostinghere.com/~adminxx is an exact copy of www.websiteA.com and they both reside on the same server. Google et al are still indexing the old URL pages. Anyone finding those URL links is going to the old URL and finding the duplicate site with no images because I had added htaccess code to block hot-linking and so the old URL is being treated like a second site and my htaccess code won't let it see the pix. Is there a way to have anyone finding those links in the search engines going to the old URL to be redirected to the new URL? Am I asking for a miracle? I think I am doomed to never have good placement again in Google because of this problem, caused my my host, who wants to charge me every time I ask him to change something he caused. Help!

[g1smd]

The entire problem is solved with the .htaccess code supplied above.

When a request for the wrong URL version is received the server issues a redirect to the correct one.

There was no need for your host to anything, and what they did do shows they don't fully understand how it works.

[grandma genie]

So, do I need to have the files changed back to their original state before I try the htaccess coding again. Also, see my remarks above giving the htaccess file I have now in the root directory. Where should the new code be put? At the top, at the bottom, in the middle? My host also had another php file created called redirect.php which has this code in it:
<?php
phpinfo();

?>

So, my original index.html file has been altered into a php file. The previous index.php file now is called index-old.php. And there is a redirect.php file. I think those are all the changes my host made to the files. If I put the 301 redirect code in the htaccess file now, will all the new php files make for a big mess? I'm not sure what to do. All I know is that what I wanted done is not happening, and my host doesn't know how to handle htaccess questions. Maybe I need a new host. Maybe I need to win LOTTO.

Jeannie

[grandma genie]

Hi all,

I'm sorry for all the confusion I am causing. I just changed everything back to the way it was on the server. My host failed to realize all my links point to the index.html page and were all broken. So that is now fixed. And, I still want to try to redirect traffic coming from this site [xyz.com...] to www.websiteA.com www.WebsiteA.com is the correct URL that I want to keep in tact. So, since the old URL is an https, how would I write the code for making the redirect? The site xyz.com is on a secure server so has the https and it does not use the www in front. Does it matter that the xyz site has a "dot" in the name, like xyz.ww.com/~adminxx. Would this be the right coding? At this point if I can get this accomplished, it will be doing alot.:

RewriteEngine on
#
# Externally redirect old mod_userdir-format requests to canonical hostname
RewriteCond %{THE_REQUEST} ^[A-Z]+\ /~adminxx/[^\ ]*\ HTTPS/
RewriteCond %{HTTPS_HOST} ^(www\.)?xyz.ww\.com(\.|\.?:[0-9]+)$ [NC]
RewriteCond %{SERVER_PORT}s ^(443(s)|[0-9]+s)$
RewriteRule ^(.*)$ http%2://www.websiteA.com/$1 [R=301,L]