Avoiding A Loop During IP Change

[cooldogs]

I'm somewhat new to .htaccess and ModRewrite rules.

I have a client using WebSite Tonight through GoDaddy, in a few days we will be switching his domain from that particular server to a new dedicated IP that will be all his.

I'm assuming the step is to have the new site waiting at the dedicated IP (and it is) and then request that GoDaddy redirect the domain to the new IP

The way the new IP is set up it could be indexed by the search engines on it's own, something I do not want to have happen, i.e., xx.xx.x.#*$! would be in the index as would domain.com - a canonical issue.

Anyway to avoid this I've been using noindex on the pages under construction at the new IP. But I want the no index gone from them of course before the domain is pointed at the new IP.

My strategy is to have an .htaccess file waiting on the new IP that redirects any requests for the ip url to the www.domain.com url. So when the site propogates the double indexing won't be a problem.

All this said what will be waiting on the new IP server while the site goes through propogation is ...

RewriteEngine On
RewriteCond %{HTTP_HOST} ^xx\.xx\.x\.#*$! [nc,or]
RewriteRule (.*) [domainname.com...] [R=301,L]

Right now it works fine, if someone approaches the new IP via using the IP number as the url it redirects them to the "old design" at domain.com on the web site tonight server since that is where the domain.com currently "resides".

What I'm concerned about, is, can this cause a loop of any kind once the site starts to propogate and the domain is pointed from the old server to the other new one.

I'm a good web "designer" but not the most technical person in the world so please forgive me for my lack of knowledge on this.

Any help would be much appreciated. Thank you.

[g1smd]

Your code is almost right. Remove the "or" from it.

In fact, as there is no casing for digits remove the "nc" as well.

You will also need a site-wide redirect from non-www to www on the new server. It's very similar to the code you already have.

You might as well also go ahead and fix things up so that named index file requests also get redirected to strip the name off. This redirect has to go before all of the others. There are plenty of example code snippets for that posted in recent days.

.

I am not sure what you mean by "put a redirect on your old host". That doesn't sound right.

What you need to do is for the DNS to be altered to point at the new server. Once that happens, people seeing the new DNS information will see the new site. They will never see the old site again, unless they have a way to access the old site via the IP address or something. As it takes time for the DNS information to the propagated around the world, some people might still be seeing the old site for a few days. Once everyone is seeing the new site (allow a week for that), you can take the old site offline.

A day or two before you change the DNS settings around, add a message to the old site saying that it is moving to new servers and that there may be a short down time in the next few days. Also state "when you no longer see this message, you are looking at the new site".

If you do it this way, then there is no danger of a loop. Requests for IP are redirected to domain name. Domain name will resolve to either the old or the new server. At the moment, everyone sees the old server. When the DNS changes some people will continue to see the old server for a few days, and some will see the new server within a few hours. After a few days everyone should be seeing the new server.

[edited by: g1smd at 3:32 pm (utc) on Oct. 5, 2008]

[jdMorgan]

The code is incorrect, in that the final RewriteCond of a rule should never have an [OR] flag on it. The [OR] flag operates between the RewriteCond line that it is attached to, and the next RewriteCond that follows that line.

In addition, the [NC] flag has no meaning if the pattern contains only numeric values. It is intended to make alphabetic pattern matching case-insensitive (NC = "No Case sensitivity")

Do not feel free to modify anything about the mod_rewrite syntax shown in the documentation: Be aware that Apache modules use very simple (and so very fast) parsers, and it's best to stick strictly to the documented syntax to avoid potentially-nasty surprises when switching servers or server operating systems.

I'd suggest:

RewriteEngine on
RewriteCond %{HTTP_HOST} ^192\.12\.34\.56\.?(:[0-9]+)?$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]

The fancy stuff on the end of the IP address pattern above allows a match if a period is appended to the IP address (FQDN), or if a port number is appended, or both. All three forms are valid (if rare), and should be handled properly.

You will certainly not want to "request that GoDaddy redirect the domain to the new IP." Doing so would work, but it would trigger the rule above for every request. Instead, all you want them to do is to update the DNS records in the zone file for your domain name, so that the domain resolves to the new IP address.

Jim

[cooldogs]

Thank you that is very helpful.

The propogation is a little confusing too, as the current WebSite Tonight location of domain.com supposedly complicates issues. When we got the new dedicated IP we were told that the switch would occur as you suggested above without much or any down time at all. But supposedly the way GoDaddy and WebSite Tonight interact the WebSiteTonight hosting has to be disabled completely before the domain.com can be pointed to the new IP, this causing some downtime during propogation, just as you would have if you switched to a standard shared IP address. The tech explained it as something where the request to point to the new IP could not be made unless the old hosting package at WebSite Tonight was cancelled first, they were blaming this on the WebSite Tonight situation.

[g1smd]

Ah, the perils of having your domain name registration and your hosting both with the same company... and working with a brain-dead company such as GoDaddy, 1&1, etc.

[cooldogs]

Based on what I've been given so far from the different places I've gathered code including this post .. would the following code work to handle the IP issue, the www canonical issue and page name change(s). With this code would the "or" have to go back into the IP RewritCon or on the end of the first page name change, if so can you tell me where?

Thank you so much for your help.

Options +FollowSymLinks
Options +Indexes
RewriteEngine On
RewriteCond %{HTTP_HOST} ^192\.12\.34\.56\.?(:[0-9]+)?$
RewriteCond %{HTTP_HOST} ^mysite\.com [nc]
RewriteRule (.*) http://www.example.com/$1 [R=301,L]
RewriteRule ^dog.html$ [domain.com...] [R=301,L]
RewriteRule ^cat.html$ [domain.com...] [R=301,L]

I'm assuming I cannot add the RewriteCond ^mysite\.com [nc] until that particular domain.com is pointing to that IP and has propogated fully.

I also added the
Options +FollowSymLinks
Options +Indexes
part of the code because that is what I found for a page name redirect, I'm not sure if it is necesary or not though.

[g1smd]

You have the order all scrambled. What is says, right now is this:

"If the requested hostname is 192.12.34.56 and at the same time the requested hostname is mysite.com then redirect to www.mysite.com".

That can never be true. The hostname has only one entry, so one will be true and one will be false.

.

You already had the correct rule for IP redirection, posted above by jd.

There are two ways to fix this. Choose one:

1. Copy the correct IP rule (both lines) and change the IP part to be the non-www domain. You'll still need the "port number" stuff just as before. Place both of those lines BELOW the IP rule you already had.

That is, each "Cond" (one for IP, one for non-www) will *each* have one "Rule" after it. Those two rules will, in fact, be identical in this case, both redirecting to www.

2. On the very first RewriteCond add [OR] to the end of the line. Now the redirect is processed if one of the statements is true.

Choose one of the above two ways to fix this.

.

You can add the rule for non-www right now. Nothing will happen until the DNS does resolve, but there can be no harm whatsoever in adding the rule immediately.

.

The very specific rules for "dog" and "cat" are OK, do *not* need any additional conditions, but *must" be moved such that they are the very first listed redirects.

The generic "IP to www" and "non-www to www" redirects must be last in the list.

[edited by: g1smd at 4:40 pm (utc) on Oct. 5, 2008]

[cooldogs]

Is this closer and/or correct ...

Options +FollowSymLinks
Options +Indexes
RewriteEngine On
RewriteCond %{HTTP_HOST} ^192\.12\.34\.56\.?(:[0-9]+)?$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]
RewriteCond %{HTTP_HOST} ^example\.com [nc]
RewriteRule (.*) http://www.example.com/$1 [R=301,L]
RewriteRule ^dog.html$ http://www.example.com/my-dog.html [R=301,L]
RewriteRule ^cat.html$ http://www.example.com/my-cat.html [R=301,L]

Am I missing "OR" anywhere it needs to be. Are the first two Option directives necessary, not sure if the [nc] is necessary, it's from sample code I got somewhere else. Thanks.

[g1smd]

You need to add the "port number stuff" to the non-www entry.

The "specific" dog and cat stuff *must* be listed first.

[edited by: g1smd at 4:42 pm (utc) on Oct. 5, 2008]

[jdMorgan]

I strongly suggest that you read the Apache mod_rewrite documentation [httpd.apache.org], and stop guessing about this stuff; One little typo can have wide-ranging and damaging effects. You can easily break your site or nuke your search engine ranking with a single typo...

The per-page redirects will have no effect; Those two rules will never be executed, since they follow the less-specific hostname redirect.

If you add a second positive-match RewriteCond, then the first RewriteCond needs an [OR] flag on it.

Again, do not feel free to deviate from the documented syntax, including the casing of flags and server variables.

This is all you should need for now, unless you wish to add the "index"-to-"/" redirect function suggested by g1smd above:

Options +FollowSymLinks
RewriteEngine on
#
RewriteCond %{HTTP_HOST} ^192\.12\.34\.56\.?(:[0-9]+)?$ [OR]
RewriteCond %{HTTP_HOST} ^example\.com [NC]
RewriteRule (.*) http://www.example.com/$1 [R=301,L]

An equivalent rule, assuming that you have no subdomains other than "www" and do not intend to add any at a later time is:

RewriteCond %{HTTP_HOST} !^www\.example\.com$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]

This simply says, "If the requested hostname is not exactly "www.example.com", then redirect to www.example.com. It will redirect requests for *anything* except www.example.com, and so covers requests for non-www, for the IP address, for case-variations, and for the FQDN and appended-port cases as well.

Don't modify this code in any way, unless you know why you are modifying it and understand all of the repercussions of the modification. mod_rewrite is *not* a "cut-and-paste from snippets found all over the Web" kind of thing, it is server configuration code. Treat it accordingly -- with a great deal of caution.

I removed the Indexes option because you did not state why you wanted it, and it's generally not a good idea to add it unless you want the general public digging through all of the directories on your server to find and download unlinked files.

Jim

[cooldogs]

Thanks Jim, I'm trying. I know it's complicated and that it has to be exact. Hence, why I am here. The Apache documentation is so confusing to me - so all I have are tutorials and what I can find by researching for sample code on the web and then changing it to fit my domain name. Sometimes it's hard to tell exactly how to do it and/or how to make it "all" come together correctly.

I have four tasks that I want to do with my .htaccess file ...

1. Solve any IP canonical issues by pointing IP number requests to the domain name www.domain.com
2. Solve any www canonical issues by pointing variations to www.domain.com
3. Change some page names via 301
4. Set up a 404 redirect

Can I try again ...

Options +FollowSymLinks
RewriteEngine on
RewriteRule ^dog.html$ http://www.example.com/my-dog.html [R=301,L]
RewriteRule ^cat.html$ http://www.example.com/my-cat.html [R=301,L]
RewriteCond %{HTTP_HOST} ^192\.12\.34\.56\.?(:[0-9]+)?$ [OR]
RewriteCond %{HTTP_HOST} !^www\.example\.com$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]
ErrorDocument 404 /error.html

I included the 404 I've used successfully on my .htaccess before when it simply had simple 301 Redirects, will it still work here with what we've done today. If so, not sure if it goes on the end or just below the page name redirects.

I thank you all very much for your patience, your the best, I really appreciate it.

Thank you.

[g1smd]

The ErrorDocument line can go anywhere.

If you are going to be using the "not www" line then you don't need the "IP" line at all now.

You would use [OR] only if you were looking at IP *or* non-www.

Make sure you add some comments before each rule so that you will know what they are for, six months from now.

[cooldogs]

Great, it's always so nice to finally "get it".

Thank you so much for your patience.

[g1smd]

In this forum, we prefer to guide you to discover the correct answer by your research and your trial and error.

Note that there is never one "correct" way to do things, but there are very many wrong ways to do this stuff.

For instance, I would have one rule for non-www and one rule for IP, (because I would most likely have more than one domain sharing space in that hosting account, and would have other rules for those too) whereas jd prefers to combine them into a single "not www" rule, most likely because the account would host only one site.

This forum tries very hard to not be a free code writing service, providing cut and paste solutions that the recipient never actually understands, and will not be able to maintain as their site evolves over time.

Doing it this way, we hope leads you to learn more about the process, and make a lot of further research.

[edited by: g1smd at 5:53 pm (utc) on Oct. 5, 2008]

[jdMorgan]

You should escape all literal periods in regex patterns like this:

RewriteRule ^do[b]g\.h[/b]tml$ http://www.example.com/my-dog.html [R=301,L]

I have no personal preference for the explicit hostname canonicalization versus the "not www" method. As I stated, it depends on the site and whether it has additional subdomains (or domains) hosted in the same DocumentRoot.

Among the several correct ways to do things, there is some room for variations in style and approach. But since this is a technical discipline, any preference for one method over another should be based on the requirements for a particular site and how it's set up.

Jim

[cooldogs]

I totally agree with you g1smd, not only is that that best way to learn it is also the most gratifying, at least for me.

I am very appreciative that you and Jim are patient enough to help through the guidance process and kind enough to let me know when I had something that would work in my particular instance.

It always adds to the thread and learning experience to have a few different inputs as well.

Thanks again.

[g1smd]

Did you also need the index redirect? If you do, you need to add that directly after the dogs and cats stuff and before the other stuff. There are some hints in: [webmasterworld.com...]

The last thing to do is to thoroughly check that it all works OK by throwing a range of expected and unexpected URLs at your server.

A tool like Xenu Linkleuth can at least partly automate that for you. It will take a few minutes to set things up, but you will be able to re-run the tests in seconds any time you want.

I detailed some of it, just yesterday: [webmasterworld.com...] - See the items numbered 1 to 4 (in bold) in one of the posts, towards the end. You'll need to do that once the DNS has propagated. Your test URLs will be somewhat different to the example there, but I hope you'll get the idea. You'll need a group for IP, a group for non-www, and a group for www.

[cooldogs]

Thank you soooooooo much G1smd, I had totally forgotten about that issue. Based on the thread you suggested I tried the lines below ... keeping in mind I had to use my IP number for now to test it. It seemed to work just fine.

RewriteEngine On
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]*/)*index\.(html?¦php)(\?[^\ ]*)?\ HTTP/
RewriteRule ^(([^/]*/)*)index\.(html?¦php)$ [#*$!.xx.xx.xx...] [R=301,L]

When ready to combine with everything else do I have it in the right place here below ... everything look workable?

For the total package, I added what I hope is a correct way to redirect the index.html and changed the page name code per Jim's suggestion with respect to literal periods.

Options +FollowSymLinks
RewriteEngine on
RewriteRule ^dog\.html$ http://www.example.com/my-dog.html [R=301,L]
RewriteRule ^cat\.html$ http://www.example.com/my-cat.html [R=301,L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]*/)*index\.(html?¦php)(\?[^\ ]*)?\ HTTP/
RewriteRule ^(([^/]*/)*)index\.(html?¦php)$ http://www.example.com/$1 [R=301,L]
RewriteCond %{HTTP_HOST} ^192\.12\.34\.56\.?(:[0-9]+)?$ [OR]
RewriteCond %{HTTP_HOST} !^www\.example\.com$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]
ErrorDocument 404 /error.html

A side note to those who might try this code: I noticed in the thread you sent me to G1smd your advice ...

Don't forget to also change the ¦ to be real "pipe" symbols. The forum software does not display them correctly. Something important to know for sure.

[g1smd]

Good to test with the IP address. You can be sure that it works, and you just change the IP address to be the domain name some time before the DNS changes over.

Ah, yes. The ¦ pipe symbols get modified. Important thing to know.

Also, you can't post 3 x in a row in the forum. I use nnn and that works.

The code looks good except for one thing.

After each RewriteRule put a blank line to break things up.

After each blank line, add a:

# Comment to explain what the next one or two lines does.

You'll still understand the code next year if it has comments.

[edited by: g1smd at 9:29 pm (utc) on Oct. 5, 2008]

[cooldogs]

Above I should have left out the line
RewriteCond %{HTTP_HOST} ^192\.12\.34\.56\.?(:[0-9]+)?$ [OR]

As Jim suggested I was not going to need it as I'm hoping the next line will cover the issue - there will be no subdomains, shared or parked domains sharing that IP.
RewriteCond %{HTTP_HOST} !^www\.example\.com$

[g1smd]

Ooops. I missed that the error had crept back in.

Actually, in this case, the rule would still work, because when the request is for the IP, the request is not for the www domain - but it is best to get it 100% right.

[g1smd]

I have revisited a three year old .htaccess file today and have had to make a pile of corrections and enhancements.

I used ^(.*)$ when (.*) would have sufficed.

I had stuff like filename.html when it should have been filename\.html - used with just a dot, it matches ANY single character (not just a literal dot) at that position (so filenameZhtml would match too).

I had end anchored domain names. In that case they would not have matched if there was an appended port number in the request.

I had used .* far too often, when a more exact pattern should have been used instead.

... and on and on and on.

[cooldogs]

I get down right embarrassed when I visit pages I did three years ago, it's amazing how far I've come since then .. and still have yet to go.

I've really enjoyed learning how to "do it right" here today. Maybe by doing my first big .htaccess file right from the start I will look back in three years on what I've done today and not be embarrassed.

Thanks

[cooldogs]

g1smd when you say "it is best to get it 100% right." do you mean leave in or leave out the ... line ...
RewriteCond %{HTTP_HOST} ^192\.12\.34\.56\.?(:[0-9]+)?$ [OR]

By the way, how do you do the box around quoted text, I can't seem to find instructions on that anywhere. Sorry for my newbieness.

[g1smd]

If the other line tests for "NOT www" (using the ! prefix), then you can leave the IP test out. It is redundant.

The alternative was to test for "^domain.com" *or* "IP" with two separate conditions.

.

You have got a good start... but next you can perhaps look at also redirecting malformed incoming links.

Just the other day someone has linked to me like:

www.example.com/thisfile.html[b]_[/b]

which was 404 of course. There is now a redirect to capture the traffic and not serve the 404 error any more.

I was alerted to the error by the duff URL appearing in the Google WMT "Crawl Report".

I already had code to redirect when the URL request had a trailing dot or trailing comma. I also added semi-colon into that redirect mix.

Yahoo SiteExplorer can be a great place to find many of the duff URLs that other sites think exist on your site.

.

Something else you can do in .htaccess is blocking nasty bots. That needs a lot of care to not block real users and not block the bots of the major search engines.

Earlier in the year I managed to block every person that used Norton Internet Security who visited a particular site. At the time I was trying to stop excess traffic caused by AVG's LinkScanner product and its' brain-dead implementation. Due to a simple typo I also blocked every Norton Security user too. I didn't notice it for a couple of weeks as I had not been looking at the logs. I just happened to see it when I used a PC away from home and was blocked. I knew immediately what the problem was, but it took a few hours to fix the problem.

[edited by: g1smd at 10:24 pm (utc) on Oct. 5, 2008]

[g1smd]

Tags that work on this forum, all like this [tag]...[/tag]

b - bold
i - italic
quote - text in a box
code - monospaced typeface

There are a few others, but I rarely use them.

[jdMorgan]

He means that one little typo can take down your server. Or worse, it can slowly and silently destroy your search engine ranking over time...

If you use the !^www\.example\.com$ pattern, then it is not necessary to check for any other hostname or for the IP number; If you use the !^www\.example\.com$ pattern, and a client sends a request with a Host header that is not exactly "www.example.com", it will get redirected.

A link to the "posting instructions" is to the left on the "Reply to this topic" screen, labeled "Style Codes".

Jim

[g1smd]

*** A link to the "posting instructions" is to the left on the "Reply to this topic" screen, labeled "Style Codes" ***

How long (of the six years I have used this forum) has that been there?

Never noticed it before. Ooops.

[cooldogs]

A link to the "posting instructions" is to the left on the "Reply to this topic" screen, labeled "Style Codes".

Thank you know I know.

[cooldogs]

While do page name changes I noticed that currently on his older Web Site Tonight server he does not have an index.html page but rather the home page is located at Home_Page.html. Can this be resolved in a simple page name change as below or is it more complicated like that
which resolves the index.html issue. Keep in mind that the "home page" will be index.html once the switch to the new IP is done.

RewriteRule ^Home_Page\.html$ http://www.example.com/ [R=301,L]