Forum Moderators: phranque

Message Too Old, No Replies

Site change of URL

possible tremendous headache-need pointers-

         

henry0

11:35 am on Sep 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I might be in a predicament, which is an understatement for other words that cannot be printed!
I have a client that is possibly (not sure yet) figuring that he cannot any longer use its Co name (I pass on details) that name is indeed part of the DN & URL

The site I ref to contains about 70 folders and over 10K files PHP and MySQL driven.

How should I be prepared for the task of URL changing?
Does the site need to be reloaded?
Actually what the whole move implies?
How will you handle the situation?
(FYI- don’t think I’ll be rude in non responding to posts I will not be back before mid pm)

Thanks

Regards
Henry

coopster

11:41 am on Sep 18, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



Is it going to be possible to keep the *old* site up during the transition? If so, set up the new site, test everything in great detail, then change the records (DNS, etc.) to point to the new site.

One thing I would check for sure would be hard-coded paths in any of your code. I try to use relative paths, that way code is reusable from site to site as long as directory structures are the same.

Good Luck with the transition!

TGecho

12:19 pm on Sep 18, 2003 (gmt 0)

10+ Year Member



If you're going to be using a new domain name, It sounds like the biggest task will be to convert the urls over to the new one. Whether that involves a massive find a and replace operation or the simple editing of a text file...

Then just move all the files (don't forget the DBs :)) to the new server (with the new domain pointing at it). Once you're sure everything is functioning, set up the old domain to redirect to the new one.

As for SE stuff, you'll have to wait for someone with more knowledge in that area.

henry0

10:30 pm on Sep 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I am back; thanks for sound advises
Henry

Zaphod Beeblebrox

10:41 am on Sep 19, 2003 (gmt 0)

10+ Year Member



And really, next time use relative paths, so not href='http://www.mydomain.com/details.asp' but simply href='/details.asp'

claus

12:49 pm on Sep 19, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> As for SE stuff

To make the Search Engines aware that a change has happened, you should leave the old site up and running for a while after the change (a month or so, perhaps a bit more).

During this time you should put a condition like this in your root .htaccess file, to make sure that all visits to your old domain will be pointed to your new domain:

RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www\.)?old-domain\.com
RewriteRule (.*) http://www.new-domain.com/$1 [R=301,L]

The above one assumes(*) that all path- and file-names are identical - ie. that only the domain name has changed.


If this is not possible, include a link to your new domain on all pages of the old domain and have nothing else than that on these pages.

In the <head> section of the source code of the pages on the old domain, put this tag:

<meta name="robots" value="noindex,follow">

You can also add this, just make sure your old pages have no content apart from the link to your new domain and the meta tags:

<meta http-equiv="refresh" content="2;url=http://www.new-domain.com/">

You should not remove the pages, as they will have links pointing to them, and you want the SE's to follow the links from these domains via your old domain to your new domain.

After a minimum of a month or so, you can start feeding the spiders 404's in stead (by deleting sub-pages - assuming you use the meta-approach) to get the remains of your old domain removed from the indexes. Then after doing that for a while - remove the index page and DNS service for the old domain, to erase it completely from the SE indexes.


On the other hand, if you can use the "rewrite" method, it will be safe to pull down all pages instantly, as a request for (a) will be redirected to (b) and the content of (a) will not be shown at all:

(a) http //www.old-domain.com/path/to/page.html
(b) http //www.new-domain.com/path/to/page.html

The "rewrite method" does not need to produce 404's. If you can - use it in stead. It's far better, as it includes a "301" status code that notifies all spiders that your pages have moved permanently. After having it up for a month or two, just disconnect DNS service for the old domain to get it deleted completely from the indexes. If you don't really dare to do this, just let it be active with the 301 on it forever and ever. It will not harm you in the SE's but it's cheaper and better to dump the old domain altogether, imho.

/claus


Edit: Added <hr>'s to clarify
(*) Notes: It also assumes that you run the Apache webserver on your old domain. Dots need to be escaped like this: "\." but not at all places, as the example shows. This part of the code: "(www\.)?" assures that it works no matter if your old domain is reached with the www prefix/subdomain or not.

[edited by: claus at 1:09 pm (utc) on Sep. 19, 2003]

coopster

1:02 pm on Sep 19, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



Wow. Great advice. Great post, claus. Somebody has done this before ;)

henry0

3:37 pm on Sep 19, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hello Claus,
I am impressed thank you so much for your tutorial
actually I wish that it will saved and used as future ref

Henry

slobizman

8:00 pm on Sep 25, 2003 (gmt 0)

10+ Year Member



I saw another version of what I think accomplishes the same thing:

rewriteEngine on
rewriteBase /
rewriteCond % {HTTP_HOST} www.olddomain.com$
rewriteRule ^(.+) [newdomain.com...] [L,R=301]

Is one any better than the other? Do they really do the same thing? I just want to make sure that every possible URL string (all the slashes, dashes, question marks and such) will be redirected. Structures all the same--I am just renaming the site on the same server (will have both active for some time).

Also, I'm confused as to if a page actually comes up that tells the user that he is being redirected to a new page. If so, can it be modified to also say something like "Note Site Name Change?"

claus

8:11 pm on Sep 25, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> Do they really do the same thing?

No. The one you posted will not redirect this request:

http //olddomain.com/

- as www is not used. Some (more experienced) users will enter domain names in the address bar without the "www.", as this saves them from four characters.

>> all the slashes, dashes, question marks and such

Everything entered after olddomain.com/ will be translated to the same address on newdomain.com/ in my example above.

>> if a page actually comes up that tells the user that he is being redirected to a new page

No. The user (browser) is just redirected, so that the new address shows in the address bar and the new page shows in the browser.

The minimum advice of one month is specifically targeted at Search Engines, not users - users will see this effect immediately, whereas search engines will need at least a full month to update their listings.

>> also say something like "Note Site Name Change?"

This is what "R=301" does. It tells the User-Agent (browser, Search Engine) visiting that the page is permanently moved to a new location, and that the User-Agent should use the new location in stead of the old one.

This is done by sending a Server Header to the User-Agent with a status code that reads "301 Permanent Redirect". It will not be displayed on the screen.


added:
If you want to make a note to your visitors you will not be able to benefit from the 301 redirect, and your old (and wrong) URL will remain in the Search Engines for as long as that page is on your site.

A 301 is the "clean way" to do it - users are redirected instantly and Search Engines are told to update their listings simultaneously.

slobizman

5:10 pm on Sep 26, 2003 (gmt 0)

10+ Year Member



Claus,

Thank you very much for the information.

Just to make sure I'm doing this right, I don't currently have the htaccess file in my home directory, so when I make it, all I put in is the code you posted above, right? No other headers or anything...?

Since I have you, could I please bother you to tell me if I have any holes in my plan? The purpose of all this is to change the name of my site to olddomain.com to newdomain.us (It's .us for a reason; I can't explain it or I'll violate the rules of naming the web site in the post, but I will have the .com, .org and .net versions URL Redirect from the registrar's control panel to it in case people key the others by mistake).

Here's my plan:

1. Change my DNS name servers in newdomain.us to what is currently in my olddomain.com name servers. Then, wait about three days to populate through the internet.

2. Upload the htaccess file with the code you described.

3. Tell my web host to point newdomain.us to my web site. I'll then have both olddomain.com and newdomain.us pointing to my one web site.

Result: when someone keys in olddomain.com/... they'll be redirected to newdomain.us/... Plus, Google should take note of all this when it crawls the site and I hopefully will retain my page ranking.

Sound right?

claus

6:21 pm on Sep 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> The plan

You should make #2 the last point in stead of the middle. The .htaccess file will work the minute it's uploaded, and if "newdomain.us" does not point towards anything at that point, your users will just be redirected to a "server not found" page.

Otherwise your plan will work perfectly. Your new domain will point to the nameservers of your host, and your host will make it point to the same web space as the old one. Then, the .htaccess file will make sure that the URL's on "olddomain.com" will be replaced by the same urls on "newdomain.us" as they should.

Google will take note immediately, and it will not appear to have done so. It takes a good while before you will see that the domain is changed in the SERPS, but the new domain will be found anyway, although you don't see it instantly. Plus visitors to the site will be sent to the new domain "automagically" by the .htaccess file although they come from Google (or anywhere else).

Wait at least until the domain is changed in the Google SERPS for all pages you can find (some will change faster than others), before you disable the old domain. A month is my best guess, so don't panic when you see that it doesn't change :)

>> retain my page ranking

Well, this is the reason that you should wait at least that month. The Googlebot is pretty quick and pretty clever, so it will find out almost instantly, but the remaining Google machinery takes time to operate, and you will have all kinds of incoming links to the old domain that needs to be "transferred/translated" into "votes" for your new domain.

The 301 is considered the "safe" method to transfer that page rank. It's also the "proper way" in terms of how web servers and domain names generally work. The DNS system does not have "history" - one day your site will be on one domain, the next day on another. Your server must tell the world that you have relocated, otherwise there are two sites in stead of one (one dead site with incoming links and one brand new one without).

The 301 status code simply says "this is the very same page, but now it's just on another location, so act accordingly". Google will need to adjust the whole linking matrix (i don't think the term "web graph" is very descriptive) to take this into consideration and it takes some time. (That's also why some pages might change URL later than others in the Google SERPS as there are differences in backlinks)

Also, i suspect that Google incorporates a little extra lag time to make sure this technique is not used ...hrm abused. (this might be wrong, it's is just my own suspicion but i think it makes sense as Google ought to be able to implement these changes much faster, and nobody would want to put a money-making site down for a month (to promote another) if they don't have to)

>> The .htaccess file

This is a plain ASCII text file - don't use any kind of formatting or anything, just write it in notepad or another pure-text editor. You don't need to put anything in it apart from the above few lines (but of course you have to edit the domain names to suit your purpose first).

The dot preceding the file name is the apache way of saying "this is a hidden file" - it must be there. There's no file extension, just the "dot htaccess".

It must be placed in your root folder. Before you upload it, you can save it on your PC as "htaccess.txt", upload it, and then rename it to ".htaccess" once it is on the server. Be sure to upload using the ASCII setting in your FTP program.

That's basically it.

Sorry about the long post, i included a couple of background things that was not in the post above.
/claus

slobizman

6:36 pm on Sep 26, 2003 (gmt 0)

10+ Year Member



Thanks so much! I can't begin to tell you how appreciative I am of your assistance.

slobizman

6:41 pm on Sep 26, 2003 (gmt 0)

10+ Year Member



Actually, i do have a htaccess file. It contains the following:

Options Indexes FollowSymLinks Includes ExecCGI
AddType application/x-httpd-cgi .cgi
AddType text/x-server-parsed-html .html

Any conflict? Do I place your code above or below it?

claus

12:33 pm on Sep 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> Any conflict?

Theres's no conflict. The .htaccess file can be used for all kinds of things that has to do with general settings on the site.

>>Do I place your code above or below it?

In you case, this part of the lines that are already there is actually required for the rest to work. It's not required in all cases (it depends on the server configuration), so i'm glad you posted it:

Options FollowSymLinks

If this line is found in your .htaccess file, you must always place the redirecting code below it. I see this line specify other options as well, that's no problem, and the order of them is not a problem either. You don't need to place the code immediately after it, just put it below the three lines - you can even include a few blank lines for readability. Or, you can include this line in-between the two blocks:

# Lines below are for redirects. Options FollowSymLinks required before them

The #-character makes it a comment that will not get executed by the server. It is needed as the first character on any line that is a comment.

Also, for others, if you cannot make the "rewrite engine" work, place the "Options FollowSymLinks" line before it, that will probably do the trick.

/claus

slobizman

2:03 pm on Sep 30, 2003 (gmt 0)

10+ Year Member



Success! Thank you so much. Armed with the knowledge you provided for me, it went without a hitch.

Too bad as soon as I got this done my web host decided to tell me I have to leave the shared server for a dedicated server. Now, I'm looking for a new web host.

I'm very confused (what's new?), however, on the process of how to move a busy, dynamic web site (1500 message board posts a day) to a new host without being down for the entire time it takes for the DNS nameservers to propagate. I've asked the question in this web site in another forum (here: [webmasterworld.com...] ), but no one is explaining this clearly enough for me. If you have the time it would be great....

claus

5:00 pm on Sep 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> move ... to a new host without being down for the entire time
>> it takes for the DNS nameservers to propagate

Those that thought out the DNS system were a clever bunch. They thought of this, and there is a method to specify how long your DNS records should be cached, to reduce that propagating time. The trick is to reduce the TTL (Time To Live) gradually before you make the shift.

If it's now set to one week,
-> reduce it to one day,
-> then wait a week for the servers to catch up,
-> then set it to an hour, wait a day for the servers to cath up,
-> set it to zero, wait an hour or two or three for the servers
-> make the change
-> set the TTL back to a week

It will not eliminate all problems, but it will greatly reduce them and make the transition more smooth.

Of course this demands that you have control over yout DNS TTL settings. If you don't have, your host has. Not all hosts know about this though, as weird as it sounds - dedicated DNS providers provide this option (that's why some people choose to host their DNS elsewhere than their website). Here's a recent thread on the topic:

Moving website to another server [webmasterworld.com]

In post #9 i've written a little bit more detail on this.

/claus

slobizman

5:33 pm on Sep 30, 2003 (gmt 0)

10+ Year Member



Thanks again. As usual, you described it perfectly.

I looked at my domain registrar and it appears that they do not have the TTL settings in their Control Panel. I guess they would be under the control of my web host.

I can imagine web hosts are not always that thrilled to work with you on this if they know they are about to lose your business!

slobizman

1:13 am on Oct 3, 2003 (gmt 0)

10+ Year Member



Claus, thanks to you I've got the web site renamed and both the old and new domains are pointing to my web site.

I have noticed, however, that in my Google search, that the web site shows up nearly in first position as it should, but the URL it shows for the site is "old-domain.localweb.com" instead of my new domain name. It used to just show up as "old-domain.com.

Any idea why it would be doing this?

claus

2:29 pm on Oct 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Nah, not really, that sounds weird. It's some configuration issue at your host i think. Somehow the Gbot ends up at that URL when following links to your old domain i suspect. It ought to change to the right (new) address after some time though if everything is now properly set up.

Btw. If you use the "link:http://my-old-domain.com" search at alltheweb (ATW produces more results than Google for this query type) you will see pages that link to your old domain - it will be a very good idea to write to these and ask them to change their links. Otherwise run the query at google and contact only the fraction that google lists.

/claus

slobizman

6:00 pm on Oct 3, 2003 (gmt 0)

10+ Year Member



I passed your message to my web host tech. Here is his reply (my domain name is changed, according to the rules here)

----------from my web host--------
I do not know why you are experiencing the behavior you report. old-domain.localweb.com is an alias to the site, not unlike www.old-domain.com or www.new-domain.us. (The primary hostname for the site had been old-domain.com. I just changed that to newdomain.us).

You may want to add another line to your .htaccess file to redirect old-domain.localweb.com like you have for old-domain.com and www.old-domain.com. That should take care of this problem.
-------------------------

Do you think the change he made could fix this?

Should I add a line to my .htaccess file as he suggests? If I do, do I just add another "RewriteCond" line like the following:

CURRENT:

RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www\.)?old-domain\.com
RewriteRule (.*) [new-domain.us...] [R=301,L]

NEW:

RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www\.)?old-domain\.com
RewriteCond %{HTTP_HOST} ^(www\.)?old-domain.localweb\.com
RewriteRule (.*) [new-domain.us...] [R=301,L]

claus

10:18 pm on Oct 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



As it's an alias to the same site i believe your rewrite condition will work like you posted it. I would escape the dot before "localweb" in line two, like this:

RewriteCond %{HTTP_HOST} ^(www\.)?old-domain\.localweb\.com 

/claus

slobizman

2:51 pm on Oct 5, 2003 (gmt 0)

10+ Year Member



Interesting, when I changed the htaccess file as I wrote in my last post, it stopped redirecting my old-domain. I had to remove it.

closed

4:12 pm on Oct 5, 2003 (gmt 0)

10+ Year Member



That's because the second line should have the OR flag when you add another RewriteCond:

RewriteCond %{HTTP_HOST} ^(www\.)?old-domain\.com [OR]

claus

4:14 pm on Oct 5, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



slobizman, i know why - it's so obvious that i should have told you so before, i don't know why i didn't think about it. Sorry about that.

RewriteEngine on 
RewriteCond %{HTTP_HOST} ^(www\.)?old-domain\.com
[OR]

RewriteCond %{HTTP_HOST} ^(www\.)?old-domain\.localweb\.com
RewriteRule (.*) http://www.new-domain.us/$1 [R=301,L]

The [OR] flag was missing, and when it's not there, an "AND" is the default. As an AND doesn't make sense in this case the rule will stop working.

/claus

closed

4:18 pm on Oct 5, 2003 (gmt 0)

10+ Year Member



While we're on the topic of obvious flags, it wouldn't hurt to add the NC flag to each RewriteCond that looks at HTTP_HOST.

slobizman

6:26 pm on Oct 5, 2003 (gmt 0)

10+ Year Member



What is the NC flag and where does it go exactly?

closed

6:54 pm on Oct 5, 2003 (gmt 0)

10+ Year Member



The NC flag is used whenever you want to make the pattern match case insensitive. NC stands for no case.

Flags are placed within brackets at the end of the line, and are separated by commas when more than one exists. An example of the use of multiple flags is your RewriteRule line.

You can learn more about mod_rewrite at:
[httpd.apache.org ]

slobizman

2:17 pm on Nov 13, 2003 (gmt 0)

10+ Year Member



Well, it's been almost two months since I've added this to the htccess file:

RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www\.)?old_domain\.com [OR]
RewriteCond %{HTTP_HOST} ^(www\.)?old_domain\.localweb\.com
RewriteRule (.*) [new_domain.us...] [R=301,L]

The forwarding has been working, but I have completely lost my PageRank, which is now ZERO. '

In addition, when I check in google with the "link:" to the old_domain it give me 12 pages, but when I do it with the new domain, I get ONE ENTRY.

What the heck went wrong?

slobizman

1:05 am on Nov 14, 2003 (gmt 0)

10+ Year Member



In addition to my last message, I'd like to start forcing out all references to my old domain, and all bookmarks to the old domain. Since my Page Rank seems to have disappeared, I guess I don't need to worry about that anymore.

So, I made up a html page that says, "If you'r elooking for "old domain" it's been renamed to "new domain." Click this link to go there and then bookmark that page."

I have both domains pointing to my server.

What is my best move now? I tried the changing the htaccess file to the following, but it gave me an Internal Server Error.

RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www\.)?old_domain\.com [OR]
RewriteCond %{HTTP_HOST} ^(www\.)?old_domain\.localweb\.com
RewriteRule (.*) [new_domain.us...] [R=404,L]

This 32 message thread spans 2 pages: 32