Huge Site Migration

[hirerobhere]

Hello. I need to migrate a site that is currently on a centos/plesk configuration to a centos/cpanel configuration machine. One problem, the site is around 150gb of data. How do I go about doing this? Tar it up manually? The cPanel built in migration has failed, obviously. Any help would be much appreciated, need this done asap.

[mcavic]

Yep, I would do a tar backup and restore it to the new machine. If you're using mysql, use mysqldump to back up the databases.

If there's a message board, or some other situation where users are making frequent changes, and you want to make sure nothing is lost, you'll want to shut the site down during the migration.

[eeek]

Tar it up manually?

How much free space do you have?

[jeffatrackaid]

Not sure why you would want to go from Plesk to cPanel - I would consider that a step in the wrong direction.

Depending on the complexity and content of the site, there are different migration strategies. Also depends on where the new and old servers reside. If you are trying to transfer that much data over the internet, you will need to use a process that can be resumed.

I typically use rsync for such large migrations but it really depends on the underlying data, how it is structured in the directories and the goals of migration.

The tools built into Plesk/cPanel and their kin are designed for mass shared hosting. I find they fail once you start getting sites over 10-20GB.

Also, if you are running a single site. I would give up the control panel entirely and have someone manually configure the server for you. This reduces complexity, software maintenance requirements, removes control panel configuration issues and provides a more stable platform.

[vincevincevince]

I suggest copying the big media files which make up the bulk of the 150Gb (I expect) first. My preference would be the scp command, use it from the terminal on the old server to push the data to the new server in bulk. If you do not have SSH access, then you can write a short PHP or Perl script to transfer directly by FTP - avoiding having to download things to your local machine and upload again.

Once the big media is moved, delete it from the old server (or just move it out of the web root).

Now, use the built in site migration feature of cPanel, it should work fine now the site is small.

When it completes, copy the big media files into the correct folder on the new server. Migration complete.

[graeme_p]

Both rysnc and scp got mentioned above and I was wondering how they compare. I use rsync a lot (even for local backups), but have not used scp.

According to various bits of info I found (including a discussion I found on the rsync mailing list) scp if often faster transfering compressed files (e.g. most media files) over fast connections (i.e. LANs). Rsync should be much faster transfering uncompressed files over the internet, or, AFAIK, when you have lots of small files only some of which have changed.

[mcavic]

There's nothing wrong with scp for copying individual files or directories. But rsync's strength is in synchronizing filesystems by determining what files to copy. As for performance, the way I use rsync, it calls ssh/scp anyway.

[graeme_p]

Sorry, I did not make that clear.

What the discussion on the rsync mailing list indicated was that sometimes you gain an advantage from rsync's ability to send only deltas etc. On a very fast network such as a LAN the overhead of doing that actually leads to worse performance, especially when dealing with compressed file formats.

This can even mean that copying an entire directory with scp can be faster than syncing it with rsync (for example if you have a directory of media files, most of which have changed). That surprised me.