homepage Welcome to WebmasterWorld Guest from 54.211.47.170
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Hardware and OS Related Technologies / Linux, Unix, and *nix like Operating Systems
Forum Library, Charter, Moderators: bakedjake

Linux, Unix, and *nix like Operating Systems Forum

    
Backup server from A to B. Sync backups.
I believe I'm almost there but how to deal with emails?
explorador




msg:4589598
 2:55 pm on Jul 2, 2013 (gmt 0)

Hi there was unsure on where to post this, the right section I mean.

I always have at least two servers, sometimes 3. The purpose is to keep sites apart and avoid having all the websites down when problems occur. Been using one server for backup purposes and it's proven to be very useful and critical. My approach has been making backups via CPANEL from server A to server B and transferring the backup via FTP, Cpanel allows you this, it's built in.

The problem is, many times I don't need to expand the complete backup (lots of gigs) to get what I want. And MOST TIMES what we need as webmasters is a magic switch to "turn off" server A and turn on server B sort of speaking. So I've been using tar files per site or group of sites, then I easily expand them on the other server. It's fast, easy. I can restore fully functional websites (php, perl, html, etc) with no problem and have it running fast, all it takes is the Domain DNS update.

Files? easy. The compressed file keeps the permissions and structure.
Databases? I created my own CMS doing wonders, and it doesn't use Mysql, it works amazingly well for me with no traffic problems, it was built from zero to grow. BUT anyway if you use databases, or when I have Wordpress projects hosted, I dump/export the databases, solved, easy to add to the automated script.

But what about email? mailboxes?. Is there anyway I could export any config regarding emails to just import-expand on server B? I want to avoid any Cpanel functions so I can automate this thing. I'm almost there, one automated script will do the backup and then will call another script on server B to begin the transfer. Easy. But still no clue on how to deal with email.

I could create all the mailboxes on both servers right? nope, if the clients change their password then will have problems accessing their email when the switch is completed. And also want to avoid double creating each email account.

Both servers are in diff hosting companies. Don't put your all your eggs on the same basket, if you work like that think twice! there is nothing more important than keeping updated backups and to have them in safe places. It will save you lots of work and pain.

Any ideas?

 

graeme_p




msg:4590364
 7:22 pm on Jul 4, 2013 (gmt 0)

No idea about your mail problem (I suspect its going to be tough, especially if the hosts use different mail server software), but I wondered why you transfer tar files rather than just rsync?

brotherhood of LAN




msg:4590374
 8:09 pm on Jul 4, 2013 (gmt 0)

Was going to say rsync, I believe i supports some 'diff' magic so you don't have to send your entire copy.

graeme_p




msg:4590595
 4:20 pm on Jul 5, 2013 (gmt 0)

Yes, rsync is very fast and efficient. It only actually copies the changed sections of files that have changed.

It still does not solve his email problem though!

One thing to try may be using rsync on the cpanel backup. It may be a lot faster than FTPing it.

brotherhood of LAN




msg:4590629
 8:07 pm on Jul 5, 2013 (gmt 0)

I guess it depends which email software you're using and how it's storing the data. I use postfix/dovecot, users & passwords are stored in one file, and all mailboxes are just regular folders and text files.

Maybe in a case of a meltdown, you could just keep the usernames and force a password reset, and just focus on keeping the mailboxes synced?

Demaestro




msg:4590638
 9:01 pm on Jul 5, 2013 (gmt 0)

Do you have physical access to the servers?

You can get these hard drive replication units and hook it up to the server with a sata cable.

What it does is you take your drive out of the server and mount it in this unit, then you mount a 2nd drive into the backup slot. You connect the unit back to the PC with a sata cable and as data is written to your drive it gets written to the backup as well. If the main one dies you simply pull the backup and swap it for the dead drive.

I have seen some with 4 slots for extra redundancy. The advantage here is the replication hardware has the software built in so no processing power or bandwidth is used by your server to do that backup.

This all requires you can access the servers though...

If you can't set that up something I have seen done is you set up an SVN repository somewhere. Then create a bash script that does SVN-ADD, then SVN-COMMIT on the mail folders. That will push new files and versions to your SVN. Point a cron task at that script and have it run nightly or whatever time interval you wish.

Not the most elegant solution but it works and isn't hard to setup.

rfs9999




msg:4591266
 8:22 pm on Jul 8, 2013 (gmt 0)

>But what about email? mailboxes?.

There are tools for synchronizing e-mail mailboxes between two servers. (I'm the author of one such product).

Alternately you can use an IMAP tool to dump all of a user's messages on one server to files and then upload them to the other server.

> if the clients change their password then will have problems accessing their email when the switch is completed.

Many mail servers have an administrator account which can be used to access a user's mailbox. You supply the username, admin name, and admin password and you are logged in as that user. (IMAP calls this
AUTHENTICATION=PLAIN).

cPanel uses the Dovecot IMAP server which does support admin access
through the use of what they call a "master" account. You can create one master which can access all accounts or you can limit it to certain users only. It's actually pretty easy to set up.

If you manage the cPanel hosting server then you can set this up yourself; if not you may be able to persuade the administrator to create a master account for your users.

Regards,
Rick

explorador




msg:4591268
 8:46 pm on Jul 8, 2013 (gmt 0)

grame: No idea about your mail problem (I suspect its going to be tough, especially if the hosts use different mail server software), but I wondered why you transfer tar files rather than just rsync?

The thing is being able of switching servers and just updating the DNS, all of that is under control now except mails because the mailboxes must exist on both servers and have the same password so the client won't have to change anything.

Why tar files? personal choice regarding redundancy. I can have the whole package on any server for quick restore and also my set up sends split files of that tar to a big email account. My sites are performing quite well and are highly optimized to save space. But yes I'm thinking on it, I'm expanding my knowledge as I was not too familiar with it. Still I prefer solutions that run out of the box, why? you know how some hosting providers can mod things and then your setup stops working as "oh yes we had to disable rsync because one of the clients..." I even had experiences with hosting disable something as simple as mkdir with custom permissions. Yes it's easy to think "well change of hosting company" the thing is, it could happen with no prior notice.

brotherhood of LAN: I guess it depends which email software you're using and how it's storing the data. I use postfix/dovecot, users & passwords are stored in one file, and all mailboxes are just regular folders and text files.
Well it's Cent Os, I'm researching on it because yes, there are files and folders accessible to me.

Demaestro: Do you have physical access to the servers?
Nope, they are on another country(ies)

rfs9999, thanks, sounds interesting, I will research around that.

Thanks everyone.

PS. Over the years I learned not to fully trust what the hosting company says, they can often turn on and off diff things on the server with no prior notice, specially on shared servers (even out of shared servers there is no guaranty). While I agree on complaining I'm a practical man so I put my efforts on solutions because that's what keeps the boat floating. Since them I've been moving the files while others fill the support forums asking why this why that.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Hardware and OS Related Technologies / Linux, Unix, and *nix like Operating Systems
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved