homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Ecommerce
Forum Library, Charter, Moderators: buckworks

Ecommerce Forum

This 41 message thread spans 2 pages: 41 ( [1] 2 > >     
Back-up strategy?
How you avoid disaster of lost data......

 11:29 am on Sep 29, 2003 (gmt 0)


We run an online store and I had a terrible week last week when due to Windows errors, re-installs etc I lost a lot of files. Have managed by using a data recovery tool to recover a lot of files but was a big wake up call for me in terms of having a back-up strategy/procedure in place ( I had nothing!)

I now plan to:

1. Upgrade to WindowsXP to utilise restore/backup features
2. Buy a big zip drive to back up hard drive files
3. Download backup files to off-site data storage facility

Anyone got any advice re backups - what do you do? Any backup management software worth investing in? Easiest/cost effective?

Thanks for any advice



 12:35 pm on Sep 29, 2003 (gmt 0)

We just finished getting our backup server running-- it's just a cheap server (a mini-ITX, actually-- we paid extra to get a cute one we could hide in a table lamp, so it's not the cheapest server on the market) and my programmer's writing a script to automatically back up all files on both of our computers every night. It's got a 160 GB drive, which ought to suffice for now.
I don't think it needs to be a terribly complex solution. And I don't have an off-site data storage solution worked out, so if the worst should happen and my home be damaged, I'm out of luck. So that's a good idea, sure. I might periodically burn the irreplaceable stuff to DVD-R and stick it in my desk drawer at work, but that's hardly professional.


 12:47 pm on Sep 29, 2003 (gmt 0)

I run Mac OS X, which is based on a variant of FreeBSD Unix. It is ultra stable and pretty secure too. My current uptime is 54 days.

I make my sites as small as possible. This means that I can do a batch download from the server to my home computer, an iMac, and keep a clone of everything there. I do a print of customer orders, without credit card details, so that I always have that in black and white. Everything else is stored on 2CO / PayPal / WorldPay whatever.

Backing up really isn't too much of any issue, as long as you can be bothered to do it. But if something goes wrong, you;ll appreciate it. I used to design on a Win box, but we took a direct lightning hit on our local substation, which in turn fried the whole comp. Might also be an idea to get one of those surge protectors, just to help keep things ticking over smoothly.


 1:27 pm on Sep 29, 2003 (gmt 0)

I have had TRAGIC events when it comes to backing up (lost 6 months of work, twice).

Here is what I learned:

1) Don't trust any backup program, script, schedule. Check it all the time, have it send you reports, make sure the files are there, test test test.

2) A backup is only as good as where it is kept. If you keep your backup on the same computer (different drive) it can be destroyed by a system malfunction (file system), a fire, theft, etc. Make sure to get an automatic rotation that is offsite (not in the same building/town).

3) Have multiple layers of backing up. Back up hourly, nightly, daily, weekly, monthly to different locations and different levels. Don't just back up the data but backup config files as well.

So, after all my loss, this what I am doing to make sure it never happens again:

- We run a linux server on RAID 1 (SCSI)
- We backup nightly (important files) to a EIDE drive
- We backup weekly to a tape drive which is swapped out monthly to an offsite location.
- I pull down copies monthly off of the server (another state) a complete set of files on my local workstation.

This protects me from:
- 1 drive in the RAID dying, no loss of data
- The EIDE drive dies, RAID still up, no loss of data
- The RAID array dies, I have lost up to my last nightly backup
- The RAID array and EIDE drive dies, I have lost up to the most weekly back up on tape
- The RAID array, EIDE, tape drive w/tape, I have lost up to the first of the month back up on tape offsite or local copy I pulled down from server.

Backing up is really about what you are willing to lose, how much and how much you are willing to pay to make sure you lose nothing.



 2:19 pm on Sep 29, 2003 (gmt 0)

><I run Mac OS X, which is based on a variant of FreeBSD Unix. It is ultra stable and pretty secure too. My current uptime is 54 days.

Me too. Which means my macho programmer boyfriend, a longtime Mac and Unix guy, can write the backup script himself.

And you're right-- it's about balancing how much you're willing to pay with how much you're willing to lose. I'm not willing to pay much, and I'm willing to lose a lot. I'm just not willing to lose it all. So I've only got one and maybe two backups (three-- the most important stuff is on a laptop too.)
But that's because I don't have a whole lot financially riding on this data, it's just important to me.
Once my freelance business takes off and i rely on my personal computer for more of my income, I'm going to start implementing off-site backup options.


 3:47 pm on Sep 29, 2003 (gmt 0)

I'm running a linux backup server that backs up my development files (client and personal) every 30 minutes. Total cost of the server was $300. Backup code is a script which runs on a cron job. The last back up of the day is saved to a daily directory and then the 30 minute logs are wiped clean and the routine starts again. I also archive a monthly in the same fashion. Yearly's are burned to a CD as are all project files when they are completed. I track the project start, end and archive dates on a spreadsheet.


 3:52 pm on Sep 29, 2003 (gmt 0)

I have a co located server and a server in my office. Our site does not change much so I make a backup every time I make changes. This is also important becasue I use a failover DNS service. It poles my server every 2 minutes for a file named working.txt to make sure the file is there and has the word working int he text file. If it does not find it the DNS fails over to my backup server at the office. I also copy it to a USB flash card that I keep on my keychain plus a tape backup.


 5:02 pm on Sep 29, 2003 (gmt 0)

>a USB flash card that I keep on my keychain

I like that. Very spy-movie. Nice touch.

Lorax, I only wish I were as organized as you are. :D


 5:10 pm on Sep 29, 2003 (gmt 0)

I back up to a CD-RW when I think I need to. I wish I was more disciplined about it. I worry about human mistakes more than hardware failure though. I make backup of files before I work on them... that's the extent of my backup plan. ;)


 5:18 pm on Sep 29, 2003 (gmt 0)

BwanaZulia is spot on.

May I add

1. Assume that everything CAN go wrong at the same time i.e. your PC stops working at the same time as your hard disk goes kaput at the same time as your backup CDs/tape get lost.

2. ALWAYS TEST YOUR BACKUP SOFTWARE. Get a spare hard disk if necessary. Try restoring your backup to the new hard disk. Does everything work like you thought it would?

3. Keep copies of original install CDs/software for all the programs you use - just in case you need to reinstall the OS.

4. Keep backups in a variety of formats. Do a tape/DAT backup. Get a spare hard disk and stick it into a caddy so you can regularly ghost your main disk. Keep downloading crucial data to CDs, they're cheap enough.

5. Call me paranoid but I ALSO have a copy of my hardware. Everything that is in my PC is duplicated in another PC that's in a cupboard. If I have a major system failure I can plug my ghost backup disk in the spare PC and be up and running within 10 minutes. (Bl**dy Windows XP does make this bit difficult)


 5:50 pm on Sep 29, 2003 (gmt 0)

2. ALWAYS TEST YOUR BACKUP SOFTWARE. Get a spare hard disk if necessary. Try restoring your backup to the new hard disk. Does everything work like you thought it would?

It is also just as important to know that you can restore. Do an annual test of the restore procedure. Probably restoring to another PC is best and test the results. It is no good doing backups and then blowing the restore because you do not know how.

There is a story a couple of years old that was going around town about the operator that blew three generations of backup because the drive was chewing up the backup tapes. Not good.

Good luck,


 5:53 pm on Sep 29, 2003 (gmt 0)

>> Lorax, I only wish I were as organized as you are.

Hah. I wasn't always this way. One too many times of losing data and having to rewrite/redevelop what I lost. Once you've experienced a few hard lessons you'll find yourself more driven to protect what you build. ;)


 6:09 pm on Sep 29, 2003 (gmt 0)

The only reason I'm in this thread in the first place is that I lost a hard drive a couple months ago.


 6:14 pm on Sep 29, 2003 (gmt 0)

Re: lost hard drive - *oog*

Been there, done that, paid in sweat and blood. Never again. :)


 6:19 pm on Sep 29, 2003 (gmt 0)

So, does anyone have an opinion about running a mirrored RAID setup on your workstation? Two drives with the exact data so if one of the hard drives go bad, you can replace it and be up and running quick. You'd still have to do backups for human errors though. Only if I had the gift of time travel... ;)


 6:31 pm on Sep 29, 2003 (gmt 0)

>> running a mirrored RAID setup

Can be very costly to set up so I don't use it. But I have thought about it for the simple reasons you mention.


 6:35 pm on Sep 29, 2003 (gmt 0)

Can be very costly to set up so I don't use it. But I have thought about it for the simple reasons you mention.

Have another look. I think you'll find it has come down considerably. If your motherboard doesn't have support for RAID1 then buy a RAID PCI card - less than 30. Then all you'll need is an extra hard disk. Again, very cheap now especially if it's IDE/SATA. You could do the whole lot for less than 100.

Mileage may vary of course, and if you're using server equipment and SCSI drives it's more expensive. Still, check latest prices and you may be surprised.


 6:36 pm on Sep 29, 2003 (gmt 0)

>lost hard drive

I didn't lose anything I was actively making money off of.

But I lost about 7 years of archived email correspondence. Letters to my first boyfriend, the email addresses of my old highschool buddies, my weekly missives to my mother, etc. It was like losing a diary and scrapbook all in one.

But, it didn't cost me any *money*... Still.

I don't know about mirrored setups on one PC. I'd rather have my backup separate-- if I drop the computer or something dumb like that, which is far more like me than anything else... If I had a bigger apartment, I'd have them in separate rooms. The most logical solution is to have them in different cities but that's more elaborate than I'm prepared to get.

I don't know, I just don't like the idea of having it all in one computer. I guess it'd be convenient if one failed-- you could switch to the other fairly seamlessly...


 7:18 pm on Sep 29, 2003 (gmt 0)

The most logical solution is to have them in different cities

There are online services where you could upload one copy :-)

Also, twist your hosting company's arm and they may give you enough storage on your own site to hold a backup of your hard disk ;-)


 7:26 pm on Sep 29, 2003 (gmt 0)

>your hosting company

I host my own.

Good solution, though. Might look into that anyway.


 7:33 pm on Sep 29, 2003 (gmt 0)

I don't really do backups very often. With email I collect off the server on several different comps, but only use the main workstation, my eMac, to actually remove the email from the server. This means I have copies of email across various individual boxes.

With the websites, I keep 'em small. Download them all, stick them onto a usb thumb drive. Then I have a backup, but can also work from the last version of the site on my laptop, which is handy. :-)

With all this talk of RAID's and replica boxes there must be some fairly sizeable operations going on?


 3:34 am on Sep 30, 2003 (gmt 0)

Yeah, I use a usb thumb drive. Wonderful gizmo. Big problem is that they're easy to lose.

There are watches now that do the same thing. Carry your website on your wrist! Then you gotta worry some hold up guy's gonna steal your watch and website backups.


 9:04 am on Sep 30, 2003 (gmt 0)

With all this talk of RAID's and replica boxes there must be some fairly sizeable operations going on?

Exactly the type of myth I've been trying to dispel :-)

RAID is very affordable now, not restricted to big operations, and you'd be surprised at how many ordinary home PCs have RAID.

Another piece of advice I forgot before:
If your hard disk does get messed up DON'T attempt advanced recovery techniques yourself unless you are very confident you know what you are doing. If you are going to be calling recovery experts - doing it earlier rather than later gives you a better chance of getting your data back.


 11:34 pm on Sep 30, 2003 (gmt 0)

Just bought a USB 120GB drive that backs up my laptop every night and our main office computer backs up via an online service every night to somewhere far, far away...


 12:47 am on Oct 1, 2003 (gmt 0)

Thumbs up on Toolman's thread. I do two things for my local workstation: (1) nightly backups to an external drive using Retrospect, and (2) weekly clones of my hard drive using CarbonCopyCloner (OS X). One thing I'm not doing that I should is backing up to a removable media and storing it offsite.

The reason I clone is because of that damn Murphy's law. Data loses happen, when? When you're up to your ears in work and trying to meet deadlines. You can spend half a day or more recovering... with the clone at least you can get back to work and deal with the recovery when it's more convenient.


 2:40 am on Oct 1, 2003 (gmt 0)

Man, we gotta lot of Mac geeks in here.


 2:54 am on Oct 1, 2003 (gmt 0)

Why get your back up? Just relax, okay?


 6:04 am on Oct 1, 2003 (gmt 0)

Having been burnt by a host going out of business on me, I'm a lot more fanatical about backups now.

I have a copy of the site's pages on all three of my workstation's HD's. On the rare occasions that I need to make changes, I make sure they get made to each copy. I also have an archive of the site's pages on a CDR.

Other data from the site that doesn't change much (configuration files, etc) are stored on one HD, and burned to a CDR every time they change (which isn't often at all.)

A weekly snapshot of all user data is downloaded to a HD, burned to a CDR, and transferred to a separate server in a different part of the country weekly.

All the user databases, password files, and email are backed up daily to a HD on my workstation.

It seems complicated, but it's not. There's little point in downloading my HTML pages -every single night- when they don't change. Ditto previous month's logs, config files, and the like. Everything really important gets backup up daily.

It's a little trickier now that I'm running three servers (and soon to be four), but 99% of the content of these machines changes less than once a week. (I have a VDS just to run webmail applications; the only real data on there is the webmail programs, which don't change except after upgrades, and user preference files... everything else stays on the mailserver.)


 6:05 am on Oct 1, 2003 (gmt 0)

When I was new to web development I used to shy away from database-driven sites because I thought it would be difficult to maintain a backup of the database.

Then I learnt about MySQL's backup function, which makes it a breeze. I use a cron script to backup the tables from a MySQL databse at 0430am every day. It saves every day's backup into a new folder on the server's hard drive, and the folder's name is the current system date and time. Every week (ideally) I download the whole folder to my hard disk.


 6:42 am on Oct 1, 2003 (gmt 0)

A second hard disk in the same box is a no-no for backup, raid or not. My wife turned off her box. When she came back home the power box had melted and destroyed both disks (as well as motherboard and tuti quanti). Not talking about theft and fire.

This 41 message thread spans 2 pages: 41 ( [1] 2 > >
Global Options:
 top home search open messages active posts  

Home / Forums Index / WebmasterWorld / Ecommerce
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved