Backups!

How would it feel to lose your entire digital portfolio?
I got to feel a taste of that on Sunday. After a Windows blue-screen-of-death, which had been more frequent lately, I rebooted and was told that my RAID-1 mirrored array had a critical failure. What? ARRGH! After much troubleshooting and testing, and being unable to rebuild the array with the disk that had supposedly failed, I bought a replacement drive. Went to rebuild the array with the new drive and the controller reported there was no array! Again, swapping drives, checking cables, changing connections, everything I could think of - no joy. Looks like I lost the entire mirrored array.

Being a computer professional and having built and maintained all of my systems for the past couple of decades, I'm confident in my technical abilities and (I thought) realistic about the fallibility of hardware. I pretty much trusted a mirrored raid with my image library and lightroom database. Thankfully, I recently built an external drive and have been backing up to that. So I didn't lose everything; just some working files and other stuff that I'll remember when I need it.

So I think I'll take some time to make sure I have DVDs of my raw images and master files. I think I'm also going to look at the Drobo or other external NAS/RAID box. You can't have enough levels of backup. Especially when it comes to your images.

This post isn't for pity. I'm sure many of us are confident that our RAID, external drive, or other strategy will keep our data safe. Drives fail. Chips fail. Power supplies fry motherboards. Have a redundant backup plan.

Yikes! Sorry about your ordeal. I'm glad you didn't lose too much. I recently had a similar scare the other day when my boot drive failed. Luckily the drive with all my pictures on it was safe and once I rebuilt my OS drive, all was there, but it put the fear of God into me so now I have an external drive to mirror the internal photo drive.

Wow - that is really scary Kalin! Glad you had most of it backed up. On the Drobo, I wish they would release an eSATA version. You would think that they should release a 2.0 version with this capability soon - especially given the price point.

Just a fyi and maybe you had already been doing this but a better practice is to have 3 drives. 1 for the OS and two for the mirror. On the mirror keep all your photos, image library, and databases on it. On the single drive install the OS. One is that this will really speed up your system as the OS doesn't have to write to two drives for every thing it does and two there is a much greater chance that you won't corrupt your array. With all the reads and writes from the OS and swapping that takes place with the OS there is always a chance that you can corrupt both drives in a mirror configuration. If the OS crashes or the single drive dies you just put a new drive in or rebuild your OS and the array is still there containing all your data.

This is what I have been doing for years along with backing up new files to DVDs every so often and putting them in a safe. Hope this helps and glad you didn't loss everything.

Brian

I agree with Brian about the 3 drive configuration, but I set up my system a little differently.

My desktop has a windows OS drive and a single drive (not mirrored) for documents and images. My NAS has mirrored drives. Keeping the OS separate makes it easier if it has to be restored. With this setup, if I add a new computer to the network or need to rebuild a computer that has crashed, I only need to copy the image catalog from the NAS and I'm up and ready. If the NAS looses an array, I still have another copy on the desktop and can reverse the process to refresh the array.

I think it all gets you to the same point.

In the computer being discussed, I actually have six drives:
The first drive is the OS drive and contains four boot partitians with different OSs (main, win64, winXP-Games, winXP-Video). I maintain different OS partitians to isolate drivers and finiky applications like games and video processors.
The second drive is a 200Gb drive for swap space, and scratch space for video editing, etc.
The third and fourth drives are 320Gb drives in a RAID-0 partitianed into various partitians for different data
Then drives five and six were 250GB drives in RAID-1 with critical data (protected, I thought).

I completely agree about keeping the OS separated from data. I rebuild OS drives 1-2 times a year when they start getting flaky. I've had pretty good luck with this and similar configs. Still, if a data drive fails that can be a pain.

Thanks for the support, guys!

I've got a RAID 1 set up and I back that up to an external hard drive weekly. I keep the external HD at work so if my house burns down, I still have a copy of all my RAW, TIFF and JPEG images.

Sorry to hear about this, Kalin. At least you were wise enough to back everything up.

computers........
I have always had a love/hate relationship with them from the very beginning

For $5 a month, mozy (www.mozy.com) will automatically upload an unlimited amount of data from your computer. I have 150GB of data uploaded there and can access it from any computer. I used to worry about external back-up drives and the like but don't any longer.

Kalin, sorry to hear about all your trouble. I have heard similar things about various RAID cards and RAID options on motherboards. Enough in fact, that I don't I feel safe using any type of RAID on a home PC. We use RAID (hardware and software) all the time at work, but these systems are built to handle it, use SCSI or SAS drives and are multi $10K servers or software, so they better work and so far always have (13 years and counting).

It seems the home PC versions of this stuff cut too many corners and just don't work well enough to put your hyper critical data on it. You might want to think twice before using the RAID-1 config again and use a Syncing program instead to keep the two drives in sync once an hour or so instead.

But thanks for brining this up for discussion. As you say, you can never be too diligent about backing up your data.

tell you what,
I lost all the raw files from our trip to Paris and a small town in germany, Einbeck. I relied on the DVD back and the DVD on which I wrote these was corrupt.
I guess it is time for redundant backups for me.

I feel sorry for your loss.

Thanks again, guys. I'll check out mozy, Brett. I've used online storage a little before but adequate upload bandwidth was always limited. Maybe it's more viable now.

Wow, that's terrible, Kalin! Fortunately, you didn't lose everything!
I only use two 500GB external hard drives, one is a mirror of the other. I also occasionally burn CDs for images that I use for enlargements. I hope this is enough, because the RAID concept just confuses me and doesn't seem all that safe, in that a corrupt file can corrupt the RAID array very easily.

Kalin,

Earlier this year I had a crash on a 250GB mirrored setup over here and wasnt able to remount the mirror and wound up losing 30-40 drum scans! Painful stuff!

RichieV

I use external hard drives and also make archival DVD backups at the end of each month which I send out of state. After hearing about Prashan's DVD problems I'm getting a little nervous. I might have to consider mozy.com as an addition, but can't imagine how long it would take to upload terabytes of files.

FYI, mozy can be customized to run whenever you prefer (e.g., at night, all the time) and the upload bandwidth can be easily changed by adjusting a slider on the interface. I took about a month for my 150GB to upload and now mozy only uploads new files when they arrive on my computer. Erik, at full speed a terabyte would easily take several months, so in the meantime you'd absolutely want a different backup system. However, after those months are over you would have some serious peace of mind. Perhaps this solution is less palatable for those who shoot multiple gigs of DSLR pics each day? I shoot mostly 4x5 and the relatively small number of DSLR keepers I leave on my hard drive are always backed up within one day.

Erik,
I'm now backing up on external HDs (2 sets) and DVDs 2 sets. One of these is kept at home, the other in my office. This is for raw+PSDs. Once C1P starts working properly I'll move to DNG+PSDs.

I think external HDs are the way to go for backups. When you calculate cost and capacity - don't forget to factor in the time you spend burning DVDs and the fact that the DVDs will probably not be full - a couple of 500GB external disks start to look like a bargain. And then there is the gradual degradation of CDs and DVDs as they sit on the shelf.

Regarding online storage services like mozy.com, what happens to your data when they go out of business or are acquired by some mega-corporation?

Please tell me you broke the mirror before trying to rebuild the array? I am serious, no joking you have to break the mirror BEFORE TRYING TO REBUILD. I am a MCSE twice over so I know 1 or 2 things about computers. I am sorry if this is too late, but maybe it will help others in the future.

"I am a MCSE twice over so I know 1 or 2 things about computers."

That gave me a good chuckle.

Brendon,
Yep, it's too late. I followed the instructions from the controller to rebuild the array. The mirror was broken. Thanks.