Your BACKUP Solutions: Redundancy Obsessive!

[ipedro]

As photographers – many of us digital – we live with the threat that our entire life's work can be wiped out in a second. It's all there, sitting in a magnetic registration on a hard drive... Even redundant systems can fail.

Here's my obssessively redundant system:



Click the diagram for a hi-res close up.

- My Aperture library sits in full on my MacBook Pro's 320GB HDD.
- The entire contents of the MBP are automatically backed up to a 1TB Time Capsule
- A second Time Machine drive is stored offsite in a bank safety deposit box
- An Aperture Vault is stored in the same deposit box
- In addition, I send all my 4 and 5 star rated photos in full res to an online SmugMug account
- Separately, I archive my company's completed project dockets to an HDD connected to the Time Capsule.
- The project dockets HDD is backed up periodically to an FTP online space.

I stopped using optical media as backup because you only discover that your backup didn't burn properly when you lose your original files and go to the DVD for desperate hope that you didn't lose it all.

My iPhoto library (personal photos) is on a Mac Mini, backed up to the 1TB Time Capsule and these photos along with my iTunes library has "backups" held in my tv and iPod.

Please share your setup and any backup nightmare and "THANK GOD I BACKED UP" stories.

 

[AlexH]

Wow, that's, well, redundant. I suppose if it's your business, you do what you gotta do.

What's up with the Time Machine drive in the safety deposit box? Do you regularly make trips to the bank and switch out drives or what?

 

[gnd]

Archiving to DVD+R does work

I stopped using optical media as backup because you only discover that your backup didn't burn properly when you lose your original files and go to the DVD for desperate hope that you didn't lose it all.

It is possible to use optical media for archiving, but you have to do it the right way. Don't just burn and forget.

First thing is to choose the best type of media, DVD+R. Why DVD+R? Here is the reason.

You need to prepare your files for the burn and verification. I calculate md5 sums for all the files that go on a certain disc. Make sure the file with the md5 sums goes on the disc.

Then burn multiple copies of each disc (I burn 3) and verify all the files on each disc against the md5 sums. Make sure everything checks out on all discs. Now you can be sure you have good copies of each set of files.

If you also use good media then you can be sure you will be able to read all the files when you need them. If you're paranoid, store the discs with the same sets of files in different locations.

 

[ChrisA]

The rule of thumb is that a backup system should:

1. maintain at leat three copies of the data on at least three different physical media
2. maintain the dat in at least two geographically different locations

But in order to do this you will likely need more than three copies of the data. for example if you have backup drives A, B and C and then when you re-fresh A the first step is to wipe the old backup. At this point for 20 minutes or so you only have two copies. Guess when the most likely time for a failure is? During a backup. You need four copies and a rotation system in order to keep three good copies at all times. Same withthe off-site copy. If you take it home then you don't have an off-site, so you need to rotate the them so that you drop one off before you bring the old one back to be refreshed.

But if your backup software does not over wipe the old backup off the disk then you can maybe do with one less copy. Copies do not need to be on a hard drive. on-line services and DVDs can be used. Yes, I've used DVD but you can't depend on them to last, you have to re-burn the backup set periodically, say once a year. and then toss out the 4 or 5 year old discs.

I've written many times that I'm pretty sure that in 50 years there will very, very few 50 year old photos. Hardly anyone does backup s correctly, not correctly enough to servive 50 years

My opinion is that any system that implements the two rules above will work.

 

[ChrisA]

Then burn multiple copies of each disc (I burn 3) and verify all the files on each disc against the md5 sums. Make sure everything checks out on all discs. Now you can be sure you have good copies of each set of files..

I've used DVD also and agree it can be done right. But rather then burn all three copies at the same time, I made just one copy and placed it in my fire safe. Then a little later I made a second copy and put in it the safe and took the set in the safe to the office. Then I made a third set. The total amount of work involved was the same but in the end at least one backup set was "fresher" and had more current copies of the files.
After a while you have to git rid of the very old copies

I got tried of burning discs and went to hard drives. The last drive I bought was a 1TB for $150. Cheap enough that now I just buy drives and rotate them around

 

[ipedro]

What's up with the Time Machine drive in the safety deposit box? Do you regularly make trips to the bank and switch out drives or what?

Whenever I go to the bank, I have my MacBook Pro on me so I just go in to the safe for 10 minutes and update the Time Machine drive.

Money can't buy the peace of mind of knowing that you have all your data stored in a bank safe in addition to the online copies, the local Time Capsule and the original files

 

[ipedro]

I got tried of burning discs and went to hard drives. The last drive I bought was a 1TB for $150. Cheap enough that now I just buy drives and rotate them around

Exactly. Drives are so cheap these days that you can buy 3,4,5 drives and just use Time Machine and Aperture Vault. Unless your entire city gets swiped by some sort of electromagnetic wave and renders useless all the hard drives in your city, you know you'll have an updated copy of all your data somewhere.

The odds are absolutely in your favor. I like those odds.

 

[jodelli]

Depending on which particular folder, I have as many as three or four copies backed up to portable media, some also on web storage sites such as Flickr and Photobucket, and all on at least three separate HDDs.

 

[costabunny]

• I use RAID5 in my Mac
• I have a TimeCapsule (250GB) for daily backups
• I run my backup script which rsyncs to my RAID5 NAS
• Once a week I run the script with a remote target (a raided debian server I rent in Germany)

Whilst I do not earn money from my Mac, I do wholeheartedly beleive in a solid backup regime.

 

[AlexH]

Whenever I go to the bank, I have my MacBook Pro on me so I just go in to the safe for 10 minutes and update the Time Machine drive.

Money can't buy the peace of mind of knowing that you have all your data stored in a bank safe in addition to the online copies, the local Time Capsule and the original files

Very cool. That would be secure!

 

[duncanapple]

Question for those of you using multiple external hard drives for backing up via time machine... if I had two drives i was swapping out with time machine, and I wanted to have complete backups on both, how would it handle that?

For example, I assume I would plug backup drive A in, configure time machine to use that as the backup volume and then do a complete backup.

Then, (again, I assume) I plug backup drive B in, configure time machine to use that as a backup volume, and do a complete backup.

What happens then, when I plug drive A back in? Wouldn't time machine be looking for drive B?

And now lets say it sees both as backup volumes - lets say I have done one full backup on each. Then I upload a bunch of pics to my laptop and want to back up to both drives. So I plug in backup drive A, and it writes just the new photos to the existing backup. What happens when I plug in drive B? Will it think it already backed up those new photos, or will it know drive B doesnt have them written yet?

Sorry if the above sounds complicated - just trying to make sure my data is safe? Here I thought I was safe just using one backup drive - after reading this I wonder if I should buy a second drive, occasionally do backups to it as well, and leave it at my parents house.... Just not sure how time machine would handle that...

Thanks!

 

[66217]

I'm also quite paranoid about my photos, but not so much.

My photos reside in an external drive at home. At home I also have another external drive for Time Machine, were I backup the photos there as well.

I burn all my photos to DVDs and store them in a safe place. I have planned on doing this every 2 weeks for new photos, and every 6-12 months to replace all the DVDs.

Most photos (3 stars and above) are uploaded to MobileMe. So far I haven't been uploading the master files, but I think I'll start doing that.

And finally, all photos are stored in an extra hard drive which I keep at the office.

 

[ChrisA]

What happens then, when I plug drive A back in? Wouldn't time machine be looking for drive B?

And now lets say it sees both as backup volumes - lets say I have done one full backup on each. Then I upload a bunch of pics to my laptop and want to back up to both drives. So I plug in backup drive A, and it writes just the new photos to the existing backup. What happens when I plug in drive B? Will it think it already backed up those new photos, or will it know drive B doesnt have them written yet?

It's simple: ALL of the information about what has been backed up and when is kept on the Time machine backup drive. So when you switch drives from A to B TM "forgets" everything about what happened with drive A. It worksthe way you'd like it to.

The only trouble with rotating out TM drives is that those TM drives have to be a bit larger then all the data on your system. This could force you into using RAID as the TM drive. Then you'd need multiple RAID boxes

I keep just one TM drive connected at all times for hourly backups and then use Aperture's "vaults" for photo archive. I have several vaults and rotate them.

 

[duncanapple]

Thanks for the help! I think I need to get a second external HD and start storing it elsewhere (ie not at my house).

 

[akm3]

I thought that Time Machine ignored Aperture libraries? Are you *sure* you are getting your Aperture library backed up?


(I'm anal too, just want to make sure you are covered...I don't know much about Aperture, I'm still with iPhoto)

 

[MacLadybug]

Amazing setup iPedro... it's beautiful!

Here's my nightmare... I still cry when I think about it. First off, I'm not a photographer. Most of what is on my HDD can be replaced with some work with the exception of my family photos collected over many years. It was about 6 years ago and we had just purchased a brand new Dell uber computer custom built with top of the line specs, the best of everything at that time. It was ginormous. I had decided that I needed to backup photos to safe-guard them. The plan was to burn backup disks then I thought I'd be real intelligent and upload them to one of those sites like iFoto (can't remember which one now). Well, a real genius would have uploaded the photos first, but I decided to burn the disk first. I was using Roxio software to burn my photo collection. The disk drive malfunctioned and froze and I was forced to restart the computer. After the reboot, I could not find one photo. Everything was gone. Not on the disk, not on the hard drive. Gone. Gone all the family photos, our trip to Ireland, our trip to England... gone. EVERYTHING. I screamed and cried... called tech support and looked all over the computer, it was gone. I even called in a reputable Computer repair company and paid them too much money to scan my HDD for the photos. Nothing. If only I had uploaded the pics first.

We've been 100% converted to Mac for 1 year and 7 months now, but who's counting. My photos are on my iMac, my new alum MB, classic iPod, iPhone and backed up on my TimeCapsule. Additionally, just for good measure I do have them burned to disks and on a LaCie 500GB hard drive. A bit redundant, but oh well.

 

[anubis]

Yeah, I put backup DVDs of the weddings I shoot in a safe deposit box. No point in backing up the data to multiple drives in my house if the house burns down.

My primary backup mechanism is Smugmug. Smugmug allows for UNLIMITED photo storage and transfer, and stores photos in the full uploaded resolution, so you get a built-in unlimited image backup solution with your standard Smugmug account. Smugmug advertises that when you upload a photo, it gets copied and stored in 3 data centers in 3 different geographic locations.

 

[dllavaneras]

My external 250Gb HD died a month ago. Now I'm getting two external 1Tb drives (one for files, the other for backup) and keeping an external backup HD at a friend's house. No sense in losing years of hard work.

 

[elcid]

How do you not own a Drobo?

 

[dllavaneras]

How do you not own a Drobo?

I hadn't heard about it until after I read your post, and I'm already stretching my budget as it is. Besides, you can't get it in my country, and even if I could find a store that gets them, it'd cost a kidney. Just for reference, a Canon XTi costs 500 bucks in the US, and down here it costs 2350 bucks.

If I get a Drobo in the future, can I set two internal drives as data drives and the other two as backup drives? If the controller automatically mirrors the content of the drives, then I'd really think about one.

 

[Digital Skunk]

The rule of thumb is that a backup system should:

1. maintain at leat three copies of the data on at least three different physical media
2. maintain the dat in at least two geographically different locations

This is pretty much what I do and that's that.

I don't carry my entire library on my MBP, just my working projects, and have everything in triplicate on the home tower and LaCie drives.

Did I mention photo and HD video? I am looking for an Apple Xserve RAID just incase anyone has seen one lying around.

p.s. Yes, Time Machine can't backup the Aperture Library file. You have to back that up separate. That's the main reason I still reference my files from the tower and have the 100+ GB of archived photos and active images backed up using Time Machine. The Aperture library then is only a few GBs in size and can be tossed on a few smaller drives.... hence my need for one large array of drives.

 

[snberk103]

I know some photographers who periodically (monthly or so) make a backup copy of their image library to an external HD, and then courier it their friend who lives on different continent (there are 4 or 5 friends who live in North America, Europe, and Asia). I don't know how they schedule it among it themselves, but their thinking is that even if the worst calamity - short of a global catastrophe - happened, a copy of their images would exist somewhere.

I don't tell them that if they actually needed to go to their off-shore copy - - then the people who would normally be buying their images probably are having their own problems.

Cheers

 

[ipedro]

How do you not own a Drobo?

A Drobo or Drobo-like solution is coming. I did say redudancy Obssessive . I'll never be fully relaxed, but I think my data is relatively safe at this point.

 

[ipedro]

Here's my nightmare... I still cry when I think about it. First off, I'm not a photographer. Most of what is on my HDD can be replaced with some work with the exception of my family photos collected over many years. It was about 6 years ago and we had just purchased a brand new Dell uber computer custom built with top of the line specs, the best of everything at that time. It was ginormous. I had decided that I needed to backup photos to safe-guard them. The plan was to burn backup disks then I thought I'd be real intelligent and upload them to one of those sites like iFoto (can't remember which one now). Well, a real genius would have uploaded the photos first, but I decided to burn the disk first. I was using Roxio software to burn my photo collection. The disk drive malfunctioned and froze and I was forced to restart the computer. After the reboot, I could not find one photo. Everything was gone. Not on the disk, not on the hard drive. Gone. Gone all the family photos, our trip to Ireland, our trip to England... gone. EVERYTHING. I screamed and cried... called tech support and looked all over the computer, it was gone. I even called in a reputable Computer repair company and paid them too much money to scan my HDD for the photos. Nothing. If only I had uploaded the pics first.

We've been 100% converted to Mac for 1 year and 7 months now, but who's counting. My photos are on my iMac, my new alum MB, classic iPod, iPhone and backed up on my TimeCapsule. Additionally, just for good measure I do have them burned to disks and on a LaCie 500GB hard drive. A bit redundant, but oh well.

That's horrible my friend... horrible.

What prompted me were two experiences.

The first: all the data that I had been safekeeping since I owned my first computer was passed on from computer to computer. One day, I had a problem with my then Windows system, and I had to do the familiar format/reinstall routine.

I popped in a new HDD and formatted it to be ready for a fresh Windows install. I formatted the wrong drive! I cleaned out all my life's worth of data.

... fortunately for me, after nearly crying, kicking holes in the walls and shouting "NOOOOOOOOOOOOOOOO!" until my neighbors knocked on the door thinking something tragic had happened, I managed to keep a relatively cool head and go through reversing the process. Back then, I had just that one computer so I couldn't look up information on how to revert to my HDD pre-format. Luckily, I had been impatient and hadn't done an exhaustive format and was able to recover everything.

I learned from that and from then on, I copied important data to DVDs on a (not regular enough) monthly, schedule.

My second shock into action came when my PowerBook, with all my data on it, got stolen. Fortunately, this happened during a trip, so I had performed one of my monthly backups just prior to leaving home. I recovered everything except for the earliest photos from that trip. Had it happened during any other time, I'd be one month behind on data.

That's when I came up with the system that prompted this thread.

I can breath pretty easy knowing that I have my data in the bank, online in a cloud somewhere (SmugMug), on a Time Capsule at home and on external HDDs in my home office.

 

[ChrisA]

Here's my nightmare... ...Everything was gone. Not on the disk, not on the hard drive. Gone. Gone all the family photos, our trip to Ireland, our trip to England... gone. EVERYTHING. I screamed and cried... called tech support and looked all over the computer, it was gone. I even called in a reputable Computer repair company and paid them too much money to scan my HDD for the photos. Nothing. If only I had uploaded the pics first.

We've been 100% converted to Mac for 1 year and 7 months now, but who's counting. My photos are on my iMac, my new alum MB, classic iPod, iPhone and backed up on my TimeCapsule. Additionally, just for good measure I do have them burned to disks and on a LaCie 500GB hard drive. A bit redundant, but oh well.

You problem was not that you didn't do the upload first, it was in your workflow. It one time the files were on a camera memory card. You need a workflow that backs up data. For example you copy them to (say) iPhoto, then let Time Machine back that up and then at the time time burn a CD and ONLY them with three copies of the files in hand would you erase the memory card.

The rule is (2) Three copies of the data on three media and (2) two geographical locations.

You current system is NOT very redundant. Not if all the data is in one building. At the very least buy a fire safe.