File server backup strategy

[xraydoc]

I've got a little home file server - used for items shared in the household as well as for Time Machine backups of everyone's computers (me, wife, 3 kids).

The server is a 2006 Mac Pro with a few hard drives in it formatted as a RAID 0 stripe (needed the added capacity; not so much for speed). Now I'm looking for a backup strategy. It doesn't need to be archival like time machine is - simply a mirror of all the data a couple times per week would suffice.

Total volume of about 3.5 to 4.0 terabytes of data (Time Machine data of 6 computers plus a mirror of the main iPhoto and iTunes libraries and a few miscellaneous items).

Suggestions for a backup strategy? Carbon Copy Cloner to a large external drive? Any way to add an existing volume to a RAID 1 mirror without destroying the data that already exists?

Thoughts appreciated.

 

[DustinT]

Personally, I'd grab a Drobo and shove a few disks into it. Then just have your backup software deposit a copy onto it. If you prefer not to spend the money, you'll want a disk array with 4tb capacity. You could buy a single 4tb external drive, but I think you'd come up a little short on capacity. That means you'll need a combination of smaller drives. You could use anything with a Raid 1, 10 or 5 controller. Speed isn't really much of a factor since its a backup of the backup.

By the way, Raid 0 has worse data reliability than just a plain old hard drive. Its really a bad idea for this application. I know that having all the capacity in a single volume is convenient, but you put everything on that disk array at more than twice the risk it was at on a single disk. Next time, use raid 5 and add a third disk. You'll have it all on a single array and have reduced your risk of data loss instead of increasing it.

 

[xraydoc]

By the way, Raid 0 has worse data reliability than just a plain old hard drive. Its really a bad idea for this application. I know that having all the capacity in a single volume is convenient, but you put everything on that disk array at more than twice the risk it was at on a single disk. Next time, use raid 5 and add a third disk. You'll have it all on a single array and have reduced your risk of data loss instead of increasing it.

I know that RAID 0's failure potential is greater than a single drive's (multiply the failure potential times the number of drives in the array). In this situation I felt the single large volume was easier to manage - particularly since at least one of the machines already has a Time Machine image larger than a whole single drive in the server.

To do a RAID 5 would require either an external enclosure or a RAID PCIe card and rewiring the innards of the Mac Pro.

I probably will have to migrate the data to something like an external hardware RAID 5 or 0+1 to give me enough storage plus redundancy. In the meantime I was curious as to what redundancy options I had without a major reconfiguration.

Sigh. Money, money, money...

 

[cutterman]

I used a NAS for this purpose (Netgear ReadyNAS NV+). They use a proprietary RAID 5-like array and are available on Ebay for ~$300. Get 4 2GB drives and you will have plenty of capacity. They are not fast but serve a backup purpose quite well. I also used it for an AV media server.

FWIW, I just retired one of these for a mac mini/Pegasus setup. PM me if interested.

 

[mainstay]

Note that RAID systems are not backup systems.

They are protection in the form of drive failure, but offer no data redundancy or versioning.

I understand that your personal systems in the house have a first stage backup (to the Pro) but the Pro itself also contains unique information that does not get a second copy placed somewhere else.

A first stage backup for the Pro would be to clone this to an external drive, whether it be NAS, big USB / firewire drive, or another computer.

But what you REALLY need is a second stage backup... one that is OFFSITE.

Personally, I backup my office and my personal files using BackBlaze for $4 a month. 360 GB of business files, pictures, and yes, music, all backed up for less than my half double decaffeinated half-caf, with a twist of lemon.

You can also use CrashPlan for comparable pricing.

CrashPlan has the added bonus of having a computer to computer backup that is free.

So, if you were operating on the cheap, you could place a secondary computer with enough capacity at another location (the office, an outbuilding, your parents house, I don't know) and have your Pro backup to this computer routinely.

 

[DustinT]

I know that RAID 0's failure potential is greater than a single drive's (multiply the failure potential times the number of drives in the array). In this situation I felt the single large volume was easier to manage - particularly since at least one of the machines already has a Time Machine image larger than a whole single drive in the server.

To do a RAID 5 would require either an external enclosure or a RAID PCIe card and rewiring the innards of the Mac Pro.

I probably will have to migrate the data to something like an external hardware RAID 5 or 0+1 to give me enough storage plus redundancy. In the meantime I was curious as to what redundancy options I had without a major reconfiguration.

Sigh. Money, money, money...

This is mostly correct, but not entirely. You can do Raid 5 under OS X using the built-in softraid. This will not perform as well as a hardware assisted raid card but it doesn't require any reconfiguration of your MP. Just add a third drive and configure the array in Disk Utility.

That procedure will erase all the data on the drives in the array. So, you will need the use of a storage system with enough capacity to hold the data temporarily. Some users have been known to 'rent' the needed equipment from Best Buy for the duration of the project.

Finally, the risk of failure is actually a little worse than the number of drives doubled. You also have to consider that any software, controller, a\c power or emi problem that causes a mis-written bit to one of the drives will usually result in the complete loss of the array. If you've been running a raid 0 array for more than a couple of months and its still up you must be pretty lucky. I know I'm beating this point pretty hard but raid 0 is just such a bad idea in your case.

As was well stated in a previous post, raid is not a backup system. And raid 0 is never a good idea for a desktop user. The points Mainstay made about offsite backups are really valid and are worth some thinking. Unless the data you have is valueless, in which case the loss won't matter.

 

[xraydoc]

This is mostly correct, but not entirely. You can do Raid 5 under OS X using the built-in softraid. This will not perform as well as a hardware assisted raid card but it doesn't require any reconfiguration of your MP. Just add a third drive and configure the array in Disk Utility.

That procedure will erase all the data on the drives in the array. So, you will need the use of a storage system with enough capacity to hold the data temporarily. Some users have been known to 'rent' the needed equipment from Best Buy for the duration of the project.

Finally, the risk of failure is actually a little worse than the number of drives doubled. You also have to consider that any software, controller, a\c power or emi problem that causes a mis-written bit to one of the drives will usually result in the complete loss of the array. If you've been running a raid 0 array for more than a couple of months and its still up you must be pretty lucky. I know I'm beating this point pretty hard but raid 0 is just such a bad idea in your case.

As was well stated in a previous post, raid is not a backup system. And raid 0 is never a good idea for a desktop user. The points Mainstay made about offsite backups are really valid and are worth some thinking. Unless the data you have is valueless, in which case the loss won't matter.

I know RAIDs aren't backups inherently. But I was backing up to the RAID 0 because I needed the single large volume and it was all I had available at the time. I do know it's volatile, but as far as I can tell Apple's Disk Utility doesn't do RAID 5 -- only 0 or 1. I don't believe SoftRAID will do RAID 5 either.

But... an update for those interested.

Decided to bite the bullet and spend some $$ earlier than I planned. Bought from OWC a Mercury Qx2 RAID enclosure, four 2TB hard drives and a PCIe eSATA card. I configured the Qx2 as a RAID 5 across the four drives (6TB formatted capacity) and installed it on the new eSATA channel. I would have used one of the two additional SATA interface posts on the Mac Pro's logic board but each already has a drive).

I disconnected the file server from the network and use Disk Utility to clone all the data from the internal array to the external RAID 5 box and made a couple of configuration changes to OS X Server to point the shares to the new RAID.

I then installed SuperDuper! and set it to 3 times per week at 1am do a smart update from the RAID 5 box to the internal array (still presently a stripe). That process, tested over the last couple of days, takes only around 15 minutes each time.

So all the data is now primarily stored on the external RAID 5 box, and three times per week duplicated on to the Mac Pro's internal stripe for a redundant backup -- even though 98% of the data on the file server is backup data to begin with.

I sleep better at night knowing I have at least some kind of backup of the backup.

At some point, mostly depending on when my funds are replenished, I'll look at some way of doing weekly or every-other-weekly off site backups and rotating out the media somehow. Internet-based solutions like Carbonite aren't really feasible for huge ~3TB data sets. My iPhoto database alone is pushing 90GB and iTunes is upwards of 1.3-1.4TB. Another matching OWC Mercury Qx2 box w/ 4x2TB drives to do a Mirrored RAID 5 would be really nice, but at seven bills it'll have to wait!

 

[DustinT]

Doc,

That sounds like a pretty sound strategy you've got. You should be quite safe.

 

[xraydoc]

Thanks.

I don't think I mentioned that this whole little expensive endeavor began because the Drobo FS I bought last year is too damn slow to be at all useful. I should have bought a Drobo with a Firewire port on it, but at the time I thought being able to stick the Drobo in the network closet would keep things nice and quiet in my office.

Well, the Drobo FS functions as designed, I suppose, but at best it'll file share at peaks of about 15MB/sec over gigabit ethernet. That's fine for Time Machine backups from my family's laptops but any kind of big transfer takes hours upon hours.

The Mac Pro file server on the other hand commonly hits peaks of 100MB/sec over gigabit. Makes a huge, huge difference when I want to back up a big project but then get back to work on it. To the Drobo FS it took hours until I could get back to work on it. To the Mac Pro server it takes minutes. Amazing.

Pic:

 

[DustinT]

Yes, I too have been a bit disappointed by the performance of the lower Drobos. I've got a Drobo Pro that Fedex is supposed to delivery any minute now that should be able to handle the same speeds as the Mac Pro you've got. Data backup is an expensive thing. But, I remind myself that I don't spend any money to develop film anymore. And that all my life's memories are stored digitally now and it doesn't seem so bad.

 

[xraydoc]

Yes, I too have been a bit disappointed by the performance of the lower Drobos. I've got a Drobo Pro that Fedex is supposed to delivery any minute now that should be able to handle the same speeds as the Mac Pro you've got. Data backup is an expensive thing. But, I remind myself that I don't spend any money to develop film anymore. And that all my life's memories are stored digitally now and it doesn't seem so bad.

Let me know what you think of it once you get it going.

 

[DustinT]

Let me know what you think of it once you get it going.

Well, I got it setup around 4 pm today and tossed in a collection of drives that I had hanging around. Honestly, this is spare and scavenged drives, no more than 2 are alike.

• 2 x 2tb
• 1 x 1.5 tb
• 3 x 1tb
• 1 x 500g
• 1 x 400g

You can imagine, some of those older drives really slow it down. I've done some testing with Aja's 16 g test. It writes 16 g and then reads the same data back. So, its a pretty decent benchmark. Results of the sustained transfers were high 70's to low 80's with peaks as high as 110. So, considering that I'm running a beta iSCSI setup on Lion with a craptastic load of drives, I'm pretty pleased.

So far I've moved several hundred gigabytes of data onto the system and it hasn't even hiccuped. I'd say it looks pretty good. I'll be interested to see if the performance improves once the release version of the iSCSI drivers are available and I swap out a couple of those older drives. I'll be looking for sustained speeds over 100 then.

Your Mac Pro setup is obviously more capable, however there's a couple of things I like about this setup. First is beyond-raid. You've owned a Drobo before, so you know what I'm talking about. Second is the power usage. My entire server closet is drawing 120 watts under load. Thats a 2011 Mac Mini, 8 bay Drobo Pro (with 8 drives in it), Airport Extreme and a couple of misc. items in the closet. If I had a working Mac Pro setup like you do, I'd have some motivation to switch to something like this. But, you're not going to gain a whole lot. I suspect the overall power usage would drop, and the beyond-raid is a really nice perk. But, at the present you'd see a slight drop in performance.

Sounds like a net zero game to me. What do you think?

 

[xraydoc]

Nice. I'm most impressed. Hopefully the '06 Mac Pro will make it a couple more years, but I'll almost certainly replace it with a Mac mini when necessary. It's by far the oldest computer I've still got in service.

I'll keep the Drobo Pro in mind for the future as well. Good to see that their upper-level products have good performance.

 

[DustinT]

Nice. I'm most impressed. Hopefully the '06 Mac Pro will make it a couple more years, but I'll almost certainly replace it with a Mac mini when necessary. It's by far the oldest computer I've still got in service.

I'll keep the Drobo Pro in mind for the future as well. Good to see that their upper-level products have good performance.

Those Mac Pros are built to such a high standard that I suspect it will be running long after its passed its useful life. Here's to hoping the Drobo Pro has half the quality the Mac Pro does.

 

[DustinT]

So, after the Drobo had the chance to sit overnight while I copied some stuff over it apparently finished some housekeeping it was doing. The results of the writes were about the same but the reads jumped up nicely to about 92 MB\s with a peak of 109. I'm attaching a couple of screen shots in case you'd like to see. Beta firmware and cheap disks be damned, this think smokes!

 

[xraydoc]

So, after the Drobo had the chance to sit overnight while I copied some stuff over it apparently finished some housekeeping it was doing. The results of the writes were about the same but the reads jumped up nicely to about 92 MB\s with a peak of 109. I'm attaching a couple of screen shots in case you'd like to see. Beta firmware and cheap disks be damned, this think smokes!

Very nice. Too bad the Drobo FS is such a dog.

 

[DustinT]

Very nice. Too bad the Drobo FS is such a dog.

Agreed! I was seriously considering one and then I ran into a deal on the Drobo Pro for the same money. Considering that the Pro is much more capable, I jumped on it.

 

[mwhities]

Agreed! I was seriously considering one and then I ran into a deal on the Drobo Pro for the same money. Considering that the Pro is much more capable, I jumped on it.

I've been looking into the Drobo FS as well. I don't really have any LARGE single files but, I do have a lot of singles files. (Movies might be couple gig? and a bunch of images at 1 to 2 megs each.) Do you think the read/write of the Drobo FS would be enough for this?

Also, how did you get the Pro for the same price as the FS? That's peaks my interest.

 

[DustinT]

I've been looking into the Drobo FS as well. I don't really have any LARGE single files but, I do have a lot of singles files. (Movies might be couple gig? and a bunch of images at 1 to 2 megs each.) Do you think the read/write of the Drobo FS would be enough for this?

Also, how did you get the Pro for the same price as the FS? That's peaks my interest.

The Drobo FS transfers a lot faster than the regular Drobo, which isn't saying much. For normal usage though, I think its fine. The problem would come in if you planned on using it to store media that you were editing. For example, if you planned on storing all the media from a HD movie project on it you would likely find the low performance would be an issue. If you stored lots of raw images from your high end dslr camera and tried to edit them all in a day, you'd be complaining.

If all your looking for is to store Time Machine backups, stream your media and generally just have a massive disk connected to your system, you'll be fine.

Numbers wise, the Drobo Pro is capable of sustained transfers in the 70-90 MB\s range. The FS can push 40-60 and the regular 'ole drobo can handle 20-30 (on a good day).

I got the Drobo Pro for a steal because it was used. A forum member here was selling it and gave me a stunning deal. Otherwise, I probably wouldn't have spent quite that much.

 

[xraydoc]

The Drobo FS transfers a lot faster than the regular Drobo, which isn't saying much. For normal usage though, I think its fine. The problem would come in if you planned on using it to store media that you were editing. For example, if you planned on storing all the media from a HD movie project on it you would likely find the low performance would be an issue. If you stored lots of raw images from your high end dslr camera and tried to edit them all in a day, you'd be complaining.

If all your looking for is to store Time Machine backups, stream your media and generally just have a massive disk connected to your system, you'll be fine.

Numbers wise, the Drobo Pro is capable of sustained transfers in the 70-90 MB\s range. The FS can push 40-60 and the regular 'ole drobo can handle 20-30 (on a good day).

I got the Drobo Pro for a steal because it was used. A forum member here was selling it and gave me a stunning deal. Otherwise, I probably wouldn't have spent quite that much.

I found the Drobo FS didn't get anywhere near 40MB/sec. Maybe half that. Perhaps it was just the combination of drives I had in it, but there's no way you could stream HD video off of it. Perhaps with 5 ultra-high performance enterprise class RAID-certified drives you might, but my experience was that it was just barely acceptable for backups and near useless for storing active projects larger than say 100MB.

Duplicating my iPhoto database on to it -- at the time around 70GB -- used to take literally 15+ hours. I had to either do it over night or forget about using that app for the better part of a day. Meanwhile, with a Mac Pro and a fast RAID in it, not only does it take 1/3 as long to copy the whole iPhoto library, it's fast enough to actually use iPhoto with the library hosted over the network. And streaming an uncompressed Blu-Ray 1080p movie off the Mac Pro is also flawless.

mwhities, feel free to test the Drobo FS and see if it's fast enough for your use, but just make sure to get it from some place that has a good return policy if necessary.

 

[DustinT]

I found the Drobo FS didn't get anywhere near 40MB/sec. Maybe half that. Perhaps it was just the combination of drives I had in it, but there's no way you could stream HD video off of it. Perhaps with 5 ultra-high performance enterprise class RAID-certified drives you might, but my experience was that it was just barely acceptable for backups and near useless for storing active projects larger than say 100MB.

Duplicating my iPhoto database on to it -- at the time around 70GB -- used to take literally 15+ hours. I had to either do it over night or forget about using that app for the better part of a day. Meanwhile, with a Mac Pro and a fast RAID in it, not only does it take 1/3 as long to copy the whole iPhoto library, it's fast enough to actually use iPhoto with the library hosted over the network. And streaming an uncompressed Blu-Ray 1080p movie off the Mac Pro is also flawless.

mwhities, feel free to test the Drobo FS and see if it's fast enough for your use, but just make sure to get it from some place that has a good return policy if necessary.

That bad, huh? How were you connecting it to your Mac? I ask because I briefly owned a Drobo (4 bay model) and it would easily do 20-25 for transfers under 100g. After that it slowed down to 12-15 which was quite irritating. By the way, I had an assorted collection of spare hard drives in it, nothing was matching. I believe the largest was 1.5t and the smallest was 400g. Actually, the 400g has since died completely!

 

[NorCalLights]

This is a little bit off of the path this thread has headed, but I feel like I should interject: make sure you have an offsite backup solution as well. I use my regular ol' Drobo for Time Machine backups, but none of that will help me if my apartment is broken into or if there's a fire, or if I need to do a restore while I'm traveling with my laptop.

And I also make sure I never keep anything on the Drobo that doesn't exist somewhere else as well. I had the Drobo filesystem crap out on me when my dog pulled out the power cord, and I lost everything on it.

So to that end: I just signed up for Backblaze. $4/month/computer for unlimited backup and restores over the internet. It might take a long time to do the first upload (mine took almost 2 weeks) but after that, it's totally seamless. The native Mac app is fantastic, and Backblaze's support got right back to me when I had a stupid question... even on a Sunday of a holiday weekend.

They also offer a service (for an extra charge) where they will overnight FedEx you a USB hard drive with your backup on it if you ever need to do a restore. I don't work for Backblaze... just can't say enough good things about them.

 

[DustinT]

They also offer a service (for an extra charge) where they will overnight FedEx you a USB hard drive with your backup on it if you ever need to do a restore. I don't work for Backblaze... just can't say enough good things about them.

Thats a pretty cool service. There's time when I can see being willing to pay for an overnight delivery. I'm trying Crashplan right now which allows backing up to a friend's pc. So, now I can just drive across town and pick up the data if I need to. But, I tend to use multiple backup services. I've got some data in Drobbox, Time Machine and now Crashplan. I just don't trust any one service to always be available when I need to restore.

 

[mainstay]

I second a local back up + BackBlaze.

I have almost all of my clients using this scheme. Some are augmented with a second local backup that is physically removed offsite on a rotating basis.

BackBlaze is cheap, has versioning (cannot overstress how important this is), and works quietly in the background. It is the ultimate set-it-and-forget-it strategy.

I have tried CrashPlan (free accounts only) twice and both times I returned to the site only to find it was not doing its job. In one instance the path to the target remote computer was disrupted and the backup didn't work, in another instance the backup to a LOCAL USB harddrive simply didn't work... I have no idea why but I axed it immediately.

I'm not saying CrashPlan doesn't work, I can only point to my experience that shows it might need a bit more admin'ing.

The only downside to BackBlaze is that it does not support network drives. It does support USB drives. And if you are feeling particularly cheap (like me) I backup all of my systems to a central data drive and then run backblaze on that one computer ($4 for 300 GB of truly important data (company files, quickbooks files, personal pictures of the family - stuff I would be devastated to lose)).

Whatever works for you, just remember, if the data doesn't exist in two locations, it isn't backed up.

 

[xraydoc]

So to that end: I just signed up for Backblaze. $4/month/computer for unlimited backup and restores over the internet. It might take a long time to do the first upload (mine took almost 2 weeks) but after that, it's totally seamless. The native Mac app is fantastic, and Backblaze's support got right back to me when I had a stupid question... even on a Sunday of a holiday weekend.

They also offer a service (for an extra charge) where they will overnight FedEx you a USB hard drive with your backup on it if you ever need to do a restore. I don't work for Backblaze... just can't say enough good things about them.

That's the problem with internet-based backups. It's just not yet a viable solution for huge datasets with current ISP speeds.

There's no way my data would be static enough for two weeks to upload and have any of the data be consistent from the beginning to the end of the upload. One change to something that was already uploaded before the rest of the upload finished and I'd potentially have a non-functional backup.

I've got 10GB of data on Dropbox that is synced across multiple machines - data that I need to use in multiple places (laptop, home desktop, office desktop), so at least that data is safe in the event of a catastrophic disaster with my file server at home. But that's mostly stuff for work, not my expensive iTunes library or irreplaceable family photos.

Again, eventually I will investigate a method for off-site backups, but right now we're talking nearly 4TB of data I'd need rotated off site. That would take months via internet, even with my FiOS 50/25Mbit service. Could rotate two 4TB external hard drives off site, which is probably what I'll have to do, but I think I better wait a month or two so the wife doesn't get annoyed given all the money I've already spent this month (including ~$750 for the OWC Qx2 and four 2TB hard drives to build out the RAID 5).

Most likely my off site strategy will be to buy two 4TB externals, mirror the data on the RAID on to one external HD with SuperDuper!, take the drive to the office, and rotate the other one home every week (one at home staying up to date, one at the office containing at most 1 week old data).