Backing up the nMP

[Michael73]

First some background:

I had a MP 3,1 with all four drive bays filled with the following:

1. Drive Bay 1 - Macintosh HD: Applications, Documents and some media (Music, iMove DB, and iPhoto stuff)
2. Drive Bay 2 - Media (movies and more music)
3. Drive Bay 3 - Uncompressed Media e.g. things that are ultimately going to be encoded (probably using HB) to Drive 2
4. Drive Bay 4 - Time Machine

My nMP arrived last Thursday and it contains a 1TB SSD. My old Macintosh HD had 970GB. In order for the migration to occur more quickly, I moved all the contents of the Music, Movies and Pictures directories off to Drive 2. This left my Macintosh HD with ~495GB.

Needing a home for the information in Drive Bay 2 & 3, I bought a Drobo 5D. So not only did I transfer all the media that used to be on those drives, but also the Music, Movies and Pictures directories as well. In setting up the nMP, iPhoto and iTunes have been pointed to the moved libraries and everything is functioning well.

This brings me to the last part of the equation...Time Machine backups.

I realize that I can partition the Drobo and use it for backups, but for some reason I'm nervous about doing so. For example, I could not remove a drive from the Drobo, put it in an external enclosure, directly attach it to a machine and restore. Therefore, I *think* buying a dedicated external enclosure for Time Machine is the way to go. So here are my questions:

1. For TM backups is there any need to go the TB route or is USB3 fine?
2. Is there any need for the TM backups to be in a RAID either striped or in parity?
3. The plan is to completely back up everything on the internal drive of the nMP, but is it necesary (or even possible) to back up some of the items on the Drobo? After all, aren't those items insulated from failure by the fact that they're in a sort of RAID anyway?

Sorry for the super long post and thanks in advance for your help!

 

[Nugget]

For TM backups is there any need to go the TB route or is USB3 fine?

USB3 is more than adequate for time machine use. Even Ethernet is perfectly fine if you wanted to do a network-attached storage device for Time Machine. Heck, even WiFi is fine seeing the popularity of Apple's Time Capsule products.

Is there any need for the TM backups to be in a RAID either striped or in parity?

I think this only makes sense if you have an unusually high need for the historical snapshot aspect of Time Machine. If your Time Machine drives takes a dive you lose your backup, but since the primary data still exists you're really only losing the old file versions and can recover your current data with no hassle by just replacing the dead Time Machine drive immediately. You'll have brief exposure during the initial backup to the replacement drive -- which is a risk but arguably a small one. That's a judgement call you have to make, but I suspect for most people it's probably not worth the expense and overhead of storing Time Machine to a RAID array.

The plan is to completely back up everything on the internal drive of the nMP, but is it necesary (or even possible) to back up some of the items on the Drobo? After all, aren't those items insulated from failure by the fact that they're in a sort of RAID anyway?

Write this on the blackboard one hundred times: RAID is not a Backup. RAID is not a Backup

RAID protects you against downtime due to hardware failure. Nothing else.

A backup is to protect you against data loss, which RAID does not really do. RAID doesn't protect you against data corruption, accidentally deleted files, or accidentally overwritten files. If your application goes crazy and writes all zeros to your thesis document, RAID will efficiently and effectively mirror those zeros to all the drives. If you accidentally save a new document on top of an important old document, raid will efficiently and effectively destroy that old document replacing it completely with the new one. RAID just means that when one of your hard drives fails you can keep running while you install its replacement. It does not mean that your data is safe.

Heck, the real beauty of Time Machine is that you can easily go back to previous versions of a file, even months or years back in time to recover a file in the state it was in at that point in time. You might not have to do that often or even ever, but if you do you'll be glad you can.

 

[Michael73]

USB3 is more than adequate for time machine use. Even Ethernet is perfectly fine if you wanted to do a network-attached storage device for Time Machine. Heck, even WiFi is fine seeing the popularity of Apple's Time Capsule products.

Good point. I have a TC that my MBP backs up to but I find WiFi slow and I'd never use it to do a full backup or restore from. I think even Apple recommends doing those via ethernet or some form of hardwire.

I think this only makes sense if you have an unusually high need for the historical snapshot aspect of Time Machine. If your Time Machine drives takes a dive you lose your backup, but since the primary data still exists you're really only losing the old file versions and can recover your current data with no hassle by just replacing the dead Time Machine drive immediately. You'll have brief exposure during the initial backup to the replacement drive -- which is a risk but arguably a small one. That's a judgement call you have to make, but I suspect for most people it's probably not worth the expense and overhead of storing Time Machine to a RAID array.

My thought here was that I could get a 2-drive enclosure and with a striped array I could get the benefits of speed and loads of space (well, double that of a single drive). I guess I was a little unsure of what happens in a striped array if one of the drives goes down? Do I lose half my information? Is this a reason for a 4 drive array that is both striped and in parity?

A backup is to protect you against data loss, which RAID does not really do. RAID doesn't protect you against data corruption, accidentally deleted files, or accidentally overwritten files. If your application goes crazy and writes all zeros to your thesis document, RAID will efficiently and effectively mirror those zeros to all the drives. If you accidentally save a new document on top of an important old document, raid will efficiently and effectively destroy that old document replacing it completely with the new one. RAID just means that when one of your hard drives fails you can keep running while you install its replacement. It does not mean that your data is safe.

Heck, the real beauty of Time Machine is that you can easily go back to previous versions of a file, even months or years back in time to recover a file in the state it was in at that point in time. You might not have to do that often or even ever, but if you do you'll be glad you can.

Excellent. Excellent. Thanks! Now the question that wasn't answered...is there any reason why items on a Drobo can't be backed up by TM? The reason for the question is because I know that the info stored in a Drobo is in some kind of different format that can't be used without the drive being in a Drobo.

 

[Nugget]

My thought here was that I could get a 2-drive enclosure and with a striped array I could get the benefits of speed and loads of space (well, double that of a single drive). I guess I was a little unsure of what happens in a striped array if one of the drives goes down? Do I lose half my information? Is this a reason for a 4 drive array that is both striped and in parity?

If you lose one of the drives in a stripe (RAID0) you effectively lose all the data. While it may be theoretically possible -- with heroic effort -- to recover half the data it's definitely not safe to plan on being able to do that. For most users I think a RAID0 stripe for a Time Machine volume is a reasonable compromise though. You're right that it can be a cheaper way to achieve a given amount of storage space or a way to achieve more storage space than can be bought in a single drive.

Excellent. Excellent. Thanks! Now the question that wasn't answered...is there any reason why items on a Drobo can't be backed up by TM? The reason for the question is because I know that the info stored in a Drobo is in some kind of different format that can't be used without the drive being in a Drobo.

No, no reason at all you can't back up data stored on a directly-attached Drobo (like your 5D) via Time Machine. Data on a network mounted Drobo (like the 5N for example) cannot be backed up via Time Machine because it is not local to the machine.

 

[spaz8]

Not sure if this adds anything, but my plan/setup is.

1 TB SSD in the Nmp (OS, apps, scratch disk?)

Drobo Gen2 (4x 2TB) is the time machine connected with a FW800 to TB cable.

Pegasus2 R4 in Raid5 (4x 2TB) is the media drives.. replaces all the internal storage I had in my MP 1,1.

 

[Michael73]

If you lose one of the drives in a stripe (RAID0) you effectively lose all the data. While it may be theoretically possible -- with heroic effort -- to recover half the data it's definitely not safe to plan on being able to do that. For most users I think a RAID0 stripe for a Time Machine volume is a reasonable compromise though. You're right that it can be a cheaper way to achieve a given amount of storage space or a way to achieve more storage space than can be bought in a single drive.

You know, the more I think about it, there's really no reason for the TM drive to be in RAID0. As I understand it, R0 is all about speed and performance which is really not what's needed from TM. If I go the dual drive array route, maybe the better option is to just create a single large TM volume to keep a longer history of backups. Thoughts? What happens in this case if one of the two drives fails?

 

[Nugget]

You know, the more I think about it, there's really no reason for the TM drive to be in RAID0. As I understand it, R0 is all about speed and performance which is really not what's needed from TM. If I go the dual drive array route, maybe the better option is to just create a single large TM volume to keep a longer history of backups. Thoughts? What happens in this case if one of the two drives fails?

What you're describing is RAID0. A non-redundant concatenation/stripe of multiple disks creating a single logical volume that is the size of all the drives added together. And if you lose any of the drives in the array you should just expect to lose all the data. The volume will be trashed and effectively destroyed. While it's technically true that some portion of the data will still exist on a non-failed drive, it wouldn't be simple or straightforward to recover it.

Even with that risk, I still think it's a reasonable choice for a Time Machine target for most users, since losing the backup data still leaves the real data intact. All you'll really lose (and be unable to recover) is the historical snapshots. That's unfortunate, but the cost and overhead to protect them is likely not worth it for most people.

 

[Michael73]

What you're describing is RAID0. A non-redundant concatenation/stripe of multiple disks creating a single logical volume that is the size of all the drives added together. And if you lose any of the drives in the array you should just expect to lose all the data. The volume will be trashed and effectively destroyed. While it's technically true that some portion of the data will still exist on a non-failed drive, it wouldn't be simple or straightforward to recover it.

Even with that risk, I still think it's a reasonable choice for a Time Machine target for most users, since losing the backup data still leaves the real data intact. All you'll really lose (and be unable to recover) is the historical snapshots. That's unfortunate, but the cost and overhead to protect them is likely not worth it for most people.

I thought R0 striped was when there are two drives and half the data is written to (or read from) each. Whereas, what I talking about was to format both drives as one large partition such that TM backups would fill up one drive and then move on to the next. Maybe I'm all kinds of confused but I would have thought in the second case if Drive 1 contained backups from January through June and Drive 2 held July through December, that if Drive 1 failed (or Drive 2 was filled up) I could just replace Drive 1 without the loss of all my backups.

 

[Nugget]

...what I talking about was to format both drives as one large partition such that TM backups would fill up one drive and then move on to the next.

That's not really how it works when you format two drives as one large partition. The data is spread across both drives from the beginning.

 

[Reno Richter]

when you join several discs into one volume it is called spanning.

see this link
http://en.wikipedia.org/wiki/Spanned_volume

 

[Nugget]

Sure, it's also called disk concatenation, and RAID0, and striping. There are plenty of terms for the concept.

 

[ogilloire]

My setup

What i did is with Pegasus2 R4:

• disks 1,2,3: 3x2TB in RAID5 = 4TB usable capacity
• disk 4 = 4TB single drive for time machine

+ I have an offsite cloud backup in Amazon Glacier with ARQ software, in case the room burns down or i get robbed or anything like that.

 

[bxs]

Michael73:

Do this for less than $350

Buy three LaCie Minimus 2 TB USB 3.0 Desktop External Hard Drive 301967 at Amazon (or elsewhere if can can find it for less)... http://www.amazon.com/LaCie-Minimus-Desktop-External-301967/dp/B0042KFLJE

Then use them as follows

1) First one used for cloning your 1TB SSD using SD! or CCC. This could be split into two equally sized partitions. One for the clone and one for other stuff if you wish safeguard. The 1TB SSD clone will serve you if ever your internal 1TB SSD becomes corrupt or the SSD fails in some way and you want to get back up ASAP. You simply boot from the clone image. Alternatively, buy a 1TB USB3 quality disk enclosure just for cloning. Clone at least once per week or more frequently if you wish. SD! and CCC can be scheduled to run automatically when you want.

2) The 2nd and 3rd ones use for Time Machine backups. Configure Time Machine to use both units alternatively. This provides a backup for Time Machine if one of the TM disks fails.

This backup method provides

1) Quick return to production use if the internal 1TB fails in some way. IMO this is unlikely as the Apple SSD is very robust.... but you can never tell when things will break.''

2) Time Machine backup for restoring your whole system if required and for restoring old versions of a file(s)

3) Time Machine data safeguarded by having two copies so that if one of the disks fail along the way you can still recover.

My policy is always to have multiple methods for recovering from disk failures. Multiple backups for Time Machine is a small price to pay for the protection it provides you with.

Multiple backups should always be on unique disk spindles. Never place multiple backups on the same spindle.

Personally, I maintain two system boot clones at all times. These would be created every 24 hours... but it depends on you data churn as to how frequently you clone.

[EDIT]
Also check your restore methods periodically. That is, boot up on the clone once per week to make sure that works. Also restore some files from Time Machine backups once per week to ensure the backups have not been compromised.

 

[rasputin666]

Sure, it's also called disk concatenation, and RAID0, and striping. There are plenty of terms for the concept.

I don't want to get picky (especially with a Demi-god ) but I do want to clarify for new readers looking for direction. Concatenated/spanning is not the same as raid0 or striping. What's the same is one large volume. But they perform, act, and write data very differently.

 

[ZnU]

I don't want to get picky (especially with a Demi-god ) but I do want to clarify for new readers looking for direction. Concatenated/spanning is not the same as raid0 or striping. What's the same is one large volume. But they perform, act, and write data very differently.

Right. This is a bit simplified, but with RAID 0 every other block is written to an alternating drive. So if you lose one of two drives, say, technically you're 'only' losing half your data, but the half you're losing is every other block of every single file of non-trivial size (more than one block). You could recover some useful data from this under some circumstances, but in most cases it's basically a total loss of everything.

With spanning, however, individual files aren't split between drives at the block level like this, so if you lose one drive, you still have one drive worth of whole files left. So you do only lose half your stuff rather than all. Spanning shouldn't lose more data in the face of one of two drives failing than you would lose if you were using them as independent volumes. In theory; I haven't done any testing of how OS X's built-in spanning function actually behaves.

By way of analogy, imagine that you have 100 tennis balls. The RAID 0 failure cuts every individual tennis ball in half, leaving you with zero useful tennis balls. The failure of the spanned volume destroys 50 of the tennis balls completely, but leaves the other 50 alone, leaving you with 50 useful tennis balls.

 

[comatory]

what's so hard about attaching another drive? why are you debating RAIDs here?

jesus simple USB2/3 disk is totally fine for any kind of backup. do you really want your backups to be safe? buy two of those drives and rotate them (possibly at different physical location).

as poster above me wrote, redundant post.

 

[Nugget]

Concatenated/spanning is not the same as raid0 or striping. What's the same is one large volume. But they perform, act, and write data very differently.

You and ZnU are absolutely right. I stand corrected.

 

[deconstruct60]

....
Heck, the real beauty of Time Machine is that you can easily go back to previous versions of a file, even months or years back in time to recover a file in the state it was in at that point in time. You might not have to do that often or even ever, but if you do you'll be glad you can.

That is only if assign enough capacity to Time Machine (TM). If backing up 1.5TB of data with a 2TB TM target drive there aren't necessarily going to be much of an archive there. For TM's archive feature to work well then need to have capacity that is substantially larger than the disk/volume that are backing up. Something like 500GB : 1000GB is going work much better long term than 750GB : 1000GB set up.

TM starts to run into a problem when the initial target drive is large or there are multiple drives targeted at one TM repository. At that point, RAID's ability to create a larger capacity virtual drive that is bigger than what can buy (for the moment) is useful.

RAID may not be a back-up, but the back-up may be sitting on top of RAID. In that context, the back-up data needs what RAID delivers ( better resiliency and availability ).

That said trying to use TM to back-up data that is about as large as current max HDD sizes represents as higher level of risk. It is far more aimed at relatively much smaller sources mapped to biggest-at-the-time targets.




----------

With spanning, however, individual files aren't split between drives at the block level like this, so if you lose one drive, you still have one drive worth of whole files left.

The data may be there, but where is the metadata about the files? If the file systems metada about the files is lost then access to those files is deeply impaired.

There isn't just user data stored on drives. The metadata about the files is stored on the same drive(s). Screw that up and not particularly different at the user access level from a drive failure.

Unless the file system itself creates the "spanning", it is not going to know that it needs to appropriately split up the metadata exactly between muliple drives. If the RAID system presents it as one drive to the file system, it will be treated like just one drive.

 

[Michael73]

I think I've come up with a plan. I realize it's not crazy bulletproof but I think it's enough for me...

Suppose I get a two-drive enclosure and 2 x 3TB HDDs. Drives 1 & 2 are both formatted with 2 partitions each with a 2.4TB and a 0.6TB partition. The two larger partitions are both Time Machine and the two smaller partitions are boot drive partitions. I'll set up TM and aim it at both. The smaller partitions I'll have CCC make nightly backups to both. BTW, my boot drive is ~495GB so a 0.6GB partition should be enough. The total amount of data I want my TM to back up is just under 1TB so a 2.4TB partition should last me 6 months to a year.

Thoughts about this arrangement?

 

[ogilloire]

Why 2x3TB? I assume you want to do a RAID1?

I'm not sure Time Machine can backup the drive it is running on, perhaps it is clever enough not to backup the backup?

Perhaps best to have 3 partitions?
0.6 + 1.2 Data + 1.2 TM?

 

[Michael73]

Why 2x3TB? I assume you want to do a RAID1?

I'm not sure Time Machine can backup the drive it is running on, perhaps it is clever enough not to backup the backup?

Perhaps best to have 3 partitions?
0.6 + 1.2 Data + 1.2 TM?

Not necessarily. Incorrect assumption. The two drives could be JBOD. If so you could do this two ways...You could create two partitions on Drive 1...TM and Bootable partition with the bootable copy made by CCC or SD then you could copy the entire drive (both partitions) to Drive 2 again using CCC or SD. Basically this is RAID1 but more manually.

Another way of accomplishing this again using JBOD is to have two partitions on each HDD (four total). Two of the partitions for TM. TM can recognize more than one place to back up, I could just aim it at both. For the Bootable piece I'm thinking I could just schedule a nightly job to have CCC copy everything to one partition on Drive 1 and then do the same for Drive 2.

In either case I wind up two identical drives - each with a 6-12 month (estimate based on past history) history of backups and a bootable partition should the drive in my nMP fail.

 

[chrisn123]

Multiple TM volumes

Drives are cheap, critical data is hard to replace.

I would throw your primary backups on your Drobo. Then buy a USB3 sled for $30 and use two OTHER HDDs (you may have lying around). Set up TM to backup to your Drobo AND each of the HDDs (three active backup sets). Use Chronosync or CCC or whatever to ALSO sync any critical data on your Drobo to your USB3 sled. All this prob fits on a 2TB HDD.

Rotate the HDDs in the Sled every few days and keep the other at work / offsite (or at least in a fireproof safe).

Simple, easy, redundant.

I do this myself, with the additional twist that the array hooked to my nMP also serves as a Time Machine targets for five other Macs in my family (using OS X Server). A 4TB HDD easily hold all my "Backups of Backups" and some extra stuff (like my Aperture library).

As an aside, handling multiple concurrent backup jobs doesn't even make the array break a sweat, but it is a single click on the Server preference to turn it off temporarily if desired.