What are you doing with your HDDs?

[Michael73]

Circling back...

So I purchased the Drobo over the weekend and finally got around to setting it up yesterday and for the most part things went pretty smoothly. Here are some notes…

I have another thread detailing this but in order to maximize the transfer of terabytes of data from an older MP that doesn't have either TB or USB3 (the two connections on the Drobo 5D), I had to get creative. As you can see in my sig, I have a new(er) MBP with TB. So, I decided to connect my MP to the MBP via an ethernet cable and plug the Drobo via the provided TB cable into the MBP…essentially turning the MBP into a hub. Ultimately this worked out quite well but it took some time as I had to manually configure the two machines to recognize each other. For some odd reason, simply plugging the ethernet cable into both machines wasn't automatically recognized by either machine. I had to manually configure the IPs, subnet masks and restart both machines. But, once completed it all worked like a charm. I can't tell you for sure what the gigabit transfer speed was but when I copied over about 500GB it took about an hour.

Aiden, et al….so the first thing I did was put two older 1.5TB drives in the Drobo. Wouldn't you know that immediately after formatting them, the Drobo gave me a warning the one of the drives failed. I simply removed the dead drive, found a two more older 500GB drives and threw them in to the enclosure. They immediately formatted and I was up and running. At this point I still hadn't transferred any data to the Drobo so no harm, no foul. Once I got the data transfer under way (about 800GB), I ran over to my local Frys and picked up two, 3TB Western Digital Red series network drives.

Utlimately, once the new Mac Pro arrives, I'll partition the drives in the Drobo so that I can use part of that space for Time Machine backups of the PCIe storage in the MP itself.

 

[mcnallym]

. So, I decided to connect my MP to the MBP via an ethernet cable and plug the Drobo via the provided TB cable into the MBP…essentially turning the MBP into a hub. Ultimately this worked out quite well but it took some time as I had to manually configure the two machines to recognize each other. For some odd reason, simply plugging the ethernet cable into both machines wasn't automatically recognized by either machine. I had to manually configure the IPs, subnet masks and restart both machines.

With the two machines connected together unless manually configured then they would look for a DHCP server on the wire, and neither machine would be a DHCP Server and so there would be a 169.x.x.x IP address that basically means then machines have self addressed and won't talk to other machines.

When you plug the two machines either via Wireless or Ethernet into your Broadband Router then that is a DHCP Server and so gives the two machines IP addresses from it's DHCP scope so the two machines can talk to each other now as in the same IP subnet.

 

[Michael73]

With the two machines connected together unless manually configured then they would look for a DHCP server on the wire, and neither machine would be a DHCP Server and so there would be a 169.x.x.x IP address that basically means then machines have self addressed and won't talk to other machines.

When you plug the two machines either via Wireless or Ethernet into your Broadband Router then that is a DHCP Server and so gives the two machines IP addresses from it's DHCP scope so the two machines can talk to each other now as in the same IP subnet.

I'm not a network guy so when I plugged in the ethernet cable and both machines in the Network pane within system preferences still showed "not connected" and that the cable was unplugged, I was confused. Since I still have a MBA under AppleCare I just called apple and they helped me work through the problem. Ultimately, they instructed me to manually configure the connection. I set the IP of the MBP to 192.168.0.1 and the MP to 192.168.0.2 but put them both on the same subnet mask: 255.255.255.0. Like I mentioned, that solution didn't work until I powered down both machines, reset the SMC on the MBP and restarted them again.

 

[Michael73]

Thanks. That sounds like a very workable approach to my needs.

I just read through reviews on Amazon. There are some reports of total failure of the Drobo unit. In a few cases this resulted in total loss of data. These cases seem to be rare, but it brings up the question of how to best protect a large quantity of data. I have about 7.5 TB of data. At present, the most critical data (about 1 TB) is backup using TimeMachine to a 3TB external HD. The remaining data which is very stable (few deletions, no updates and new files added periodically) is backed up to two external HD using chronosync.

I'm going to buy a new Mac Pro when it comes out. A drobo with 2 4TB and 3 3TB drives will give me some room for room for growth and protect me from a single disk failure. I can also keep my current TimeMachine drive. If I want to increase my level of protection, what are my options? Buying a second Drobo with another 17TB of storage seems like overkill. Is there a middle ground?

Rich, our situations are nearly identical. Nearly all my sensitive data which changes most frequently is on my boot drive. Although it's a 2TB drive, only a little less than 1TB is actually used. This information is backed up to a 3TB Time Machine drive located in the fourth drive bay internally to my MP. Drive bays 2 & 3 are both 1.5TB drives with fairly static information (mostly media like movies and music). Neither of these drives are backed up today (knock on wood, I've never had a failure). However, both of these "media drives" started out life as 500GB drives so when I outgrew them, I copied the files to the 1.5TB drive and set the 500GB drives aside in case of drive failure.

So, the Drobo actually accomplishes a few things…first, the media drives are now in RAID so if one of the drives fails, they are now backed up. Second, if I understand how this all works correctly, not only do I have a Time Machine drive, but that information is in RAID as well. So, if my Time Machine drive fails, I can pull it and replace without skipping a beat.

IMHO, having a DROBO and Time Machine combination is going to go a long way in protecting against data loss. Short of the entire Drobo failing, someone breaking in and stealing the nMP / Drobo, a lightening strike that fries all the electronics, a flood or fire you've reduced your chance at true data loss well into the single digits or below. To further safeguard your data you can consider using the variety of free (or paid) cloud services for off-site storage…think dropbox or Google Drive.

FWIW, since I own my own company and work from home, I added a special computer / electronics rider to my home insurance policy that provides an extra layer of replacement coverage should the computers and network equipment be stolen, damaged, etc.

 

[Rich.Cohen]

not only do I have a Time Machine drive, but that information is in RAID as well. So, if my Time Machine drive fails, I can pull it and replace without skipping a beat.

… you've reduced your chance at true data loss well into the single digits or below.

FWIW, since I own my own company and work from home, I added a special computer / electronics rider to my home insurance policy that provides an extra layer of replacement coverage should the computers and network equipment be stolen, damaged, etc.

Putting your TimeMachine on a RAID is a good idea. I'm going to move my TimeMachine to a RAID drive too.

Risk analysis is one of the things I do for a living. For me a 1% probability of total data loss over 3 years would be unacceptable. I want something much lower than that. I suspect that the Drobo will meet that need. I'm talking about permanent loss of data, not downtime to repair.

A few years ago I reviewed a requirement for a client where they wanted 99.99% availability for a system. When I asked if they would accept 1 hour of down time each year, they said no. I suggested that they either increase the required availability or add a mean time to recover requirement.

I also have a home-based company. I just checked my policy. I have a rider that covers my computers against accidental damage like spilled coffee or dropping the computer for up to $10,000. That is the maximum limit. However, theft, fire or lighting damage is fully covered by my base policy. I guess I'm OK unless the new Mac Pro itself costs more than $10K.

 

[Michael73]

Putting your TimeMachine on a RAID is a good idea. I'm going to move my TimeMachine to a RAID drive too.

Risk analysis is one of the things I do for a living. For me a 1% probability of total data loss over 3 years would be unacceptable. I want something much lower than that. I suspect that the Drobo will meet that need. I'm talking about permanent loss of data, not downtime to repair.

Two things…even if the Drobo fails and the box "dies," there's no reason why you shouldn't be able to go out and buy a new Drobo, move the drives into that enclosure and continue on your merry way.

Second, don't forget that if the Time Machine is backing up your entire boot drive (which by default it does, unless you've told it items to exclude) then even in the event of a catastrophic loss where the Drobo and the RAIDed Time Machine becomes unrecoverable, your main boot drive is still very much intact.

I also have a home-based company. I just checked my policy. I have a rider that covers my computers against accidental damage like spilled coffee or dropping the computer for up to $10,000. That is the maximum limit. However, theft, fire or lighting damage is fully covered by my base policy. I guess I'm OK unless the new Mac Pro itself costs more than $10K.

Long ago, I started my career working in the financial services industry…mostly life, but I have some experience in health and P&C. Take a second look at that rider. Yes, your insurance company will cover the cost to replace the computer, but what about the cost for data recovery on your hard drive from that spilled coffee? Also, what about the cost to go out and buy new software? Also make sure that accidental damage includes peripherals like monitors, UPS systems, printers, scanners, etc. Personally, I'd look for an all perils policy versus a named perils one. In other words, everything is covered (all perils) unless it's specifically excluded. A named perils policy is sometimes much more difficult to get stuff covered by.

 

[Rich.Cohen]

Exactly!
But MTBF figures today is in the order of 1 million hours for enterprise disks (thats more than 100 years). This is not the expected life span of a single disk but related to how many of, say 10,000 disks, that is expected to fail the first month or so. No disks lasts 100 years, 10, maybe 20 years if you are lucky, but not much more than that.

Once upon a time (1980's) I was a reliability engineer. MTBF is a statistical estimate of the time it will take for half the sample set to fail the first time. (The is some argument about dealing with multiple failures for the same item. In some cases they are ignored and in other cases they are treated as just another data point.) So if they are saying 1,000,000 hours MTBF, they are claiming that 50 out of 100 drives will last 114 years.

Now there is a problem: You can't test a batch of new products for 100 years to get real statistical data. The solution is to assume a failure distribution and extrapolate.

Most electronics and some hardware have an infant mortality problem - the chance of failure in the first week is much higher than the chance of failure in the 50th week. After the infant mortality problem the failure distribution is usually exponential - the chance of failing during week n+1 is just a bit greater than the chance of failing during week n and the factor is a constant. So if you run your testing well past the infant mortality period and long enough to get a statistically meaningful number of failures, you can do a pretty good estimate. I don't remember the numbers but for 1,000,000 hours MTBF I'd suspect hundreds of devices for about a year of testing.

I can't see a product being held for a year to complete such testing. Now after the product is in actual use and under warranty you can get some much better data because your sample size is very much larger. On the other hand you have to estimate the actual time to failure for each reported failure since the customer usually doesn't have that information.

Of course run time isn't the only factor. As noted by others disk spin-up/down cycles can be more important than duty hours. If I recall correctly certain kinds of aircraft engines are much more sensitive to the number of starts than the number of operating hours. Aircraft engines where easy to calculate because the customer always had exact records of operational use, both hours of operation and number of starts, and also maintenance events. Better data, more meaningful analysis.

It may well be that a disk can have an MTBF of 1,000,000 hours if you never shut it down and a very much lower MTBF if you shut it down once a week for 10 minutes. The MTBF of most light bulbs is much longer if you never turn them off. If I were a marketer, I wonder which numbers I'd put in my sales literature?

 

[haravikk]

Since I last posted in this thread I've decided to go the route of a self-build NAS by buying a 9x 5.25" drive bay case, putting a good PSU in it and a set of a RAID controllers. Not cheap (whole lot's going to come in around £600-700 once I'm done I think) but it'll give me tons of room for drives in almost any configuration I want (up to 15x 3.5" drives, or a massive 36x 2.5" drives!). Plus if suitable Thunderbolt controllers become available I can swap those in to boost my speed.

Putting your TimeMachine on a RAID is a good idea. I'm going to move my TimeMachine to a RAID drive too.

It's a great thing to do; with the above setup I'm hoping to have all of my external storage on RAID 5, so that's both my Time Machine backups and my soon to be external primary storage.
The latter will be using an SSD to create a Fusion Drive, so while I'm (mostly) protected against HDD failure, the SSD is still a weak point, but I figure the SSD speed is a reasonable risk, plus my backups should keep me safe in the event that it is the drive to fail.

There are some caveats to remember though; namely that if a drive in a RAID-5 set fails and you swap in a new one, it pushes the remaining drives pretty hard during the rebuilding process, especially if you try to keep using the drives during that time. This is why RAID-6 has become popular if you can't afford some of the other options. I believe Drobo's disk management lets you mix and match various RAID benefits, but I dunno how much freedom you get with five disks?

Also, when thinking about back-ups you should think about the worst case; i.e - your whole house burning down. Personally I also have a NAS in a kind of out-building that should be fairly safe even in the event of a house fire, plus it gives me a third copy of my data (primary drives, direct attached backup, network attached backup). And I'm thinking I might even go fully paranoid and have the NAS back itself up to CrashPlan or some other backup service.

Obviously that's a bit overkill, but it gives me a kind of tiered backup; backups to direct attached storage every hour, backup to NAS every hour to two hours (depending on how big the backup is) and cloud backup overnight. Direct attached backup still lets me use time machine with speed and convenience, while the NAS keeps everything extra secure (also handles backup for my other machines).


Sorry, this turned into an assay, but I thought I'd share my storage setup. My storage requirements aren't like a 4k video editor's would be, but I have 3tb internal, 3tb external (getting a fourth terabyte for historic backup as part of this change), 6tb NAS and soon to be unlimited cloud backup. I'm going to be pretty pissed if I lose anything

 

[Michael73]

If you started out with 5 brand new drives in a Drobo 5D and then sequentially replaced one drive each quarter ad infinitum, from a statistical perspective, I would guess that it would dramatically reduce the chances of failure.

On a related note, I believe you can get a drive sled and use SSD drives in the Drobo 5D. From a reliability perspective, without moving parts, an SSD's chance of failure is very small compared to a traditional HDD.