Is RAID Relevant Anymore

[Average Pro]

I've spent the last few days outlining how to upgrade/replace external hard drives. The one thing I can't answer is if I really need a RAID (1, 5, 10, etc) anymore. Memory is so affordable, that I 'm thinking of doing away with RAID configurations and replacing them with backups instead.

Here is the proposed 4-bay enclosure configuration:
Disk 1-2 (RAID 0) Yes, I know I said no more RAIDs, but Disk 1-2 are for speed.
Disk 3 (back up of Disk 1-2) Automatically backs up anytime Adobe/Phase One closes.
Disk 4 (back up of Disk 3) Automatically backs up after each X day/week(s).

There is another separate backup of Disk 4 which runs on a monthly basis and gets stored at a separate location.

Can anyone explain how running a RAID (1, 5, 10, etc) would be beneficial?

Thank you for your time.

 

[kenoh]

Lol. RAID is relevant still. RAID is for fault tolerance not just performance. At the risk of teaching you to suck eggs and I apologise in advance if I am... The R means redundant - thats the fault tolerant bit...

 

[Average Pro]

True. Using car tires as an analogy...
Tires are cheap, so instead of using run-flats (which will keep the car running, but slower if a tire gets damaged), I have two new spare tires ready to go.

 

[kenoh]

True. Using car tires as an analogy...
Tires are cheap, so instead of using run-flats (which will keep the car running, but slower if a tire gets damaged), I have two new spare tires ready to go.

Yes, the idea being you can use the spare while you replace the punctured one....

Being careful not to break the spare until you get it repaired...


Then the different RAID levels are about balancing the risk and getting more or less fault tolerance with a trade off against storage capacity used to enable that tolerance. For example in an enterprise storage array you can lose multiple disks and be able to recover the data by hot swapping the dead drives with new ones, then the RAID setup is able to rebuild the data and structure that was on the drives that died.

Arguably beyond mirroring, a little overkill for a home user.

 

[Allyance]

When I was building servers fault tolerance was more important on the server, where down time was critical. In those days there was no "Cloud" to backup your data, for small business users it was a tape backup system. Today there are a multitude of cloud storage sites that would far more economical than a RAID backup drive. You could even have 7 different sites and use a different one every day if you were really paranoid!

 

[Menel]

I feel like for home/small business use, RAID is toast. But its really user to user you gotta decide for your own situation.

If you want speed, RAID stripe is only a stop gap, just go all the way to SSD.

If you are needing redundancy, RAID mirroring. But mechanical drives have gotten way more reliable if you are picky on make model. But still doesn’t cover fire, theft, virus corruption, user accidental deletion... its expensive and ONLY covers a drive mechanical failure... lot of expense to,cover only 1 failure mechanism.

With fast internet widely spreading, cloud or,offsite redundancy is so much better. But that varies. A good chunk of my family, across 3 different states, 3 different ISP, and 5 cities have $60-70 1000/1000 fiber. Just setup scheduled sftp scripts to sync. I know its not ubiquitous, but its starting to feel like it.

Then RAID5 just helps with storage/cost scale efficiency.

All have difficulties id rather avoid.

Definitely all of my software oriented buddies where data isn’t too massive.... carbonite/mozybbackblaze etc cloud.

If you have massive video libraries... trickier.

 

[Average Pro]

Kenoh,
I agree with your (as well as respondents) points. An no worries about the "suck eggs" comment. It did make me laugh as I haven't heard that since grade school. All is good.

When I started this, the priorities were speed and back ups. RAID is not a back up solution because if you delete a file or the data gets corrupted, it applies that across the entire RAID. I don't require SSD type speeds as I'm working with photographs. I'm still debating whether to work from:
- RAID 0 (with multiple backups) or
- RAID 5 (with multiple backups)
[doublepost=1511566974][/doublepost]I should add that I'm not interested in cloud storage.

 

[ChrisA]

..

Can anyone explain how running a RAID (1, 5, 10, etc) would be beneficial?

Using your scheme if a disk fails you loose either hours or days of work. With RAID if a disk fails you continue to work without stopping.

RAID 0 has exactly 1/2 the reliability of a standard disk drive. It is pretty much a data loss waiting to happen. But it might be a little faster. RAID 5 is very reliable especially if you set up a double parity disk. Then you can have two failures at the same time which is VERY unlikely. Raid 0 and 5 are polar opposites

You should look into cloud storage for backup. For $5 per month you get continuous off site backup. Of if your house caught fire and you had to run out you would not loose any data. continuous off site is hard to do one your own for the $5 price.

SSD might not work for you if you have a lot of data, But if your library will fit on a 1TB drive. This is the way to go. It is very reliable and very fast.

 

[Average Pro]

Chris,
Good point. However the only data lost is what I was working on at the given time. Recall that the backup automatically occurs when the program closes.
I could immediately switch to the back up if under a time constraint. Then remove the affected HD (from RAID 0), insert new drive and rebuild RAID 0.

 

[ChrisA]

Chris,
Good point. However the only data lost is what I was working on at the given time. Recall that the backup automatically occurs when the program closes.
I could immediately switch to the back up if under a time constraint. Then remove the affected HD (from RAID 0), insert new drive and rebuild RAID 0.

But you have to "do stuff". With RAID you can put off doing anything

Ys, in your case it hardly matters as you are just one person. Where RAID starts to matter is when you have dozens of people doing work on the system. When dozens of them have to roll back to a backup and likely need help doing so.

 

[Average Pro]

Good point.

I think I'm headed towards RAID 10. I'll get faster and more reliable performance than RAID 5 and won't require a 3rd party software solution since RAID 10 can be setup via the macOS Disk Utility. The requirement for larger HDDs won't be an issue as prices are quite reasonable.

Backups will spin off the RAID 10.

Thanks for the advice and input.

 

[fcortese]

Using your scheme if a disk fails you loose either hours or days of work. With RAID if a disk fails you continue to work without stopping.

RAID 0 has exactly 1/2 the reliability of a standard disk drive. It is pretty much a data loss waiting to happen. But it might be a little faster. RAID 5 is very reliable especially if you set up a double parity disk. Then you can have two failures at the same time which is VERY unlikely. Raid 0 and 5 are polar opposites

You should look into cloud storage for backup. For $5 per month you get continuous off site backup. Of if your house caught fire and you had to run out you would not loose any data. continuous off site is hard to do one your own for the $5 price.

SSD might not work for you if you have a lot of data, But if your library will fit on a 1TB drive. This is the way to go. It is very reliable and very fast.

It is timely that this thread came up. My LaCie 2big6T RAID 5 experienced my first ever HD crash on one of the drives. It houses ALL of my photos. Simply pulling out the crashed HD and using the mirrored one resulted in no lost time. A call to LaCie and a replacement HD, free of charge since it was still under warranty, arrived yesterday However, I do have multiple redundancies with back ups to 2 different external HDs using Carbon Copy cloner. One on site in my house the other kept off site at my office. I must admit I am a littel bit paranoid becuase I do use an off site cloud service as well but at $60/yr it translates into 4 or 5 less Starbuck coffees a year. So IMO RAID is still alive for those who want to use it.

 

[OreoCookie]

Short answer: no, RAID is not obsolete, and if you need more storage than one hard drive can provide you should use a RAID level other than 0 so that you have protection against hard drive failure. In many cases, the point of a RAID is to provide not just protection against data loss (which can be mitigated by using backups), but be able to work without interruption. For businesses this is crucial. Don't cheap out on that and bet on the fact that it is more likely than not that your data won't fail — that is not a good idea.

Regarding failure rates: Newer hard drives have an annual failure rate of between 1-2 %. So if you run 4 hard drives for 4 years with an annual failure rate of 1.5 %, i. e. a hard drive will not fail with a probability of 98.5 %. Sounds very high, doesn't it? If you don't have any redundancy, the probability that something goes wrong with at least one of the hard drives within 4 years is actually 100 % - 98.5 %^16 = 21.5 %. If the failure rate is 2 %, then this probability rises to 27.6 %.

 

[QuantumLo0p]

RAID is still very relevant to performance; if you need it. SSD's excel for their intended purpose but fall flat on their faces when tasked with writing large amounts of data. SSD's caches are truly fast while the flash memory of SSD's is still very slow. Software such as Samsung's Magician with Rapid Mode helps but only a little. In the end you will be limited by the cache on the SSD.

 

[OreoCookie]

RAID is still very relevant to performance; if you need it. SSD's excel for their intended purpose but fall flat on their faces when tasked with writing large amounts of data. SSD's caches are truly fast while the flash memory of SSD's is still very slow. Software such as Samsung's Magician with Rapid Mode helps but only a little. In the end you will be limited by the cache on the SSD.

You are probably thinking of a performance characteristic that is specific to SSDs with TLC memory chips, which are found in cheaper, entry-level SSDs. And even in those, the problems occur only if you have sustained writes >2-12 GB (the DRAM cache size depends on the model). SSDs with MLC or SLC memory chips are not as susceptible. For example, one of the best SSDs on the market, Samsung's 960 Pro NVMe SSD with MLC memory writes a 20 GB file in 11 seconds, that is 1,800 GB/s. Note that this model comes with 512 MB-2 GB DRAM cache, so 20 GB doesn't fit in it. This is far beyond the capabilities of spinning platter hard drive-based RAIDs (unless they have an SSD cache or so, of course).

Moreover, RAID5 and its variants have their problems with writes, the well-known write hole: the transfer speed at which you write is slightly less than the speed of the slowest hard drive in the RAID5. And that is surely well below that of an SSD. Add to that the much lower latency of SSDs compared to spinning platter hard drives.

Therefore, it makes no sense these days to advise against using SSDs on performance grounds: transfer rates are an order of magnitude faster, access times are more than an order of magnitude faster, and even in the worst case scenarios, an SSD will still outperform a RAID. After all, this is why SSDs are frequently used as caches for RAIDs, precisely because they are much faster.

You want a RAID or spinning platter hard drives if you need a lot of storage and don't want to sell your kidney to get it. And there are still plenty of applications where the speeds offered by RAIDs and traditional hard drives are plenty, so the price hike would not make financial sense.

 

[ChrisA]

You are probably thinking of a performance characteristic that is specific to SSDs with TLC memory chips, ....

...You want a RAID or spinning platter hard drives if you need a lot of storage and don't want to sell your kidney to get it. And there are still plenty of applications where the speeds offered by RAIDs and traditional hard drives are plenty, so the price hike would not make financial sense.

I'll second the above. It is spot-on. But I can add that one application for spinning disk RAID is exactly what the OP wants, backup space for an entire office. For this use case speed hardly matters the spinning disks are more than fast enough. What matters the most is reliability.

What I would recommend is to buy TWO RAID boxes. Keep one on the local network and use it for Time Machine. Then place your second RAID box in some remote location that has a good Internet conation and tell the second box to keep continuously synchronized with the box. Synology has an easy why to set this up. Now if the first one fails or is stollen you have an offsite backup. You also have spare parts. It is important to keep the second box off site. This protects from fire or theft.

Do remember to setup full disk encryption on the RAID.

Yes you can also backup the RAID to a cloud provider and you might want to do this but owning your own second RAID allows for very fast recovery and a spare box if your primary backup chassis fails.

That is one thing about RAID, the hardware failure is the box itself. I've have two of them fail. Eventually the power supply of something goes. It is good to have spare parts

 

[OreoCookie]

I'll second the above. It is spot-on. But I can add that one application for spinning disk RAID is exactly what the OP wants, backup space for an entire office. For this use case speed hardly matters the spinning disks are more than fast enough. What matters the most is reliability.

What I would recommend is to buy TWO RAID boxes. Keep one on the local network and use it for Time Machine. Then place your second RAID box in some remote location that has a good Internet conation and tell the second box to keep continuously synchronized with the box. Synology has an easy why to set this up.

Agreed. For backup purposes I recommend a quality NAS box from a vendor such as Synology or QNAP. I love my Synology, and can only recommend it. It comes with software that does exactly what ChrisA says. Here, transfer speeds won't matter too much because you will be limited by the networking connection. And if a lot of people access your NAS, you can put NVMe SSDs into the newer ones to act as a cache.

 

[pmxperience]

I use a RAID0 setup in my 2012 Macbook Pro. Having a 1Gbps I/O across two SSDs is absolutely invaluable to me. I wouldn't be able to keep up with my own pace, otherwise.

 

[QuantumLo0p]

You are probably thinking of a performance characteristic that is specific to SSDs with TLC memory chips, which are found in cheaper, entry-level SSDs. And even in those, the problems occur only if you have sustained writes >2-12 GB (the DRAM cache size depends on the model). SSDs with MLC or SLC memory chips are not as susceptible. For example, one of the best SSDs on the market, Samsung's 960 Pro NVMe SSD with MLC memory writes a 20 GB file in 11 seconds, that is 1,800 GB/s. Note that this model comes with 512 MB-2 GB DRAM cache, so 20 GB doesn't fit in it. This is far beyond the capabilities of spinning platter hard drive-based RAIDs (unless they have an SSD cache or so, of course)....

I completely for got about the newer M.2 ssd's! However, based on non manufacturer tests, many older ssd's do indeed suffer from the cache overload and at much less data than 2-12GB which you speak of; 300MB+... A quick search returns a lot of real world test data.

 

[Razeus]

I just got my QNAP 451+ up and running over the last week with WD's 8TB Red drives (shucked from Easystore externals from Best Buy).

RAID is highly relevant. I run RAID 10 (though I may switch to RAID 5 for more storage).

YOU STILL NEED BACKUPS.

 

[MCAsan]

Curious as to what prompted the original question. What OS change or new device replaces the various RAID stacks?

 

[OreoCookie]

I completely for got about the newer M.2 ssd's! However, based on non manufacturer tests, many older ssd's do indeed suffer from the cache overload and at much less data than 2-12GB which you speak of; 300MB+... A quick search returns a lot of real world test data.

Yes, and I linked to real world test data.

Even consumer-grade SSDs are on par with hard drives under the most favorable conditions for the latter. As soon as you add random seeks to the mix, the SSD will vastly outperform spinning platter hard drives. Especially if you stick to quality SSDs from manufacturers such as Samsung or Intel.

RAIDs are no longer the weapon of choice for performance, it is about volume size and redundancy.

 

[MCAsan]

Let me know when you find a 8TB SSD for the same price as an 8TB HDD.

 

[OreoCookie]

Let me know when you find a 8TB SSD for the same price as an 8TB HDD.

Yup, I forgot the obvious one, price, in my last post.

 

[ChrisA]

Let me know when you find a 8TB SSD for the same price as an 8TB HDD.

You would like never need an 8TB SSD for still photos. You only need fast access to thrones you just shoot and are working on. a 500GB SSD would work, then you move the processed data to a RAID archive.

But if you are shooting a lot of video then SSD might be worth it. Yes 8TB of SSD is expensive what what did it cost you to shoot the video? Video is very expensive to shoot because of lights, location, the people . The labor cost to shoot is far more than the storage cost. But even so FCPX lets you edit low-res proxies so an HD is fast enough