Best drives for raid array for photo library

[jbg232]

Not sure if this should be posted in the photography or mac pro section but since it is more of a technical question I posted it here.

I'm planning on getting a new mac pro for photographic work that I do (in aperture 3). Currently I have a 1TB external hard drive which is almost full and my camera's images are each about 30MB. I'm thinking I need about 4TB of storage to last me 3 years based on my usage calculations for the last 3 years. I'll be getting an accelsior PCIe SSD for my main drive and active working project but will then be transferring the masters to the 4TB media array after done working on them. Both the system and the 4TB drives will be backed up regularly.

For the 4TB array I was thinking of 2 WD Black 2TB drives in Raid 0 as opposed to 1 4TB drive (I already have a backup solution so please do not mention this when replying). I was also thinking that for the performance it might be better to get 3 cheaper 1.5TB drives or 4 cheap 1TB drives.

Here's my question: What do you think is the best way to create this media array, ie, what gains can I REALISTICALLY expect in aperture 3 with utilizing a raid array? I really don't see the hard drives being a bottleneck past a 2 drive array but am wondering if anyone thinks differently. And a second question would be which drives would you recommend and why.

 

[lastmile]

Unless you're backing up very frequently I wouldn't recommend RAID 0 at all. If you want a more current project (or recent project) storage and don't want to pay the price for a larger Accelsior, get another SSD and stick it in the optical bay.

Below are some options and rough prices. I just did quick checks on Amazon and rounded off. Might want to double check my math on totals. And for RAID solutions you might get away with non-RAID drives.

1x 4 TB - If the drive goes you loose everything since your last backup (Hitachi = $300)

2x 2 TB drives - No RAID - you've got a backup solution and if you do loose a drive the other 2 TB are still there (WD Caviar Black = $400; WD Caviar Green = $275)

4x 1 TB drives - No RAID - same as above but you only lose 1 TB (WD Caviar Black = $450; WD Caviar Green = $400)

2x 4 TB drives - RAID 1 (software) - redundancy, but a long rebuild time during which your second drive could fail. But if you backed up recently you could copy any files not included in the last backup to another drive before starting the rebuild (Hitachi = $600)

3x 2 TB drives - RAID 5 (hardware) - drawback here is cost, it takes more high capacity drives, a hardware raid controller, and unless you go with the much maligned Apple RAID card you need one of those systems to mount the drives so that they can draw power from the logicboard but be connected to the RAID controller, or an external case (WD RE4 = $750, RAID controller = $500?, Drive mounting system/cables = $150? = $1,400)

4x 2 TB drives - RAID 6 (hardware) - same as above with one more disk (WD RE4 = $1,000, RAID controller $500?, Drive mounting system/cables $150? = $1,650?)

* Question for the forum - there is no software capable of RAID 5 or 6 right?

4x 2TB drives - RAID 10 (software) - redundancy and the speed you were looking for from a RAID 0 though again I'd go with a second SSD (WD RE4 = $1,000)

 

[deconstruct60]

* Question for the forum - there is no software capable of RAID 5 or 6 right?

Yes there is ..... kind of ..........

If asking for a solution specific to Mac OS X then it still isn't quite finished......

http://tenscomplement.com/

It is promising but this roll-out seems slower than the Intel Xeon E5 roll out.

However, there are several other much more mature options:

If fire up a box with Solaris/ZFS installed, it has been done for years now.

Linux has mdadm. ( https://raid.wiki.kernel.org/index.php/Linux_Raid)

FreeNAS has it ( and ZFS http://doc.freenas.org/index.php/Hardware_Requirements )

Some entry level NAS boxes use software RAID (because nothing else is running on those machines and capped by 1Gb Ethernet interfaces anyway. Often a customized variant of mdadm, FreeNAS or inspired by one of those two. )

RAID 5 (and 6 ) have a write hole. There is brief period where the drives are in an inconsistent state. If power is lost during that state they can screw up part of the drive. That typically why some folks advocated for slightly less power hungry device that can be powered long enough to put the disks into a consistent state and power off.

The other argument is that software RAID 5 is too computationally heavy for CPUs to do. ZFS has an assumption that there are multiple cores and not running in a maxed out core state.

 

[lnikj]

FWIW, I am also a Photographer (& hobbyist musician) and should the Mac Pro update arrive today I plan on the following solution.

A Crucial M4 256Gb system/apps drive, a pair of 1TB Velociraptors as RAID 0, and a 4Tb Hitachi 7K4000.

I currently have a MacBook Pro and keep my 1Tb photo collection on a 2TB WD Black (esata card) and use Synk (http://decimus.net/Synk) to keep that constantly mirrored to a 2Tb WD Green (FW800).

That has worked well so I would keep the Velociraptor RAID 0 array constantly mirrored to a partition on the Hitachi by the same method.

I figure that I don't need to keep more that 1TB of photos and a couple of hundred Gbs of audio samples on the Velociraptors at any one time so older stuff in the photo library as it grows would be shunted off to another partition on the Hitachi.

 

[deconstruct60]

I'm planning on getting a new mac pro for photographic work that I do (in aperture 3). Currently I have a 1TB external hard drive which is almost full and my camera's images are each about 30MB. I'm thinking I need about 4TB of storage to last me 3 years based on my usage calculations for the last 3 years. I'll be getting an accelsior PCIe SSD for my main drive and active working project but will then be transferring the masters to the 4TB media array after done working on them. Both the system and the 4TB drives will be backed up regularly.

The Accelsior is an odd choice. Is there a reason why? It is not likely to get the performance metrics on the OWC site moving around lots of jpegs or lossless compressed RAWs.

Also if working with referenced images
http://documentation.apple.com/en/a...:doc:uid:Aperture-UserManual-91292BSR-1038622

You can just put the RAW masters on the RAID drive in the first place. Then don't have to "move" them later when done. [ Or can move just the RAWs with images of the highly valued shoots to either a different location for longer term storage or copy/duplicate temporarily to faster disk. ]

For the 4TB array I was thinking of 2 WD Black 2TB drives in Raid 0 as opposed to 1 4TB drive (I already have a backup solution so please do not mention this when replying).

Well if this back-up solution is exactly the same size as the 4TB drive it doesn't really meet all the attributes of a back-up solution.

I was also thinking that for the performance it might be better to get 3 cheaper 1.5TB drives or 4 cheap 1TB drives.

The probability that the RAID array fails increases with more disks. The more disks you add the more failures you can get.

 

[jbg232]

Thanks everyone for the replies, my main question was related to WHICH hard drive (ie brand) you would get for the raid 0 to keep the cost reasonable but performance high enough.

My reasoning for not putting the masters on the ssd are simply that they will not fit (I have >30K images right now and had much more before.)

Anyone who uses aperture a lot knows what a performance pig it is. Editing images/importing/browsing etc require really intense hardware when working with 30MB raw files. I believe the active working project and the library should definitely benefit from being on the ssd but after I finish working on something, for the interest of space, I will move the masters over to the internal raid 0 where I feel they will still have decent performance if I want to go back to them.

Because the photos are referenced masters in my own folder scheme I use time machine with a 4TB drive to back them up and every now and then copy everything over to a separate drive for an offsite backup.