I have massive photo libraries, several TB's worth. I do a lot of amateur photography with my Canon 5d Mk II. As you can see from the 2 snaps below, I also Print with a wide format 12 ink system. The results are stunning, yet amateur. This is a 22meg pixel full frame digital camera. With these large file sizes, transfer times get a big long, especially when the 16GB Compact Flash card is full. The bottleneck being USB 2.0.
I'm still not sure of what your current capacity requirements are or what your growth rate is.
For example, sit down and figure out the avg. file size of the pics you're shooting (personal + business), multiply by the number of photos per year, and then multiply by say 3 (i.e. constant rate of capacity growth, planned for 3 years).
From the bits of information you've posted, it actually appears to be the following:
Avg. File size = 60M
Avg. # of files per year = 1000
Then you get 60M * 1000 * 3 = 180GB over a 3 year period. But I'm not certain on this (no idea if you've any growth or not, or if I've under-estimated the annual # of files generated). So the math could be inaccurate.
Granted, 180GB isn't that much in terms of capacity, so it probably won't matter if say the annual # of files is 5x that (still under 1TB over 3 years).
Now it might not seem like much, but with RAID, the smallest details tend to be critical. Not just in getting the right hardware, but there are significant costs involved with hardware solutions (hitting $5k for a modest
DAS isn't that hard to do).
This is why it's in your best interest to sit down and really think about this carefully (your wallet will thank you, as in uncertain instances, the "safe" thing to do is over-estimate to cover variances, and that translates into real money). Particularly disks when a hardware RAID card is used, as they're more expensive than their consumer counterparts (by more than 2x on the largest capacity disks for example). It adds up. Quick.
I then use various programs to post process and convert to TIFF files for printing. Files sizes reach to 60Meg although I have had a few in excess of 120Meg. As you can imagine, write times to single 2TB hard disks that store these files gets rather long, so the hard drives themselves act as another bottleneck.
If these applications are anything like Photoshop, the storage system isn't as big a bottleneck as you might think (go back and re-read what happened with
lbeck's implementation - he got carried away, when the limitation was how Photoshop handles reads and writes = single threaded operation).
I recently purchased the Adobe 5.5 Suites for windows and Mac. However I have been using Canon RAW software and XARA Designer Pro software which are great products in their own right, but once you hit a certain level of creativity, well it seems all roads leads to Photoshop. Essentially I have hit my creative limit with XARA Designer Pro and need more tools available to me, hence I am starting classes on Photoshop in a week and will be integrating Photoshop into my post processing work flow. It was Photoshop, that got me looking for a new computer (because I heard how slow it can be from XARA which is slow enough when your pushing lots of files) and or enhancing the one I have. So I choose to enhance my 06 MP 1,1 and wait until Apple upgrades it's Mac Pro platform so as to include native I/O improvements.
Well, the 2006 already has limits on the OS X side (EFI 32 based firmware, so graphics cards are already limited, as will be OS X once it goes to a 64 bit Kernel exclusively), but will last longer as a Windows machine (will still be able to stuff in a PC graphics card, and continue to upgrade Windows).
As per USB limitations, that's solvable with an inexpensive 3rd party card (not as fast as it could be due to the PCIe 1.0 spec, but a bit more than 4x faster than USB 2.0). Should be quite an improvement for you, without costing an arm + leg + first born.
I was actually going to buy a $5500 2010 12 Core 32 Gig RAM 8 TB's of Baracuda's with a USB 3.0 card and 2 60GB SSD's plus a Quadro 4000 card - but I could not pull the trigger knowing native USB 3.0 and SATA 6G was missing.
You wouldn't be able to use that much machine. Photoshop is limited to 2x cores. Filters/effect layers can push it up a bit (i.e. 4x cores), but you won't be able to use all 12, so it's a waste of money.
You'd be better off with an SP Hex, as it not only has a fast clock which is good for single threaded or limited threaded applications, but you can run other portions of CS5.5 that can use more cores if you ever begin to use those (i.e. After Effects or DVD encoding via Media Encoder). The audio applications do as well, but I presume you don't do anything with sound ATM.
Even if you buy a new machine now, you'll still need to add in upgrades like memory, storage, and a USB 3.0 card. All of which are cheaper via 3rd party suppliers.
So it comes down to are you willing to wait until say ~ March 2012 for the next systems, which are likely going to cost more money, or go ahead and get a faster system now. Which BTW, still will not contain USB 3.0 (they will have a couple of SATA 6.0Gb/s ports). Intel isn't putting USB 3.0 into their chipsets yet (intentional delay), in order to help get TB adopted.
Another thing to keep in mind, is OS/applications disks rely on random access read performance, which isn't anywhere near able to saturate SATA 3.0Gb/s, let alone 6.0Gb/s (i.e. random access read performance is typically under ~ 70MB/s on a fast SSD, yet 3.0Gb/s is good for ~275MB/s of sustained throughput). Far lower than the sustained sequential read speeds you're seeing (over 400MB/s figures).
BTW - I have 4 new 2.0 TB WD Caviar Black WD2002FAEX. Still in wraps. Same for my new Crucial M4 256Mb SATA 6G 2.5" SSD.
Given what you've listed so far, I don't think you even need a hardware RAID card.
Currently, I'm thinking of the following:
- ODD Bay 2 = SSD for OS/Applications disk
- ODD Bay 2 = 2nd SSD for Photoshop scratch
- HDD Bay 1 - 4 = 4x Caviar Blacks in a RAID 10 configuration via Disk Utility
Cheap, doesn't need a RAID card or enterprise grade disks, or special adapters (save the ODD bay if you don't want to DIY something). Speed is that of a 2x disk stripe set (RAID 0), but also has an n = 2 disk redundancy.
Toss in a USB 3.0 card for your CF card reader, and an eSATA card for connecting a backup system (green drives are perfect for this duty), and you're done.
So if I am reading you correctly you're suggesting that I purchase an external RAID enclosure, perhaps 4-6 drives and then purchase enterprise level drives, and then use the MP's 4 internal bays as the RAID backup, but that the 4 internal bays would need to be set up using the MAC OS software raid solution? Would it not be better to backup to single drives, or would I simply mirror the external RAID?
I suspect you don't need this at all from currently available information. This could change, but I'd need more information in order to shift my way of thinking.
Now if you were burning through more capacity than can fit inside a MP, or needed throughputs faster than what 4x ports can deliver, then Yes, you'd need a hardware RAID solution. So if this is the case, I need to know.
