Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

MP 7,1 Safest unsafe storage solution?

G

Grumply

macrumors 6502
Original poster
Feb 24, 2017
285
194
Melbourne, Australia
Hi guys,

Trying to sort out a local storage solution for a new 7,1 build (intended primarily for Davinci Resolve).

In my 5,1 Mac Pro I worked off the following setup:

- Boot/Applications Drive: 256GB 2.5" SSD
- Cache/4k Raw Files Drive: 4TB 2.5" SSD RAID 0
- Media Drive: 18TB 3.5" SATA HDD RAID 5 (4x6GB)

For the new build I want a lot more raw speed to deal with the increase in raw footage that I've been working with. So I was going to go all NVME M.2 (via 4x M.2 PCIe Cards), but as I've come to understand the importance of 2-bit MLC Nand for the volumes of data I'd be pushing around, budget has forced to me reconsider a few things.

I'd liked the idea of doing 4x M.2 SSDs in RAID 5 configurations, but it sounds like that doesn't really work well, and RAID 10 makes things too expensive, so I'm looking at simply having RAID 0 setups for my Media and Cache drives, and a separate boot/applications drive, but also adding an additional "Project File" drive built of two 2.5" SATA SSDs in a RAID 1.

I figure this will give me the most speed possible for actually working with my footage (all of which will be backed up externally to cheaper spinning drives). But gives me the security of having mirrored redundancy for all of my actual project files (so that anything structural to my grades/edits will be protected).

So the final layout will look something like this:

- Boot/Applications Drive: 1TB NVME SSD
- Cache Drive: 1TB NVME RAID 0 (2x 512GB Samsung SM961/960/970 Pro)
- Media Drive: 8TB NVME RAID 0 (4x 2TB Samsung 960 Pro)
- Project File Drive: 256GB 2.5" SATA SSD RAID 1 (2x 256GB Samsung Evo)

Does this sound like a reasonable compromise to speed + safety?

And is there anything else I'd need to move to the "Project File" drive for safety? Would my Resolve Gallery have to live there to protect the grades I've created for each project?

I'd appreciate any thoughts/suggestions.

Cheers
 
  • Like
Reactions: chfilm
How are you gonna install the sata ssds?
I would probably skip the project file drive and just do an overall daily clone backup to a spinning 8TB drive and be done with it..
 
The SATA SSDs I'd mount either via a PCIe card (like the Sonnet Tempo) or simply plug directly into the two SATA ports on the motherboard.

Are there cloning backup systems that will only update files that have changed? Or does it rewrite the whole drive every time?
 
  • Like
Reactions: chfilm
Both Carbon Copy Cloner and SuperDuper! do incremental back ups that only write data that has changed. They do, however, have to read through all the metadata of the source drive and destination drive to determine which files have changed, so there is some time required for that.

I wonder if you might want to consider putting your data in a ZFS file system, either within the Mac or as your destination? ZFS checksums both your data and metadata (APFS only checksums metadata), and makes it possible to send true incremental backups to a drive destination in a fraction of the time that rsync-based backup solutions require (Carbon Copy Cloner and SuperDuper! both use rsync). One caveat: booting off a ZFS file system is still pretty experimental, so I’d only suggest it for data drives.

Details on ZFS here:

OpenZFS on OS X

openzfsonosx.org openzfsonosx.org
 
Your storage requirements are very similar to what we desire for our filming and video editing workloads.

At this time we typically deal with client's camera data sizes of around 2TB. We also know that this is likely to just get bigger and bigger over the near future.

We are an Adobe exclusive shop with the odd FCP X use. Our projects can last for a week and up to six weeks, and we have to employ specialized freelancers at times.

We also go offsite to deal with some of our Projects.

We also will rent out our equipment at times when not being used in the office.

We have decided to embrace the new 2019 MP7,1 at this time and placed our order for one Dec 12th 2019.... but since have changed the order and now have a delivery of Feb 7-13, 2020.

Our current order is as follows...

Hardware:
  • 3.2GHz 16‑core Intel Xeon W processor, Turbo Boost up to 4.4GHz
  • 32GB (4x8GB) of DDR4 ECC memory
  • 1TB SSD storage
  • Radeon Pro Vega II with 32GB of HBM2 memory
  • Apple Afterburner card
  • Stainless steel frame with wheels
  • Magic Mouse 2
  • Magic Keyboard with Numeric Keypad - US English
  • Accessory Kit
  • AppleCare+
In addition to this order we plan to add the following items...

  • 8x 32GB R-DIMMs to the stock 4x 8GB R-DIMMs
  • Promise Pegasus J2i, and will have 8TB and 16TB HDDs
  • Add a Sonnet SSD M.2 4x4 PCIe Card [Thunderbolt Compatible] - (x16 bridge provides x4 PCIe 3.0 bandwidth to each SSD)
  • Install in the Sonnet card - 1x 2TB and 3x 1TB Samsung 970 Evo Plus NVMe M.2, expecting to get 5 GBytes/sec to 6.5 GBytes/sec with the Sonnet/Samsung setup as RAID-0.
  • Aproximate cost for these add-ons is just under $3,000 (includes sales tax)
  • We purchased from Amazon, Sonnet, Samsung directly, and the J2i was purchased from Apple. WE bought a few things from MonoPrice as well (a few cables).
  • We currently have various devices for bulk data storage; Promise Pegasus R8, R6, R4s and several OWC ThunderBolt2 4 Mini's (TB4m) each having 4x 5TB 2.5" Seagate Barracuda HDD and at times when required will use OWC 2.5" SSD SATA III units in them setup as RAID-0 for optional high external i/o demands.
  • Our Macs are interconnected with 10G fabric for file sharing.
Here's how we intend to use the storage space for the MP7,1...

  • Extra RAM - 8x 32GB R-DIMMs (all 12 DIMM slots will be used with this along with the stock 32GB), providing 288GB memory resource. A large chunk of this is to allow the kernel buffer cache to grab/consume, as needed, very large chunks of RAM to assist with fast i/o.
  • Extra 16TB Seagate HDD for the Promise Pegasus J2i provides 24TB SATA III internal bulk data storage. The stock 8TB Toshiba will be backed up to an 8TB partition on the 16TB Seagate, and remaining 8TB on the Seagate to be configured into 2x 4TB partitions for holding Time Machine backups and backups for the 4TB Sonnet RAID-0.
  • Sonnet SSD M.2 4x4 PCIe x16 Card with 4x Samsung EVO Plus blades; one 2TB blade and three 1TB blades. The 2TB blade will be split into two 1TB pieces; one being APFS and the other HFS+. The 1TB APFS will serve as a daily Carbon Copy Cloner (CCC) clone for the internal Apple 1TB SSD. The remaining 4TB (1TB on each of the 4 blades) will be setup as HFS+ and configured as RAID-0 with Disk Utility. The 4TB RAID-0 is for extremely high-speed i/o in the range of 4.5 GB/s up to 6 GB/s. For example, with a 100GB file in RAM wanting to be transferred to the Sonnet's RAID-0 will be possible to do in around 20 to 30 seconds.
  • Client camera raw data will be staged to the Sonnet/Samsung 4TB RAID-0, and backed up to the 4TB partition on the Seagate's 4TB space in the J2i ASAP before work on the new project starts.
  • The 8TB Toshiba in the J2i is for storing recent Project data and possible some current Client data. This 8TB will be backed daily to the 8TB partition on the Seagate's 8TB partition.
The overall goal is to have a system that is essentially self contained and requires no other external devices to haul around, with the single exception of a display. It also allows for easily having the MP7,1 ready for renting out when not in use in the office.

As we switch between Projects the current Project's working data will be backed up to external storage space and archived physically for safekeeping.

We also employ CrashPlan (Code42) for backing up important data to offsite location. This runs in the background and can be controlled to use little computing resources when we need the full power of the MP7,1 for the ongoing Project.

If we find the Sonnet/Samsung 4TB RAID-0 has insufficient space we can either upgrade the blade sizes and/or add another Sonnet/Samsung card.

Given the stock MP7,1 weighs in at around 40 pounds and with adding extra internal components the overall weight will be substantial and possible approach 60 pounds. For this reason we opted to purchase the optional wheels allowing us to easily move the MP7,1 around the office as needed at times, and without causeing bodily harm to employees.
 
  • Like
Reactions: dpzim
Grumply said:
....

So the final layout will look something like this:

- Boot/Applications Drive: 1TB NVME SSD
- Cache Drive: 1TB NVME RAID 0 (2x 512GB Samsung SM961/960/970 Pro)
- Media Drive: 8TB NVME RAID 0 (4x 2TB Samsung 960 Pro)
- Project File Drive: 256GB 2.5" SATA SSD RAID 1 (2x 256GB Samsung Evo)

Does this sound like a reasonable compromise to speed + safety?

....
Click to expand...


First, that "boot Applications" drive is the T2 drive of the Mac Pro 2019 or yet another PCI-e card?
In the latter case, that is a pretty high number of slots occupied (to point may impinge on other uses ).

Second, this is highly dependent upon what is being put into the "Cache Drive". If Resolve is copying media files there on the presumption that the primary media storage drive is "slow" (or at least "slower") then this is probably backwards. ( i.e., the RAID stripe width for the Cache should be higher than than the primary working store . )
If Resolve is more so using that as "scratch" space for intermediate working product that doesn't fit in memory then this would be OK ( as it is a new set of storage so not another faster place to get the data from). The issue there would be size of the intermediate timeline can stored on scratch (where that is now and were likely to go over next couple of years).

If SATA SSDs were working for the resolutions you are primarily working with now then may not need a RAID 0 there. A 1-2TB NVMe drive on a simpler card may work more than reasonably as well and be more cost effective ( 2-4x card holders are substantially a higher price that could be spent directly on storage. )

Older MLC that is "filled to the brim" doesn't necessarily have better endurance that TLC that has much lower relative working set size. ( e.g., 80+ % all the time versus 50% all the time). The 970 Pro is at a different level than the. It also varies with size 960 2TB isn't what the 512GB are. 970 EVO 2TB versus an
previous generations much smaller Pro is at least even match most of the time.

[ If the $/GB were better an Intel 900P or p4800x would work better as RAM memory backing store that getting written to excessively. About 6x the endurance of those 970 Pros but pay for that 'privilege'.
]


Finally, RAID 1 more so gives more operational "uptime" than "safety". You can tolerate a drive failure and just keep going on the project. A card like a Sonnet Tech Tempo Dual is pretty long. The Mac Pro 2019 slots 6,7,8 aren't that long. Putting it into slot 5 is a bit of a waste because that is a x16 slot (for a x4 card). If there is a no long term plan for MPX Bay 2 then have room, but if dense packing the MP 2019 with lots of other cards over time, that can turn into an issue long term.
 
deconstruct60 said:
First, that "boot Applications" drive is the T2 drive of the Mac Pro 2019 or yet another PCI-e card?
In the latter case, that is a pretty high number of slots occupied (to point may impinge on other uses ).
Click to expand...

The boot drive would be the Mac's internal T2 drive(s), yes.

deconstruct60 said:
Second, this is highly dependent upon what is being put into the "Cache Drive". If Resolve is copying media files there on the presumption that the primary media storage drive is "slow" (or at least "slower") then this is probably backwards. ( i.e., the RAID stripe width for the Cache should be higher than than the primary working store . )
If Resolve is more so using that as "scratch" space for intermediate working product that doesn't fit in memory then this would be OK ( as it is a new set of storage so not another faster place to get the data from). The issue there would be size of the intermediate timeline can stored on scratch (where that is now and were likely to go over next couple of years).
Click to expand...

The cache drive is meant to be just that - a drive for render cache files. In a RAID 0 to allow them to render out as quickly as possible.

The larger 8TB media drive is simply for holding the project's raw footage, audio, other assets, and feeding it into Davinci as quickly as possible. So ideally, none of the drives have to be both written to and read from at the same time.

deconstruct60 said:
Older MLC that is "filled to the brim" doesn't necessarily have better endurance that TLC that has much lower relative working set size. ( e.g., 80+ % all the time versus 50% all the time). The 970 Pro is at a different level than the. It also varies with size 960 2TB isn't what the 512GB are. 970 EVO 2TB versus an
previous generations much smaller Pro is at least even match most of the time.
Click to expand...

I'm less worried about endurance (or maximum speeds) than sustained speeds. As everything I've read comparing the different nand types, shows speeds fall completely off the cliff with 3-bit TLC nand as soon as the drive's cache runs over (which would happen all the time with large raw video files). Whereas 2-bit MLC nands are able to maintain much more consistent speed regardless of file size.

When moving large raw files around frequently, anything TLC or QLC-based sounds like a waste of time - the justification for spending so much on a machine is precisely so that it won't slow down. But that weakness is apparently embedded in the 3 and 4-bit nand technology.

deconstruct60 said:
Finally, RAID 1 more so gives more operational "uptime" than "safety". You can tolerate a drive failure and just keep going on the project. A card like a Sonnet Tech Tempo Dual is pretty long. The Mac Pro 2019 slots 6,7,8 aren't that long. Putting it into slot 5 is a bit of a waste because that is a x16 slot (for a x4 card). If there is a no long term plan for MPX Bay 2 then have room, but if dense packing the MP 2019 with lots of other cards over time, that can turn into an issue long term.
Click to expand...

I vastly prefer the idea of a Raid 1 mirror, to having to worry about running backups on a daily basis. For me, that's mostly about simplicity/laziness.
 
dpzim said:
The half-length x4 slot 8 is already taken by a short I/O card, but x8 slots 6 and 7 accept full-length cards like slots 1-5, no?
Click to expand...

Technically slot 7 is full sized. However the full size extension puts that slot in proximity of the CPU heat sink. If insert a 1.25 - 1.5 wide card into that slot then even closer to the CPU heat sink. That slot is better off with a 1/2 or 3/4 card.

But yes, slot 6 would work also if there is clearance from the neighbor slots (i.e., those stick to being more strictly just sub 19mm ).
 
With some further reading and spreadsheeting, I'm now kinda leaning towards the idea of going for a much simpler setup for the Media Drive, and putting a single high-density U.2 SSD (the Micron 9300 Pro 15.36TB or Max 12.8TB) on to a PCIe to U.2 card.

Being NVME I'd still have 3500MB/s read/write times (which realistically should suffice for even the biggest raw video files). And there's less of the stress that a RAID 0 adds to the equation.

Another option (at a comparable price) is 4x 4TB Samsung 860 2.5" SATA SSDs, in a RAID 0, on a Sedna 4x 2.5" SATA PCIe card. Which would give me 16TB, around 1800-2000MB/s read/write, and the advantage of MLC Nand - all for roughly the same price as the single 9300 U.2 drive, or 8TB of Samsung 960 M.2 drives + a 4x M.2 card.

Has anyone tried U.2 (via PCIe) in a Mac before?

And between the Micron 9300 Pro and 9300 Max models (which are identical, but for an extra 2.5TB being used for over-provisioning in the "Max" model, hence the lower capacity), can anyone speak to which would be more suited to Video data? How significant is the extra over-provisioning likely to be, when the drives primary purpose is to be read for colour grading and video editing (the write-intensive stuff will mostly happen with cache files to the M.2 RAID with MLC-based SSDs).
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back