Mac Pro Best Drive Configuration

[kschendel]

I litterally don't get why people even tinker with stuff like Fusion Drive.

The answer is simple, and has been stated repeatedly, but I don't mind posting it again:

Money. (and to a lesser extent, time.)

Given sufficient money, it's a no-brainer that pure SSD in a PCI slot is the best performer. That sort of thing isn't always in the budget. For someone cost constrained, or someone trying to make best use of existing bits and pieces, the Fusion drive is a reasonable attempt to make the best of things. It will work really well in some usage scenarios and will work poorly in others. In those cases where the Fusion algorithms don't work out well, one can try manually separating files into SSD files and HD files, but that costs time and effort which a user may not care to expend.

 

[owbp]

I get why would somebody want Fusion drive, but for me the safe way was always SSD for OS and some apps, Raid 0 HDDs for User folder, user specifics apps and general work. Next to that I have one HDD for storage and one for Backup.
Reading this forum gave me dozen ideas of how to fill out MP and how to utilize different drives, but for now this is my cost effective way.

 

[h9826790]

NMVe Fusion Drive on a cMP? Are you sure you want to do that? It sounds protential problems more than benefits.

 

[usna92]

Well I have built not one, but two fusion drives to attempt to solve a couple problems on my network. Using the instructions here: https://bombich.com/software/files/tools/Creating_a_Fusion_volume_with_a_Recovery_HD.pdf
As well as a review of https://blog.macsales.com/15617-creating-your-own-fusion-drive helped me both create a recovery partition and create a couple of fusion drives. The first was a 256GB SSD with 750GB HD to use as the main boot drive, user accounts, etc. The second was a 60GB SSSD with 1TB HD to serve as random storage. Creating both of those will still allow for an internal raid to house the network iTunes/plex library and other long term storage. The raid will be back up to external storage and eventually offsite. I will say that diskutil is your friend, but you need to be careful and patient, because making a mistake with diskutil can and will end your day. I did have some issue repurposing the MacBook Retina SSD, but that was because of the partitions that already existed on the drive, and that required a boot into recovery mode to ensure I could whack those from the terminal. I haven't yet benchmarked the new system. I am still doing some long term copying, but it is definitely faster than just a HHD alone, and I know it is not as fast as it could be with SSD alone, but right now seems like an interesting compromise.

 

[JedNZ]

Some of my questions remain unanswered (until my M.2 NVMe SSD arrives). If anyone has real world experience and answers for these questions that would be great - takes away the uncertainty:

1. Can I put a FD together for my User data using a M.2 NVMe SSD (like the Samsung 960 EVO 256GB I've ordered) and a 3TB HDD? It's only for User data - my actual boot partition is on a Samsung 850 Evo 500GB in a Accelsior S SATA III PCIe card. -- I'm satisfied I can do this based on comments above and from other sources.
2. If so, would 250GB be big enough? Or should I go for a 500GB or larger model M.2? -- Think I'm going to have to test this myself.
3. What speed difference should I experience? Negligible, moderate, huge? -- ditto - will wait for the M.2 SSD to arrive and will report on my experiences.

 

[usna92]

I am not sure what your speed will look like, I will benchmark my configuration this evening after I am done shifting things around. I do know that performance will get slower on average over time as the drive fills up. Just the nature of a fusion drive. I think anything larger than 256MB and you are really just diminishing your returns with a fusion drive. I think a 500GB drive is sufficiently large enough to stand alone. I just didn't have one on hand, which is why I went fusion drive with the smaller SSDs I did have. To be honest, had I been thinking, and thought this out a little, I maybe should just connect all the SSD's I have lying around in a JBOD and just treat them all as one drive rather than screw with a fusion set up.... Something to consider in the future maybe....

 

[Squuiid]

So I've got my macOS on the Accelsior S (Samsung 850 EVO SSD), and my main User data is still on a 3TB Seagate ST3000DM001 7200rpm. I have another admin account on the SSD for troubleshooting, or if the 3TG fails or isn't connected when I boot.

A word of advice, regularly backup the data on that ST3000DM001 3TB or avoid using it if at all possible. I've had 6 of these disks die on me and I'm certainly not alone in that. The Seagate in-warranty replacements also died eventually. These disks are utter garbage. See links below for more info:
https://en.wikipedia.org/wiki/ST3000DM001
https://www.hbsslaw.com/cases/seagate

I've also found Crucial SSDs to be hit and miss. I, and many others had a lot of problems with their MX100. The 850 EVO is a solid choice, you did well to swap over to it despite the lower capacity.

 

[owbp]

What a bad timing, I have just bought that Seagate because of great experience and longevity of 1tb Barracudas. Didn't know about that...

 

[Intell]

What a bad timing, I have just bought that Seagate because of great experience and longevity of 1tb Barracudas. Didn't know about that...

Not all Seagate drives are bad. Most are rather good, that particular drive was very bad.

 

[F00l_0n_the_h1ll2]

So my statement is not answering any of your FS questions, but i have a very snappy experience with having the OS on one internal SSD and the main user on an other. I do not believe, that you could not reduce the users load so it would fit to a SSD you could afford and just put the itunes- and the foto library on a fusion drive or a HDD, as i did. The working environment is more than fast enough and for working on my ≥900GB foto library the internal HDD access seems to be sufficient.

I have no SSD installed in the PCI-e slot to gain extra speed, but i´m thinking about it, as i got an new, unpacked Samsung 850 Pro SSD 512GB for half of he regular price coincidentally. I only cut simple HD videos(1600x1080) from time to time - i do not have to handle 4K videos all the time, as you might...
Also if you are a photographer it might be different,as you need to transfer a lot of raw-images, as your daily threath...

So intense & accurate planning and scaling might help more than hardware-updates and fusion drives.
For me it is easy: most used data -> the OS and the main user on separate SSD´s work very well; because i use 2 independent channels. RAM usage is pretty advanced in Yosemity and El_Capitan, so a lot of caching is happening automatically. If you have a full load of RAM you will profit way more in performance, than you could ever with a fusion drive. No scientifically reports i could provide for this - just decades of personal experience.

 

[AidenShaw]

Not all Seagate drives are bad. Most are rather good, that particular drive was very bad.

Agree completely. I only buy Seagate and HGST drives, you'll never see WD in one of my systems unless it was the only OEM choice.

But, the ST3000DM001 was utter garbage. But only that particular model. After many DMA's, Seagate replaced all of them with ES3 3TB models which have been fine.

It happens. Chevrolet has made a lot of decent cars, but they also made the Vega. Utter garbage.

 

[owbp]

But, the ST3000DM001 was utter garbage. But only that particular model

Yes, that's the drive I've bought after using ST1000DM003s for years and being impressed with their reliability, silence, temperatures and speeds. Well, you live and you learn...

 

[JedNZ]

Bugger. Mine is a ST3000DM001-1ER166. But I've put it through a bit of stress for a couple of years now and it hasn't failed yet, so fingers crossed. Will ensure SMART is checking it regularly.

As far as an alternative to a FD goes, I'd rather not split my User data (Photos, iTunes library, Videos/Movies etc) across different SSDs. First, I don't want to have to spend any more money on purchasing more SSDs. Second, I've used FD a bit in the past and loved the speed boost it gave (iMac 9,1 - Mac Mini 4,1 - cMP 1,1>2,1). I haven't experimented or used RAID (1,5, JBOD etc) before, so I'm nervous about going down that route (even though there are similarities with FD technologies).

Anyway, I'm give it a try out and see how it goes when my M.2 SSD arrives.

 

[h9826790]

Some stat about HD failure rate. Obviously the sample size vary a lot between different models, therefore not really a good / fair comparison. But still interesting to see that the Seagate ST1500DL003 has a failure rate that >90% .

 

[JedNZ]

The Samsung 960 NVMe has arrived. But I can't get my cMP to recognise it.

I used the git (?) method to create a NVMe kext for macOS 10.12.3 (which matches my system).
https://github.com/RehabMan/patch-nvme

All confused now as to what to do. Grr!

 

[JedNZ]

I managed, somehow (can't remember how I did it now!), to actually get my cMP to see and mount the Samsung 960 - formatted it, tested read/write speeds (R ~1850, W ~1500). Then my cMP crashed, and kept crashing on startups. Reinstalled the macOS 10.12.3, tried some more things, and Crash crash crash!! Booted into my other internal and external drives in order to fix my main boot drive - finally fixed my working OS volume. But the Samsung 960 is nowhere to be seen.

I've searched and read an awful lot of info on this subject to the point nothing is making sense to me. So many sites talk about Clover and Hackintosh. I'm using a cMP 4,1>5,1 and need a detailed, step-by-step solution to sorting this out. My Samsung 960 will only contain User data - it's not a boot volume - so I'm not trying to do reinvent the wheel. Any help or guidance appreciated, particularly for anyone who has made it work on their cMP.

 

[h9826790]

No, you are not inventing the wheel, but not far away. Someone has already written the driver for NVMe SSD, however, from my memory, users reported not stable (and yes, I only talk about user data NVMe SSD, not the bootable one).

If you end up can achieve stable NVMe usage in cMP, and use that to make fusion drive. You are the very first one to achieve that in this group (in fact, the first one I know in this universe). And I think you will be the only guy that can provide the step by step guide.

That will be a good news to most of us if NVMe is a stable high speed PCIe SSD solution on the cMP (data / scratch disk only). It seems the driver's development is already stopped. So, unless Apple provide the stock driver in the future OS update. I doubt if we can possibly achieve to a level that NVMe SSD is considered stable to use in cMP. And this is why I said in post #28, you may have trouble more than benefit.

 

[JedNZ]

Okay. Finally got it working (again). A Samsung 960 EVO 500GB NVMe SSD in a PCIe adapter.

Steps:
1. Disable SIP by booting into your Recovery Drive (Command+R at startup) and select Terminal from the Utility menu. Enter this command line: csrutil disable; reboot
2. Curate an NVMe kext using the GITHUB process - instructions can be found here. The version I made is specific to my MacOS - 10.12.3 - unknown if this will work in subsequent versions of macOS, or future versions. The file that was output is called: HackrNVMeFamily-10_12_3.kext
3. Use Kext Utility (download site here) to install the above created kext. So simply drag the kext file to Kext Utility - it will place it at System > Library > Extensions - will also fixed permissions and flush caches etc.
4. Navigate to System > Library > Extensions and remove (backup a copy somewhere) IONVMeFamily.kext
5. Update: Step 5 is not needed. The HackrNVMeFamily-10_12_3.kext only needs to be in System > Library > Extensions. [Original step: Option+Drag (copy) the HackrNVMeFamily-10_12_3.kext to Library > Extensions manually. Then use Batchmod (or other utility) to set the permissions to root:wheel ]
6. Delete IONVMeFamily.kext if it exists at Library > Extensions
7. In Terminal, type this command: sudo kextload (include a space at the end) then copy the HackrNVMeFamily-10_12_3.kext file from System > Library > Extensions . The final command will look like this: sudo kextload /System/Library/Extensions/HackrNVMeFamily-10_12_3.kext
8. Check Disk Utility to see if the NVMe drive is listed. If it hasn't been formatted you may get a dialog box inviting you to format it. If not, then do a Restart
9. On Restart my NVMe showed up on the Desktop (I had formatted it from an earlier attempt).

I have my Samsung 960 EVO 500GB NVMe in PCIe slot 3. I'm getting Read: 1400MB/sec, and Write: 1500 MB/sec.

Questions - can you help:
Q1. The Samsung 960 EVO is capable of Read: 3200 MB/sec, and Write: 1900 MB/sec. Anyone know how I can squeeze out more of that speed? Partial answer: Theoretical maximum speeds in PCIe slots 3 and 4 mean you can only get up to 2000MB/s as they are only x4 (4 times 500MB/s). Not sure I can better my results, but ideas welcomed.
Q2. Can anyone confirm if I moved the PCIe card to PCIe slot 2 (x16) would I get faster speeds? Answer: Does not achieve better results. In fact, a known (so I've read) bug means you actually get half the speed!! not twice the speed. Leave the NVMe PCI adapter in PCIe slot 3 or 4.
Q3. Anyone know how to update the RecoveryHD to include HackrNVMeFamily-10_12_3.kext so I can mount and access the NVMe FD? I used this method but the NVMe doesn't register (doesn't mount, doesn't show in Terminal using diskutil list or in Disk Utility etc.

Next up/to do:
a. I'll enable SIP again and see if it still works. Works!
b. I'll create a Fusion Drive with my NVMe drive and a 2TB WD Black Caviar. Works!
c: Create a User account on this and see if I can logon to that User account. Works!
c. I'll then dd (copy) my entire 2TB of User data onto the FD and see if that works. Works!
d. Restarting holding the Shift key down - no problems and doesn't affect the kext.

Even shows up correctly in System Information NVMExpress, with TRIM support.



Thanks to h9826790 for the encouragement to do this.

 

[h9826790]

Questions - can you help:
1. The Samsung 960 is capable of Read: 3200 MB/sec, and Write: 1900 MB/sec. Anyone know how I can squeeze out more of that speed?
2. Can anyone confirm if I moved the PCIe card to PCIe slot 2 (x16) would I get faster speeds?

Next up/to do:
a. I'll enable SIP again and see if it still works.
b. I'll create a Fusion Drive with my NVMe drive and a 2TB WD Black Caviar. I'll create a User account on this and see if I can logon to that User account.
c. I'll then dd (copy) my entire 2TB of User data onto the FD and see if that works.

Even shows up correctly in System Information NVMExpress, with TRIM support.
View attachment 688635 View attachment 688640


Thanks to h9826790 for the encouragement to do this.

Thanks for sharing the detailed step by step guide, that's a valuable knowledge for us. And looking forward for your Fusion drive guide / report /review.

Anyway, That 3200 / 1900 is based on PCIe 3.0 x4 bandwidth. Since cMP only has PCIe 2.0 slot. Therefore, the max you can achieve is PCIe 2.0 x4 for this SSD. Unless there is a magic adaptor that can turn PCIe 2.0 x16 bandwidth into PCIe 3.0 x4 to the SSD, my understanding is impossible to achieve that marketing speed on the cMP at this moment.

Of course, you can try the x16 slot. However, some adaptor + SSD combo may even slower in the x16 slot due to some strange bug. But anyway, worth a try.

 

[JedNZ]

Updates (updated above):
Next up/To do
a. Works!
c. Completed and Works! Just have to spend the next few days syncing my 240GB Photos library to iCloud. Even on Fibre (960/400) it's gonna take days. Would be faster I'm sure to delete the iCloud Photos and completely send them all back up again.

Added a question:
Q3. Anyone know how to update the RecoveryHD to include HackrNVMeFamily-10_12_3.kext so I can mount and access the NVMe FD? I used this method, but the NVMe doesn't register – doesn't mount, doesn't show in Terminal using diskutil list or in Disk Utility etc.

I can't really enjoy any of the speed advantages of having a NVMe in a FD yet as my cMP is fully tasked by Photos syncing with iCloud again. If anyone has a solution of ratios please let me know.

 

[JedNZ]

The FD continues to work beautifully (there I go - jinxing it now!).

Observations
Although I'm still syncing 240GB of Photos and Videos to iCloud (CPUs running at 50% continuously - see note 1 below), I'm impressed at just how snappy my cMP feels (very subjective as haven't got scientific results I can compare). I converted a couple of movies I've purchased off iTunes to mp4 format, and copied them back to iTunes and the copying alone was impressive (bursts up to 800 MB/s). Of course the difference is the difference between my User data being stored on a HDD versus my new FD (NVMe + HDD). I should see a lot more of this gain once Photos stops syncing (it's hogging my CPUs and 10-20 MB/s reading/comparing my Photos and Videos).

Startup to login screen is much snappier, but I believe that's due to me replacing the USB 3.0 PCIe adapter with my NVMe PCIe adapter (I didn't use USB 3 much, if at all, anyway).

Login screen to Desktop is a bit faster (feels like 20% faster?). I appreciate some drive access routines won't be significantly faster. But opening and using Photoshop, Illustrator and inDesign files feel much faster (50% faster? for opening and saving).

Finder copying/moving/Item Info etc all appear a little faster. Again, hard to quantify this without speed test results, which I don't have.

Playing games via Steam is fairly snappy also. My Steam games are located in my User account, so loading games is significantly faster than dredging them off the HDD.

Clearly the more I use an application and/or files, the more likely they'll be prioritised to reside on the NVMe part of the FD. That's where any real speed improvements will be experienced. But I'm generally happy with the speed improvements. This will dilute over time as I get more used to it. I can only imagine what a 2TB/4TB NVme drive would be like…

Note 1: Apple have to sort out the stupidity of having to re-sync every time you change the HDD (especially when the User data is a direct copy).

 

[superparati]

Hi JedNZ,

Have you downloaded the NVMeGeneric.kext developed by Macvidcards?
http://www.macvidcards.com/nvme-driver.html

I want to try your solution but you are not talking about the kext which should be installed under /Library/Extensions/. And before to do it I want to be sure I've all files in my hand.

Thanks!

 

[JedNZ]

I tried NVMeGeneric.kext, and it would not work for me. Crash, crash. crash. Hence why I went for the Github method because it allows you to curate a version for macOS 10.12.3 - so in my opinion it appears more up to date than the method that Macvidcards hosts. Your experience/luck may differ.

 

[sterile stork]

This has been a very interesting read--thank you for sharing your experience and learnings. I'm curious what PCIe card you're currently using to house the NVMe drives--I've found the OWC offerings via their Accelsior line, is that what you've been using?

What I'd like to do is get a two-NVMe blade card setup, if possible, and create a RAID to boot from.

The FD continues to work beautifully (there I go - jinxing it now!).

Observations
Although I'm still syncing 240GB of Photos and Videos to iCloud (CPUs running at 50% continuously - see note 1 below), I'm impressed at just how snappy my cMP feels (very subjective as haven't got scientific results I can compare). I converted a couple of movies I've purchased off iTunes to mp4 format, and copied them back to iTunes and the copying alone was impressive (bursts up to 800 MB/s). Of course the difference is the difference between my User data being stored on a HDD versus my new FD (NVMe + HDD). I should see a lot more of this gain once Photos stops syncing (it's hogging my CPUs and 10-20 MB/s reading/comparing my Photos and Videos).

Startup to login screen is much snappier, but I believe that's due to me replacing the USB 3.0 PCIe adapter with my NVMe PCIe adapter (I didn't use USB 3 much, if at all, anyway).

Login screen to Desktop is a bit faster (feels like 20% faster?). I appreciate some drive access routines won't be significantly faster. But opening and using Photoshop, Illustrator and inDesign files feel much faster (50% faster? for opening and saving).

Finder copying/moving/Item Info etc all appear a little faster. Again, hard to quantify this without speed test results, which I don't have.

Playing games via Steam is fairly snappy also. My Steam games are located in my User account, so loading games is significantly faster than dredging them off the HDD.

Clearly the more I use an application and/or files, the more likely they'll be prioritised to reside on the NVMe part of the FD. That's where any real speed improvements will be experienced. But I'm generally happy with the speed improvements. This will dilute over time as I get more used to it. I can only imagine what a 2TB/4TB NVme drive would be like…

Note 1: Apple have to sort out the stupidity of having to re-sync every time you change the HDD (especially when the User data is a direct copy).

 

[JedNZ]

I just bought a generic NVMe PCIe card off eBay for about US$7 (free shipping from China).
Search on eBay: M Key M.2 NG SSD To PCI-E X4 Slot Adapter Converter Card

My system is still running perfectly with the NVMe FD. Now that I've been using my main apps and files a bit, things are becoming a little snappier (Finder processes are) - I suspect my oft used apps and files moved to the NMVe side of the FD is what makes the difference. The biggest overall gain is gaming (not that I do much of that) due in large to the size of the files, videos sequences etc.

But I don't see read/write speeds much over 200MB/s for everyday activities because I suppose I don't do much stuff that actually needs to push and pull files at lightning speed (maybe when I do my next video project it will be different).

So I have to concede that this experiment of creating a FD with a NVMe SSD is slightly overkill (and expensive) for what I do. But I'm sure there will be others of you who will get a lot more benefit out of it, so I suppose it was worth it just to share the experience. Kinda nice to squeeze a little more out of my cMP so it reaches it's unintended potential.