MP All Models PCIe SSDs - NVMe & AHCI

[Jigga Beef]

Had two Samsung 970 NVME fail on my 5,1 in the last month, I've tried searching this thread but its so long I can't seem to put it together. Anyone have a 2TB NVME drive that has been stable for them?

My Samsung NVME 960 worked for a few years without issue.

 

[trifero]

Had two Samsung 970 NVME fail on my 5,1 in the last month, I've tried searching this thread but its so long I can't seem to put it together. Anyone have a 2TB NVME drive that has been stable for them?

My Samsung NVME 960 worked for a few years without issue.

I have two, but 512GB. Work like a champ. ABout 1 year both of them.

 

[Mike Richardson]

Monterey is now installed. So far running 24 hours stable with the Intel 660p 1 TB and the Micro Connections adapter card.

FileVault encryption occurred without a hitch.

 

[sippin40oz]

Hi, I am trying to get a WD SN750 working on the 2013 MAC Pro which I bought due to the compatability guide in the thread. Bit of a long shot but but has anyone come across issues using it? I am using a Sintech adapter and isn't recognised. A test Samsung Evo plus works ok as does the SN750 via USB. Guessing there is some incompatibility going on. Running Monterey on the OEM SSD and as the Sammy EVO being recognised it doesn't seem to be an adapter/MAC Firmware issue. Came across another guy with SN750 problems but searches are not showing widespread problems.

 

[krakman]

Hi, I am trying to get a WD SN750 working on the 2013 MAC Pro which I bought due to the compatability guide in the thread. Bit of a long shot but but has anyone come across issues using it? I am using a Sintech adapter and isn't recognised. A test Samsung Evo plus works ok as does the SN750 via USB. Guessing there is some incompatibility going on. Running Monterey on the OEM SSD and as the Sammy EVO being recognised it doesn't seem to be an adapter/MAC Firmware issue. Came across another guy with SN750 problems but searches are not showing widespread problems.

FYI: I have a WD SN750 running fine in 2015 MacBook Pro. I tried a SN770 in my 2013 Mac Pro but it would KP under load.

Have you tried reset the NVRAM and SMC with the SSD installed?

 

[tsialex]

FYI: I have a WD SN750 running fine in 2015 MacBook Pro. I tried a SN770 in my 2013 Mac Pro but it would KP under load.

Have you tried reset the NVRAM and SMC with the SSD installed?

late-2013 Mac Pro is a lot more finicky with SSDs and M.2 blades that work perfectly fine with MacPro5,1 or MacPro7,1 for months fail with the MacPro6,1 or are not even recognized.

The blades on the first post here are mostly tested with MacPro5,1 or MacPro7,1, maybe we can improve that and add notes for blades that don't work or don't work correctly with a MacPro6,1.

 

[juju01]

I have a MacPro3,1. Planning to wipe mac os and install proxmox. Here is my plan:

1. Install proxmox os on PCIe card in one of the x4 slots. Since this is just for boot/os, I don't really care about speed. However, I am looking for recommendations for card with 2 slots so I can install proxmox with raid 1. PCIe card should be solid but not expensive.
2. Install a PCIe card in x16 slot for VMs and such
3. Use all 4 SATA bays as storage for a TrueNas VM.

I am not sure

1. Which PCIe card / storage drives to get for the x4 slot
2. Or , rather, if it makes sense to get one PCIe x16 card with 4 slots ( like the Highpoint 7101A) and use slot 1 & 2 for the Proxmox os and Slot 3 &4 for the virtual machines?
3. Also, is possible to put the cards in slot 1 &2 of the Highpoint in raid 1?

Also I assume since I am moving away from mac os completely, I shouldn't worry about drivers and such?

Will appreciate if someone can chime in about this question. Can't find any answers regarding compatibility of all the adapters discussed when not using macos. e.g nvme as linux boot drive on 3,1.

 

[joevt]

Will appreciate if someone can chime in about this question. Can't find any answers regarding compatibility of all the adapters discussed when not using macos. e.g nvme as linux boot drive on 3,1.

I don't know about reserving PCI devices for VMs such having one GPU for the VM and one GPU for the host.
But for storage devices a VM can use a virtual hard drive stored anywhere so that the host can still use all the drives.

As for booting NVMe, Linux can boot using EFI on MacPro3,1, so it should be possible to load NVMe driver before the Linux booter starts even without modifying the Mac's firmware.
https://forums.macrumors.com/threads/mac-pro-3-1-nvme-support-upgrade-guide-questions.2194878/

MacPro3,1 can actually connect 6 SATA drives while still have two PATA optical drives.
https://forums.macrumors.com/thread...r-from-an-open-hdd-slot.2335335/post-30871041

For two M.2 PCIe card, I suppose one of the ones with ASMedia PCIe switch should be sufficient. I haven't tried them though. I think first post has some mentions.

I don't think drivers are an issue in Linux. Mostly not an issue for macOS either...

 

[juju01]

As for booting NVMe, Linux can boot using EFI on MacPro3,1

Thanks @joevt - Just want to confirm, for a linux system (no macos), I can boot off a pair of nvme drives (raid1) in the PCIe x16 or x4 slot without any modifications at all to the 3,1 ? No install of rEFInd or RefindPlus or OpenCore EFI bootloaders ? What about enabling PCIe 2.0? Just need to find a good adapter with heatsinks, thats it?

 

[joevt]

Thanks @joevt - Just want to confirm, for a linux system (no macos), I can boot off a pair of nvme drives (raid1) in the PCIe x16 or x4 slot without any modifications at all to the 3,1 ? No install of rEFInd or RefindPlus or OpenCore EFI bootloaders ? Just need to find a good adapter with heatsinks, thats it?

Not RAID unless it's a hardware RAID that behaves as a normal block device in EFI.

Maybe Software RAID can work if that's a feature of your Linux distro - the booter needs to be in a non-raided part of the disk, load a driver for the software raid, then continue booting. I don't know anything about Linux RAID.

If grub or whatever is smart enough, then I suppose you can do without rEFInd, RefindPlus or OpenCore. Those other three can load EFI drivers but I don't think you need any.

You need to be able to change NVRAM Driver#### and DriverOrder variables to make it load NVMe driver from EFI partition of SATA disk if you want to boot NVMe. Maybe you can do that from Linux installer.

 

[juju01]

You need to be able to change NVRAM Driver#### and DriverOrder variables to make it load NVMe driver from EFI partition of SATA disk if you want to boot NVMe

@joevt - not sure what you mean here. Can you elaborate? Also what about enabling PCIe 2.0?

 

[joevt]

@joevt - not sure what you mean here. Can you elaborate? Also what about enabling PCIe 2.0?

MacPro3,1 doesn't come with EFI driver for NVMe, therefore you cannot boot directly from NVMe, unless you make the MacPro3,1 load the NVMe.efi driver from a SATA disk using the Driver#### NVRAM variables. You can also use Driver#### to load the FixPCIeLinkRate.efi driver which will ensure that PCIe 3.0 or PCIe 4.0 devices put in slot 1 or slot 2 will operate at PCIe 2.0 speed instead of just PCIe 1.0 speed.

Linux has a commandline for setting Boot#### NVRAM variables. I think it can do Driver#### variables as well.

 

[juju01]

Linux has a commandline for setting Boot#### NVRAM variables. I think it can do Driver#### variables as well.

@joevt So I did a little digging around installing Proxmox ( Debian ) on any system that requires nvme bootloader setup. A few of the posts I have found mentions Clover EFI Bootloader. Are you familiar with it? It requires the use of a USB drive to hold the nvme driver provided by clover. Similar to your approach. But not sure it also allows the setup of driver nvram variables. Let me know what you think about it.

 

[tsialex]

@joevt So I did a little digging around installing Proxmox ( Debian ) on any system that requires nvme bootloader setup. A few of the posts I have found mentions Clover EFI Bootloader. Are you familiar with it? It requires the use of a USB drive to hold the nvme driver provided by clover. Similar to your approach. But not sure it also allows the setup of driver nvram variables. Let me know what you think about it.

Clover should never be used as a boot loader for real Macs, it will overwrite the VSS store/corrupt the BootROM.

Since it's a MacPro3,1, that have only one VSS store, it's even easier to trash the NVRAM volume/BootROM this way than with newer Macs.

 

[juju01]

Clover should never be used as a boot loader for real Macs, it will overwrite the VSS store/corrupt the BootROM.

@tsialex thanks for the heads up. Will avoid it then. I also found out that during a proxmox install with zfs, an EFI System Partition is created on the install disk. ( will be the nvme boot drive in this case ). Wondering if I can leverage that partition to install the NVMe.efi and FixPCIeLinkRate.efi drivers @joevt recommends. Anyone installed proxmox on 3,1 here?

 

[tsialex]

@tsialex thanks for the heads up. Will avoid it then. I also found out that during a proxmox install with zfs, an EFI System Partition is created on the install disk. ( will be the nvme boot drive in this case ). Wondering if I can leverage that partition to install the NVMe.efi and FixPCIeLinkRate.efi drivers @joevt recommends. Anyone installed proxmox on 3,1 here?

No, before the NVMe EFI module is loaded, any NVMe drives/blades are completely ignored by the MacPro3,1 firmware, they don't exist to the point of view of the Mac Pro firmware. You have to use a SATA (USB/FW or anything that the MacPro3,1 is aware/bootable) disk for loading any EFI modules that you gonna need. You can't boot from a NVMe drive directly without adding the NVMe EFI module to the MacPro3,1 firmware.

 

[joevt]

You can't boot from a NVMe drive directly without adding the NVMe EFI module to the MacPro3,1 firmware.

Different levels of directly:

1) load booter from NVMe which loads kernel from NVMe
MacPro3,1 can't do this unless you put NVMe.efi in the firmware.

2) load nvme.efi driver from SATA, load booter from NVMe which loads kernel from NVMe
MacPro3,1 can do this using Driver#### or rEFInd or RefindPlus or Open Core (but not Clover) to load NVMe.efi.
This method is not as direct as #1 but it's pretty close - in both cases, all of linux is on the NVMe.

3) load booter from SATA and kernel from SATA or NVMe while most of Linux is on NVMe.
I don't know how to do this kind of Linux install.

 

[tsialex]

Different levels of directly:

1) load booter from NVMe which loads kernel from NVMe
MacPro3,1 can't do this unless you put NVMe.efi in the firmware.

2) load nvme.efi driver from SATA, load booter from NVMe which loads kernel from NVMe
MacPro3,1 can do this using Driver#### or rEFInd or RefindPlus or Open Core (but not Clover) to load NVMe.efi.
This method is not as direct as #1 but it's pretty close - in both cases, all of linux is on the NVMe.

3) load booter from SATA and kernel from SATA or NVMe while most of Linux is on NVMe.
I don't know how to do this kind of Linux install.

1) is not possible with unmodified MacPro3,1 firmware, 2) and 3) are chain loading/indirect methods/whatever - don't forget that the question was if it was possible to load the NVMe EFI module from the ESP of the NVMe drive/blade without any other device, which is not possible without adding the NVMe EFI module to the BootROM.

 

[joevt]

1) is not possible with unmodified MacPro3,1 firmware

Yes, putting things in the firmware does mean modifying the firmware.

2) and 3) are chain loading/indirect methods/whatever

Sure, they're chain loading, but in the case of #2, the thing doing the chain loading is Apple's EFI, not a third party EFI app. And the thing that Apple's EFI is loading is just a driver - it's still Apple's EFI that loads the linux booter, so in that regard it's not chain loading since no intermediate EFI app is loaded.

don't forget that the question was if it was possible to load the NVMe EFI module from the ESP of the NVMe drive/blade without any other device, which is not possible without adding the NVMe EFI module to the BootROM.

Right.

 

[Dayo]

install the NVMe.efi and FixPCIeLinkRate.efi drivers

If you use MyBootMgr (link in my signature) to set RefindPlus and OpenCore up, you will get the following on an MP31:

1. NVME Support via RefindPlus (no need to hack your firmware or load a separate driver)
2. APFS Support via RefindPlus (no need to hack your firmware or load a separate driver)
3. FixPCIeLinkRate driver loaded by RefindPlus (if you select the option to include this in MyBootMgr)

 

[juju01]

@joevt @tsialex @Dayo - Thanks for your comments above. My needs have changed a little bit. I went with a PCIe SSD adapter for the x4 slot - to house a SATA SSD from owc - as the boot drive for proxmox. Works well. No hiccups at all. No need to use an NVME here.

However, I still want to use the x16 slot on the 3.1 to house a 1TB NVME purely for VM storage duties. No boot drive. Will this work without all the mods suggested above? A single or double NVME adapter will work for me. Will this new sonnet card work ?

 

[PeroPenx]

Hi everyone,

So far I have been using DeLOCK 90305 card in my Mac Pro 5.1, a two-blade card with ASMedia ASM2824 chip in the second x16 slot, and GPU in the first one. Speed (two Samsung 970 EVO Plus 1TB) was as it should be - around 2700 MB/s without any improvement with two blades in RAID (I tried it just for measurement purpose), max speed was as before 2700 MB/s.

I managed to get a relatively cheaply new Sonnet 4x4 silent card and replaced the old DeLOCK. Blades are still not in RAID, the first one is a system disk. I planned to buy two more and make a RAID out of three blades.

But! Speed of both blades in Sonnet is still the same as before in DeLOCK - 2700 MB/s. without any speed acceleration whatsoever.

Is there a way to make it faster (the first post says the expected speed is around 3200 MB/s) and if I buy another blade(s) and make a RAID can I expect 6200 MB/s?

P.S. all speed measurements were done with Blackmagic speed test

 

[joevt]

Hi everyone,

So far I have been using DeLOCK 90305 card in my Mac Pro 5.1, a two-blade card with ASMedia ASM2824 chip in the second x16 slot, and GPU in the first one. Speed (two Samsung 970 EVO Plus 1TB) was as it should be - around 2700 MB/s without any improvement with two blades in RAID (I tried it just for measurement purpose), max speed was as before 2700 MB/s.

I managed to get a relatively cheaply new Sonnet 4x4 silent card and replaced the old DeLOCK. Blades are still not in RAID, the first one is a system disk. I planned to buy two more and make a RAID out of three blades.

But! Speed of both blades in Sonnet is still the same as before in DeLOCK - 2700 MB/s. without any speed acceleration whatsoever.

Is there a way to make it faster (the first post says the expected speed is around 3200 MB/s) and if I buy another blade(s) and make a RAID can I expect 6200 MB/s?

P.S. all speed measurements were done with Blackmagic speed test

Are you doing RAID 0?
Use pcitree.sh to check the PCI link width and link rate for everything.
Use AmorphousDiskMark.app. Check each NVMe individually before doing RAID.
Before doing RAID, try using ATTO Disk Benchmark.app to measure speed of multiple NVMe at the same time.

 

[PeroPenx]

Are you doing RAID 0?
Use pcitree.sh to check the PCI link width and link rate for everything.
Use AmorphousDiskMark.app. Check each NVMe individually before doing RAID.
Before doing RAID, try using ATTO Disk Benchmark.app to measure speed of multiple NVMe at the same time.

Thanks for answering,

at the moment my problem is not RAID but rather speed in blackmagic disk test that didn't showed speed increase when changing card from delock to sonnet (it should due to better switch on the card). and yes, my idea was to buy two additional blades and make RAID 0 out of 3 blades (4th is system and it, of course, stays separate)

but I did what you told me to and here are results:

1. pcitree.sh gave me empty list (nothing) just #===================================. Ok, I don't have DirectHW.kext installed and I tried sudo nvram boot-args="debug=0x144" but again nothing

2. Amorphous gave similar results for both blades (first one is system disk) but kind of strange fall in later results





3. ATTO testing both blades in the same time also gave strange results. System disk is much slower in writing in higher MiB with really significant drop. I got the same result while testing only system disk in ATTO

 

[joevt]

at the moment my problem is not RAID but rather speed in blackmagic disk test that didn't showed speed increase when changing card from delock to sonnet (it should due to better switch on the card). and yes, my idea was to buy two additional blades and make RAID 0 out of 3 blades (4th is system and it, of course, stays separate)

I'm not sure why you would expect a speed increase without RAID.
The delock is x8 which is enough for a PCIe 2.0 slot to do PCIe 3.0 x4.

but I did what you told me to and here are results:

1. pcitree.sh gave me empty list (nothing) just #===================================. Ok, I don't have DirectHW.kext installed and I tried sudo nvram boot-args="debug=0x144" but again nothing

If you are using Open Core, then the boot-args need to go in the config.plist. A restart is required.
You also need to install pciutils.
Type sudo lspci -A detect to get a list of methods. The one we want is AppleACPIPlatformExpert, which doesn't require DirectHW.kext
but does require debug=0x144

2. Amorphous gave similar results for both blades (first one is system disk) but kind of strange fall in later results

Those results look good. They indicate that the blades are running at PCIe 3.0 speed (I don't think there was much doubt about that since you have a MacPro5,1). You can just click the SEQ1M QD8 button to do the benchmark faster.

3. ATTO testing both blades in the same time also gave strange results. System disk is much slower in writing in higher MiB with really significant drop. I got the same result while testing only system disk in ATTO

You got good result at the 8MB block size. Maybe the rest are from throttling? Maybe need to wait awhile before trying again. You can change the settings of the ATTO benchmark to only include 8MB to 16MB block sizes to make the benchmark go faster.

You don't show the ATTO result for the total. The total should be near 6000 MB/s for x16 card maybe (guessing) 3500 MB/s for x8 card.