MP 1,1-5,1 Slow internal transfer speeds on new NVMe SSD

[bllx]

I recently installed a PCIe SSD in my 5,1 cMP.
The carrier is a Sabrent Model EC-PCIE NVMe M.2 SSD, £15; and the SSD is a 1TB Kioxia M.2 2280 PCIe Gen3x4 SSD, £66.50. I'm on a budget and the switched fancy carriers are way too much.

BlackMagicDesign speed tests gives speeds around 1400 MB/sec for read and write on the PCIe SSD, which is what I was expecting, and around 150 MB/sec for HDs on the SATA bus, and around 250 MB/sec for a SATA SSD.
The PCIe SSD boots macOS fine and seems to run well.

However, if I do a file copy operation in either direction from/to the PCI SSD from/to an internally mounted SATA device, the speed usually starts reasonably (ie around the max of the slower device), but quickly drops to 5-10MB/sec. If I watch the speed carefully, it seems to have bursts of full speed, then a period of very slow 5-10 MB/sec speeds, then full speed, then very slow, etc...
I have tried the PCIe SSD in both 16 lane slot (slot 2) and 4 lane slots (slots 3 and 4). In the 16 lane slot it seems to perform a little better but still with major slowdowns.
This issue reminds me of an issue I have had for years with USB 3 drives mounted via a PCIe USB 3 card in slot 4: exactly the same thing happens. I always assumed the USB 3 card might be faulty. However given the same happens with the NVMe SSD card, it would appear to be a bottleneck problem on the PCIe bus.

Does anyone else have this issue or can shed any light on it?
Also- is there a macOS utility which can tell me the temperature of the NVMe SSD device?

Many thanks.

 

[MediaGary]

I don't have the issue, but my first guess from the low price that this SSD is a DRAM-less design. I'm also guessing that you've formatted it in APFS, and that the combination of the extra housekeeping that APFS does, in combination with a long transfer is causing this high/low throughput problem. Try formatting it in HFS+ (if not already) and see if things improve.

As for temperature monitoring, I've been using iStat Menus from Bjango.com. It's fairly comprehensive in providing temperature data from the 5,1 and includes all the drive temperatures, HDD, SATA SSD, and NVMe SSD.

 

[h9826790]

I recently installed a PCIe SSD in my 5,1 cMP.
The carrier is a Sabrent Model EC-PCIE NVMe M.2 SSD, £15; and the SSD is a 1TB Kioxia M.2 2280 PCIe Gen3x4 SSD, £66.50. I'm on a budget and the switched fancy carriers are way too much.

BlackMagicDesign speed tests gives speeds around 1400 MB/sec for read and write on the PCIe SSD, which is what I was expecting, and around 150 MB/sec for HDs on the SATA bus, and around 250 MB/sec for a SATA SSD.
The PCIe SSD boots macOS fine and seems to run well.

However, if I do a file copy operation in either direction from/to the PCI SSD from/to an internally mounted SATA device, the speed usually starts reasonably (ie around the max of the slower device), but quickly drops to 5-10MB/sec. If I watch the speed carefully, it seems to have bursts of full speed, then a period of very slow 5-10 MB/sec speeds, then full speed, then very slow, etc...
I have tried the PCIe SSD in both 16 lane slot (slot 2) and 4 lane slots (slots 3 and 4). In the 16 lane slot it seems to perform a little better but still with major slowdowns.
This issue reminds me of an issue I have had for years with USB 3 drives mounted via a PCIe USB 3 card in slot 4: exactly the same thing happens. I always assumed the USB 3 card might be faulty. However given the same happens with the NVMe SSD card, it would appear to be a bottleneck problem on the PCIe bus.

Does anyone else have this issue or can shed any light on it?
Also- is there a macOS utility which can tell me the temperature of the NVMe SSD device?

Many thanks.

What kind of files? Large / small files can make a huge difference here.

For your info, if you try to copy a bunch of very small files (e.g. thousands of 4KB files) from your HDD, the speed can go below 1MB/s (regardless the other side is a NVMe).

 

[bllx]

So I made some progress in diagnosing this.

Tried different PCI slots for the PCIe SSD card, swapped graphics card back to Apple original (just in case), tried different file sizes in the transfers…nothing made any difference; most copies were at 5-10 MB/sec. Temperature of the PCI SSD was around 40, varying only a little.

Finally I realised that any copy operation involving a SATA or PCI SSD and an HD, was always very slow, but SATA HD—>SATA HD copy operations were almost 200MB/second. Focusing on PCI SSD-->SATA SSD and vice versa I occasionally got super fast speeds, but then minutes later, changing nothing, it drops to 5-30MB/sec, and stays like that. Will carry on trying to work out if the speed change correlates with any indicators.
Incidentally, the HDs are HFS+ and the SSDs are APFS.

The PCI SSD is my main system drive now so I dont really want to format it HFS+; and I suppose the only way to be sure would be to try copy operations between 2xHFS+ formatted SSDs. I’ll look into it

 

[StellarVixen]

I recently installed a PCIe SSD in my 5,1 cMP.
The carrier is a Sabrent Model EC-PCIE NVMe M.2 SSD, £15; and the SSD is a 1TB Kioxia M.2 2280 PCIe Gen3x4 SSD, £66.50. I'm on a budget and the switched fancy carriers are way too much.

BlackMagicDesign speed tests gives speeds around 1400 MB/sec for read and write on the PCIe SSD, which is what I was expecting, and around 150 MB/sec for HDs on the SATA bus, and around 250 MB/sec for a SATA SSD.
The PCIe SSD boots macOS fine and seems to run well.

However, if I do a file copy operation in either direction from/to the PCI SSD from/to an internally mounted SATA device, the speed usually starts reasonably (ie around the max of the slower device), but quickly drops to 5-10MB/sec. If I watch the speed carefully, it seems to have bursts of full speed, then a period of very slow 5-10 MB/sec speeds, then full speed, then very slow, etc...
I have tried the PCIe SSD in both 16 lane slot (slot 2) and 4 lane slots (slots 3 and 4). In the 16 lane slot it seems to perform a little better but still with major slowdowns.
This issue reminds me of an issue I have had for years with USB 3 drives mounted via a PCIe USB 3 card in slot 4: exactly the same thing happens. I always assumed the USB 3 card might be faulty. However given the same happens with the NVMe SSD card, it would appear to be a bottleneck problem on the PCIe bus.

Does anyone else have this issue or can shed any light on it?
Also- is there a macOS utility which can tell me the temperature of the NVMe SSD device?

Many thanks.

You forgot to say which OS are you using? Are you using Monterey? I have heard from the Hackintosh community that Monterey is not playing nice with some NVMe SSDs, especially ones from Samsung.

 

[MediaGary]

You won't have the option of reformatting the boot drive as HFS+ (I thought you were strictly dealing with data drives) assuming you're running a recent macOS. It's good you've done some more testing, and are documenting the combinations of from/to. Eventually the "eureka" moment will happen.

 

[h9826790]

So I made some progress in diagnosing this.

Tried different PCI slots for the PCIe SSD card, swapped graphics card back to Apple original (just in case), tried different file sizes in the transfers…nothing made any difference; most copies were at 5-10 MB/sec. Temperature of the PCI SSD was around 40, varying only a little.

Finally I realised that any copy operation involving a SATA or PCI SSD and an HD, was always very slow, but SATA HD—>SATA HD copy operations were almost 200MB/second. Focusing on PCI SSD-->SATA SSD and vice versa I occasionally got super fast speeds, but then minutes later, changing nothing, it drops to 5-30MB/sec, and stays like that. Will carry on trying to work out if the speed change correlates with any indicators.
Incidentally, the HDs are HFS+ and the SSDs are APFS.

The PCI SSD is my main system drive now so I dont really want to format it HFS+; and I suppose the only way to be sure would be to try copy operations between 2xHFS+ formatted SSDs. I’ll look into it

If you suspect that's APFS related, and want to remove the drive's limitation for testing purpose. You can create a RAM drive (e.g. 10GB RAM drive), then format it to APFS and use it as the to / from drive for files copying test (to / from all your other drives).

And then, format the same RAM drive to SATA and carry out the same tests.

This will allow you to know if the file system make any difference.