MP 7,1 RAID 0 - Speed Test with Sonnet PCI NVMe Card installed

[choreo]

I just installed this PCI Card last night in PCI Slot #4 of my 2019 Mac Pro:

Sonnet M.2 4x4 Silent PCIe Card (Four M.2 NVMe SSD slots • Add your own SSDs up to 32TB)

Four M.2 NVMe SSD slots on a PCIe 3.0 x16 card. Silent operation. Add your own SSDs up to 16TB. Devotes four lanes of PCIe 3.0 bandwidth to support full performance to every SSD.

www.sonnetstore.com

I mounted (4) Samsung 1TB 970 EVO Plus NVMe M.2 modules to it.

Here is a Black Magic Speed Test on one of the NVMe sticks before I configured them together as RAID 0 with Disk Utility (all four modules performed equally well):

After configuring all 4 as RAID 0 - this is what I am getting:

Sonnet advertises that I should be getting about 11,700 (theoretical read/write). Any idea why I would be getting substantially less?

 

[mythos]

FYI my results with the same setup.

 

[Schismz]

<Shrug>

Sonnet M.2 4x4 Pcie Card New Version?

With that exact setup I get an average of 6.5-7GB/s write, 8-8.5GB/s read on 8TB RAID0:

forums.macrumors.com

I believe Sonnet advertises an aggregate 11,700 on all 4 blades, but not in RAID0. As far as I know Highpoint is faster and may, or may not, have possible upside and downside. I just purchased the Sonnet because I've been using their hardware for All Eternity and haven't ever had any problems with it. It's always extremely stable and works as expected.

Also my use case often involves many small files, at which point the Amazing Speed Advantage of all SSDs kinda grinds to a halt as there really isn't a perceptible subjective difference between Apple's SSDs and Sonnet going 2-3x faster. The only thing that's winning in this space I'm aware of at the moment is Intel Optane.

Intel Optane 905P 1.50 Tb Solid State Drive - Internal - Pci Express Nvme (Pci Express Nvme 3.0 X4) - Newegg.com

Buy Intel Optane 905P 1.50 Tb Solid State Drive - Internal - Pci Express Nvme (Pci Express Nvme 3.0 X4) with fast shipping and top-rated customer service. Once you know, you Newegg!

www.newegg.com

The knowledge-base of @tsialex and what's inside his brain, could probably provide much better insights

 

[choreo]

Thanks for the replies... looks like I am in the ballpark!

 

[handheldgames]

Thanks for the replies... looks like I am in the ballpark!

If you were to take the card and drop it into a Hackintosh with a true x16 slot, one would expect to see around 12k MB/s. The Sonnet and Highpoint cards use the same PCIe switch. Unfortunately it looks like the Mac Pro / Apple Tax is hitting NVMe raid performance.

By design.. The 7,1 has 2 Pools of only 16 PCIe3 lanes. Pool A and B.

With a GPU in Pool A. Only 8 PCIe lanes are available if the GPU is dropped down to x8.
In Pool B, the thunderbolt I/O card uses 4 PCIe lanes and cannot be moved to Pool A.

If your Sonnet card is in Pool B. It's likely running at an x8 electrical connection. :-( Interrogating the PCIe bus with PCI tools should provide insight as to why a $6000+ 7,1 Mac Pro is running the NVMe Raid at the same speed as a 10 year old 5,1 Mac Pro.

Although my math aligns with the reported performance, hopefully I'm wrong the 7,1 architecture is not that limited.

 

[Schismz]

FWIW, different file-size benchmarks.

Apple OEM SSDs (4TB in my case):

Sonnet w/4 Samsung 2TB 970 EVO Plus (APFS RAID0):

 

[OkiRun]

FWIW, different file-size benchmarks.

Apple OEM SSDs (4TB in my case):

View attachment 934582

Sonnet w/4 Samsung 2TB 970 EVO Plus (APFS RAID0):

View attachment 934586

Your speeds seem much faster than Choreo reported....
Is there a reason?

 

[Schismz]

Your speeds seem much faster than Choreo reported....
Is there a reason?

Yes, I'm running ATTO benchmark across spectrum of multiple filesizes showing R/W speeds, vs. Hitting the Big Button on default test for BlackMagic (for which my results are pretty much same as everybody else in this thread, above).

Disk Benchmark Software for macOS | ATTO

Use the ATTO Disk Benchmark Software to test HBAs, hard drives, solid state drives, and hybrid drives. Free macOS download.

www.atto.com

For large files it's hitting about 11k MB/s read. If planets aligned, ideal situation occurs, and everything is perfect, RAID0 is coming very close to claimed 11.7K theoretical max for Sonnet.

 

[Jon Manterola Herbozo]

Hi!! I have a samsung 970 EVO installed with pcie adaptor into my MAC PRO (early 2009) and works great! 1400 speed more or less... but I'm reading in forums that with a double pcie adaptor and 2 nvme drives in RAID 0, I can double the speed??? into almost 3000??? is this true??? which double pcie to m2 adaptor do you recommend me??? I would like to be sure that this will work in my mac before buying another 970 EVO and a new adaptor... because if not, I prefer stay as I am... thank you so much!!

 

[tsialex]

Hi!! I have a samsung 970 EVO installed with pcie adaptor into my MAC PRO (early 2009) and works great! 1400 speed more or less... but I'm reading in forums that with a double pcie adaptor and 2 nvme drives in RAID 0, I can double the speed??? into almost 3000??? is this true??? which double pcie to m2 adaptor do you recommend me??? I would like to be sure that this will work in my mac before buying another 970 EVO and a new adaptor... because if not, I prefer stay as I am... thank you so much!!

Did you noticed that you are asking about MP4,1 on a thread about MP7,1? Please read the first post of the thread below, it's the reference thread about PCIe drives on Mac Pros:

PCIe SSDs - NVMe & AHCI

This is a general info thread for blade SSDs that can be used in the Mac Pro. This is a WikiPost so anyone with the proper credentials may edit it. NVMe SSDs can be used as a boot drive in the MP5,1 and MP6,1 with the latest firmware installed (beginning with version 140.0.0.0.0 for MP5,1 and...

forums.macrumors.com

You will need a PCIe switched dual/quad M.2 adapter to overcome the speed limitation of the PCIe v2.0 slots of your Mac Pro.

 

[Jon Manterola Herbozo]

Did you noticed that you are asking about MP4,1 on a thread about MP7,1? Please read the first post of the thread below, it's the reference thread about PCIe drives on Mac Pros:

PCIe SSDs - NVMe & AHCI

This is a general info thread for blade SSDs that can be used in the Mac Pro. This is a WikiPost so anyone with the proper credentials may edit it. NVMe SSDs can be used as a boot drive in the MP5,1 and MP6,1 with the latest firmware installed (beginning with version 140.0.0.0.0 for MP5,1 and...

forums.macrumors.com

You will need a PCIe switched dual/quad M.2 adapter to overcome the speed limitation of the PCIe v2.0 slots of your Mac Pro.

thank you!! and with the dual pcie adapter and 2 nvme in raid 0, I will reach much more speed right?? thanks!!

 

[tsialex]

thank you!! and with the dual pcie adapter and 2 nvme in raid 0, I will reach much more speed right?? thanks!!

Don't hijack threads… If you have doubts after you read the first post of the thread I linked, you asked any doubts there.

 

[Jon Manterola Herbozo]

Don't hijack threads…

sorry I dont understand well.. my english is not very good..

 

[tsialex]

sorry I dont understand well.. my english is not very good..

Don't ask questions on threads that don't apply to your doubts, this is hijacking.

If you still have doubts after you read the first post of the thread I linked before, you make your questions there, not here.

 

[Jon Manterola Herbozo]

Don't ask questions on threads that don't apply to your doubts, this is hijacking. If you have doubts after you read the first post of the thread I linked before, you ask about there.

oooooh ok!! sorry! I didn't understand that... I will ask in that link then... thank you very much

 

[LeonPro]

The Sonnet card was supposed to arrive today. FedEx re-scheduled the delivery for tomorrow, of course.

Can't wait to install these cards on them. Planning to do a RAID 10 (2+2 mirrored).

My question is: can Mac OS do this in Disk Utility? I understand Sonnet doesn't provide licensing for SoftRAID the way OWC does.

Can anyone shed light. Thanks.

 

[OkiRun]

The Sonnet card was supposed to arrive today. FedEx re-scheduled the delivery for tomorrow, of course.

Can't wait to install these cards on them. Planning to do a RAID 10 (2+2 mirrored).

My question is: can Mac OS do this in Disk Utility? I understand Sonnet doesn't provide licensing for SoftRAID the way OWC does.

Can anyone shed light. Thanks.

View attachment 971538

When I connected external memory drives for the first time, I had to reformat/wipe them in order for the 7,1 to recognize and display them in finder. Will you have to do that with these?

 

[LeonPro]

When I connected external memory drives for the first time, I had to reformat/wipe them in order for the 7,1 to recognize and display them in finder. Will you have to do that with these?

Good question. Since I'm utilizing brand new, unopened sticks that I haven't opened yet - this will have to be done.

However for those carrying over sticks they previously are using in another container card, I assume this will already be recognized and ready to use with the same files you were using it with.

For configuring in RAID 10 which is what I aim to do with these sticks, that's where I'll need help for someone who has the same arrangement on how to configure with Apple's Disk Utility. And if someone with the same Sonnet M.2 4x4 card can confirm it doesn't include a license for SoftRAID.

 

[Peter Fitskie]

FYI my result with the Sonnet Silent and 4x 970evo 1TB RAID0. Max read speed is close the max as advertised.
BTW the card doesn't include a license for SoftRAID.

 

[LeonPro]

FYI my result with the Sonnet Silent and 4x 970evo 1TB RAID0. Max read speed is close the max as advertised.
BTW the card doesn't include a license for SoftRAID.

Thanks for confirming no license for SoftRAID.

 

[choreo]

About the exact same results here using ATTO test...

 

[LeonPro]

I guess mine seems about right, since I'm running RAID 0 on only 2 x 2GB sticks. The other 2 x 2GB sticks is the mirror backup.

This is using 5GB files.

I know this was asked previously, but can't find the thread. What are you seeing with the PCI allocation? When I installed this card, I am getting 125% over limit as a result in one pool and 100% in the other.

And to think the slots aren't even occupied fully. How does this work if it's over the limit then?

 

[choreo]

I guess mine seems about right, since I'm running RAID 0 on only 2 x 2GB sticks. The other 2 x 2GB sticks is the mirror backup.

This is using 5GB files.

View attachment 973556

I know this was asked previously, but can't find the thread. What are you seeing with the PCI allocation? When I installed this card, I am getting 125% over limit as a result in one pool and 100% in the other.

And to think the slots aren't even occupied fully. How does this work if it's over the limit then?

View attachment 973560

This is what I ended up with by manually configuring...

 

[cobra521]

Leon,

Interesting!

I have two of the Sonnet Silent m.2 cards installed, only in slots 4 & 5 since each is x16, and my POOL A/B Allocations are identical to yours.

I have the feeling this is not a problem - at least I hope not! I think I recall seeing a comment on the site to the effect that this kind of pool allocation is OK, but can't find it now...

Tom

 

[LeonPro]

Yeah, I think I read somewhere this should be okay but hoping someone can technically confirm why.

In any case, I did play around with the manual configuration and didn't result to anything better:

- 100% and 125% just switched around in either pool

- 200% and 0% in each pool

I also read somewhere that you can utilized slot 3 to avoid this issue but I'm reserving slot 3 for an add-on GPU in the near future.