MP All Models PCIe SSDs - NVMe & AHCI

[Melbourne Park]

FYI - Ordered my AquaComputing Kryo M2 card ....

Just waiting on my 970EVO Plus to put in it now

Well done. Glad you got it pretty quickly!

The 970 Evo Plus is the one to buy for that card, for sure. With the lower prices of cards, the multi-card devices are somewhat questionable IMO. I'll include a blade price list, but I haven't put in 4 TB drives, maybe I should - there's a big saving buying a single card and putting a single large blade in it. 4TB blades were very rare a couple of years ago - they are around now.

I don't think I'll buy the 970 Evo Plus cards for my HighPoint 7101A-1 though. Because people report having had noise issues from the fans in the HighPoint 7101A-1 card. Hotter cards mean more HighPoint fan noise I presume. I know I can control the speed by using Highpoint software, but that can create issues when upgrading from Mojave. I know I can change to a quieter fan as well. There's more innovations as well. My Pulse RX 580 GPU will stay in slot #2, and the HighPoint will go in slot one. So I doubt I'll buy the Evo Plus blades.

I know the now rare and costly Hynix R31 Gold runs cool - I don't know of others which run cool. I know the 970 Evo runs cooler than the Plus version, but it's a lot more expensive. And slower, and very hard to find.

On price, I am tempted to buy the Western Digital S750 blades. The ultimate would be the 970 Pros but they are rare and cost a lot. The cheapest was the Western Digital Blue 550 but I don't know anything about it so i did not list it.

I have loved RAID 5 but its not practical or needed now in a limited slot card rather than in a server task. So I'm going to run RAID 0 use one card as a non RAID boot drive in one of the 4 slots.

Here's a list of the likely blades I've found, sorted in order of Cost per Terabyte, converted for the Australian price including our 10% sales tax, but I've converted it into $US, by multiplying my Australian price by 0.71:

Spoiler: Cost Per Terabyte

$106.15 per TB - 2 TB Blade: Samsung 970 EVO Plus PCIe-3
$112.89 per TB - 1 TB Blade: Samsung 970 EVO Plus PCIe-3
$112.89 per TB - 1 TB Blade: Western Digital BLACK SN750 PCIe-3
$113.59 per TB - 1 TB Blade: Sabrent Rocket Q PCIe-3
$117.15 per TB - 2 TB Blade: Sabrent Rocket Q PCIe-3
$118.41 per TB - 2 TB Blade: Western Digital Black SN750 PCIe-3
$124.25 per TB - 1 TB Blade: Samsung 980 PCIe-3
$127.45 per TB - 2 TB Blade: ADATA XPG SX8200 Pro PCIe-3
$131.35 per TB - 1 TB Blade: Western Digital Black SN750 PCIe-3
$134.19 per TB - 1 TB Blade: ADATA XPG SX8200 Pro PCIe-3
$134.19 per TB - 1 TB Blade: Samsung 970 EVO PCIe-3
$135.12 per TB - 2 TB Blade: Sabrent Rocket PCIe-3
$156.19 per TB - 1 TB Blade: Sabrent Rocket PCIe-3
$161.10 per TB - 1 TB Blade: Samsung 980 PRO PCIe-4
$171.32 per TB - 1 TB Blade: Western Digital Black SN850 PCIe-4
$175.73 per TB - 2 TB Blade: Samsung 980 PRO PCIe-3
$203.79 per TB - 1 TB Blade: Toshiba SSD XG6 PCIe-3
$204.21 per TB - 1 TB Blade: KIOXIA EXCERIA Plus PCIe-3
$232.17 per TB - 4 TB Blade: Western Digital Black SN750 PCIe-3
$303.89 per TB - 1 TB Blade: Samsung 970 PRO PCIe-3

Looking at the Western Digital 4 TB S750, it's cost per TB is about double the 1 & 2 TB price. So ... if one buys a Highpoint blade for lets say $US 400, and one buys 4 WD S750 blades it would cost $451.56 + card = $851.56. But buy a single slot card for lets say $50, and the 4 TB WD S750 for 929. Total cost is $979.

So it's better to buy the Highpoint if you need 4 TB (and speed).

But for 2 TB, there is no financial comparison (exclude productivity):

$US 400 Highpoint + $212.30 = $612.30 (OK some buy the Highpoint $100 or so cheaper).

Single $50 Card + $212.30 = $265.30. About a $350 saving ...

So unless you need speed or more than 2 TB, or fault tolerance etc, single cards make a lot more sense these days. When the cost of the larger - 4TB and up - blades drop, it won't help the makers of the multi-blade cards. I don't know why someone doesn't make a single 16 channel card ... there are some low cost duel blade 8 channel duel cards though I think ...

Thank goodness my wife doesn't read these threads ... and I wonder about the real benefits of booting from one of the 4 slots of a 16 channel card compared to SSD in a Sata slot.

 

[tsialex]

So I'm going RAID 0 with a boot drive in a slot (there being 4 slots).

A dedicated blade for macOS and a RAID array for data will work fine, but support for bootable RAID arrays that are not identified as a single instance (like iMac Pro and 2019 Mac Pro T-2 storage controller dual NAND modules) is only up to 10.13.6 and HFS+ - AppleRAID and SoftRAID are not supported since Mojave for macOS booting. For general 3rd party hardware, official bootable RAID support ends with High Sierra/HFS+, don't matter if the card you bought support bootable arrays with Windows or Linux (like HighPoint SSD7103, for example).

You can hack it, albeit losing support for native Software Updates, with Mojave and Catalina, but with BigSur/Monterey sealed containers is practically impossible to do it and a real nightmare to maintain with the constant Software Updates being issued by Apple nowadays.

 

[Melbourne Park]

A dedicated blade for macOS and a RAID array for data will work fine, but support for bootable RAID ...

Thanks. Yes, I knew RAID boot was not possible since 10.13. But little software runs best or at all on High Sierra - and the same would go for Highpoint's software I reckon.

I thought a few years ago that RAID 5 on the Highpoint 7101 would result in a loss of 25% capacity over four similar blades, and maybe not a huge hit on RAID 0 speeds. But the issue with RAID 5 is the time to rebuild. When Highpoint spoke of RAID, I always presumed RAID 5 was part of it, and it is now - but only recently I gather.

But the real issue is for me is complexity - to even run Highpoint's own controller and RAID software complicates things, especially - for me at least - when one makes the jump to Opencore, which seems inevitable for classic mac pros. Being able to run the Apple Raid removes a lot of complications with Highpoint RAID software, Opencore, etc. Some don't even bother with RAID on the 7101.

Which is why I want a cool running blade setup - I think to control the fan in the 7101 requires using the Highpoint software. Hence me not instantly buying 970 Evo Plus blades, which are very good value at the moment. They were announced 3 years ago ... but they must be huge seller to prosumers, gamers etc.

I'd love to know if the Western Digital S750 blades run cool?? They aren't much slower it seems compared to the 970 Evo Plus blades?

 

[TECK]

The last thing I want to do is start hacking things around

Can you plug your headphone jack into a device? Is the same thing for the 2pin cable.

 

[TECK]

Ordered my AquaComputing Kryo M2 card

You realize you will get low speeds with that adapter, right? Don’t expect Syba or other Pcie16 similar card performance, Kryo should have less then half of Syba speed with an 970 Evo Plus.

 

[Fastsavage]

Deleted

 

[Fastsavage]

You realize you will get low speeds with that adapter, right? Don’t expect Syba or other Pcie16 similar card performance, Kryo should have less then half of Syba speed with an 970 Evo Plus.

What sort of speeds do you get ? - I will be upgrade my processors and RAM to the same as yours in the coming days

 

[TECK]

What sort of speeds do you get ? - I will be upgrade my processors and RAM to the same as yours in the coming days

I had this card before and I upgraded to Syba because the ridiculous low speeds. If my memory is good I used to get 800Mbps versus 2500Mbps with Syba. Remember that Kryo is a Pcie4, you cannot expect performance from it. If you want to take advantage of an nVME drive then use a proper Pcie16 card, don’t cheap out on $200. Otherwise stick with SSDs.

I went with Syba versus Highpoint because it was half the price and my goal was to use two blades only, one for Mac and one for Windows.

 

[KeesMacPro]

I had this card before and I upgraded to Syba because the ridiculous low speeds. If my memory is good I used to get 800Mbps versus 2500Mbps with Syba. Remember that Kryo is a Pcie4, you cannot expect performance from it. If you want to take advantage of an nVME drive then use a proper Pcie16 card, don’t cheap out on $200. Otherwise stick with SSDs.

The I/O speed with a single NVMe card should be ~ 1500mb/s .
This is indeed what I'm getting with the KRYO and a 970 EVO.

If you got ~800mb/s something's wrong.

 

[Fastsavage]

The I/O speed with a single NVMe card should be ~ 1500mb/s .
This is indeed what I'm getting with the KRYO and a 970 EVO.

If you got ~800mb/s something's wrong.

Yeah. That’s the speeds I am hoping for

 

[TECK]

The I/O speed with a single NVMe card should be ~ 1500mb/s .
This is indeed what I'm getting with the KRYO and a 970 EVO.

If you got ~800mb/s something's wrong.

To me 1500 is low. I get 2500 on an Evo Plus with Syba. Pro gets even higher, people reported 3500. You’re spending $50 on a cheap card to get crippled nVME speeds.

 

[MarkC426]

To me 1500 is low. I get 2500 on an Evo Plus with Syba. Pro gets even higher, people reported 3500. You’re spending $50 on a cheap card to get crippled nVME speeds.

You are inferring 1500 is slow and assuming the user needs the absolute fastest nvme.....
An nvme in a Kryo adapter is damn fast compared to a hdd or ssd on a sata port.
A Kryo is one of the best 'lower price' cards for cMP, but by no means cheap compared to some of the crap ones out there.

 

[GlynH]

Can you plug your headphone jack into a device? Is the same thing for the 2pin cable.

What sort of question is that?

I didn't say I couldn't do it...I said I don't want to do it.

-=Glyn=-

 

[tsialex]

You are inferring 1500 is slow and assuming the user needs the absolute fastest nvme.....
An nvme in a Kryo adapter is damn fast compared to a hdd or ssd on a sata port.
A Kryo is one of the best 'lower price' cards for cMP, but by no means cheap compared to some of the crap ones out there.

Another thing, the transfer latency is identical and small non-contiguous throughput or 4K/4KQD32 random transfers for all effects are the same as when connected to a PCIe v3.0 x16 switched card, only when transferring large files sequentially (like when video editing or copying VM images) the greater throughput of the blades connected to the PCIe v3.0 switched card are really faster than when directly connected to a PCIe v2.0 x4 slot.

It's really weird to me that even power users and people with extensive tech knowledge frequently use throughput as the main measure of speed, while latency is what really matters most of the time.

Forgot to add, 4K and 4KQD32 are the majority of the disk access and not the sequential transfers.

 

[GlynH]

To me 1500 is low. I get 2500 on an Evo Plus with Syba. Pro gets even higher, people reported 3500. You’re spending $50 on a cheap card to get crippled nVME speeds.

To me 1500MB/s is fast. ?

I get 250MB/s on SATA HDD or SSD. I spent £14.99 / $20 on a Sabrent and I don't classify that as crippled...on paper its six times faster than SATA FFS!

Yeah sure the Syba, Highpoint etc. might be up to twice as fast again (not that you would notice much on a boot drive or in everyday use IMHO) but compared to the Sabrent could be 10-30 times more expensive?

Great if used as a scratch disk for video editing, copying huge files or indulging in the time honoured tradition of willy-waving...

Glad it rocks your world but;

1. People put too much faith in benchmarks & paper specifications...it's how it performs in real life and the cost to achieve it that matters IMHO
2. Not everybody is made of money or else we would all have fully loaded 2019 Mac Pro's and be driving Bugattis while bragging that Ferraris are crippled!

Having said that I hope to end up with a Highpoint, Sonnet or OWC one day mainly to free up slots and add expandability and I'll take the added performance as a bonus but I'll have to pass on the 7,1 and Bugatti for...well the rest of my crippled life really! ?

-=Glyn=-

 

[Melbourne Park]

Another thing, the transfer latency is identical and small non-contiguous throughput or 4K/4KQD32 random transfers for all effects are the same as when connected to a PCIe v3.0 x16 switched card, only when transferring large files sequentially (like when video editing or copying VM images) the greater throughput of the blades connected to the PCIe v3.0 switched card are really faster than when directly connected to a PCIe v2.0 x4 slot.
...

I've been trying to read specs etc. on nvme blades, and many have had fairly recent "upgrades", mostly to increase improve the latency but at the cost of moving very large files. The XPG 8200, Samsung 970 Evo Plus, and several other prosumer blades have had those changes and at the same time have come down in price.

In Australia, its difficult to understand different electricity contracts, and mobile phone contracts. The blade manufacturers are playing the same tricks ... like the good old days of clock speeds mean't your CPU was faster, or Megapixels mean't your camera was better. But weith the blade manufacturers, they've changed the tech and characteristics and left the name just the same ... in the end, it may make no perceptable difference to most users I guess.

 

[KeesMacPro]

To me 1500 is low.

Well, 1500 is what it is . nothing to do about it.
My point was that if you get ~800 , something is wrong.

I get 2500 on an Evo Plus with Syba. Pro gets even higher, people reported 3500

Sure, and I get ~ 2700 I/O in my IOCrest(=Syba) with 2x 970 EVOs.

You’re spending $50 on a cheap card to get crippled nVME speeds.

Personally I would refrase it like e.g.:
I only spend 50,- to get a massive speed improvement . more than sufficient for 95% of the applications.

I see nothing "crippled" here.
If I connect a SATA III drive to a SATA II port or vice versa, would you describe that as "crippled" ?
If so, all drive connections on the cMP can be seen as "crippled", regardless SATA or NVMe.

For more demanding applications I spend ~ 250,- for a IOCRest which saturates the PCIe connection of the cMP.

 

[0134168]

You are inferring 1500 is slow and assuming the user needs the absolute fastest nvme.....
An nvme in a Kryo adapter is damn fast compared to a hdd or ssd on a sata port.
A Kryo is one of the best 'lower price' cards for cMP, but by no means cheap compared to some of the crap ones out there.

1.500 is incredible fast. Dont dude it. You wont notice any difference in "normal" use, as those speeds are sequential transfer. 90% of the use is random transfer.

 

[GlynH]

To me 1500 is low. I get 2500 on an Evo Plus with Syba. Pro gets even higher, people reported 3500.

So having had my inquisitive button pressed I've just done a real-world warm boot test on my cMP comparing the time to boot from three different devices.

With logic expecting the NVMe M.2 blades to give me the quickest boot time I was surprised to find exactly the opposite whereby the SSD in Bay 2 was by far the quickest. Now it is running Mojave vs High Sierra on the two blades so I suppose I am not comparing like for like so maybe I should level the playing field by using the same macOS on all devices and re-run the test but it was interesting never the less.

I rebooted from a working macOS install by choosing a different startup disk from System Preferences and recorded the time it took from the monitor having the signal dropped (indicated by the sync LED on the front panel going out) to the start-up chime sounding and then from the startup chime to the appearance of the login screen

1. Samsung 970 Pro in Sabrent NVMe adaptor in PCIe Slot 4
High Sierra 10.13.6
18 secs to chime
1 min 55 secs to logon screen

2. Samsung 970 EVO Plus in Sabrent NVMe adaptor in PCIe Slot 3
High Sierra 10.13.6
15 secs to chime
2 min 21 secs to logon screen

3. Samsung EVO 860 in Drive Bay 2
Mojave 10.14.6
14 secs to chime
21 seconds to logon screen

On the face of it the SATA SSD was by far the quickest but as I mentioned above that was running Mojave vs High Sierra on the two blades so I am hoping to install either High Sierra on the SSD or Mojave on both blades and rerun the test to see if it makes a difference.

Now this is not a scientific test more a quick & dirty one and I do not wish to imply that one will always be faster than the other but I did find it interesting.

I was amazed though to see just how much quicker the SSD was in booting the machine compared to the blades but maybe there is a logical explanation that would explain that?

I do have some other questions about the boot process but will start a new thread for that as I don't want to drift off topic in this one.

-=Glyn=-

 

[MarkC426]

Don't waste your time with a HS test.
It is a known fact that a Sata SSD boots quicker than an NVMe.

Don't ask me why, I read it somewhere in this thread I think.....

 

[tsialex]

Any PCIe blade will be slower to boot when compared to SSDs installed on the southbridge SATA ports.

The MacPro firmware takes considerable time to get the blade ready since all PCIe space needs to be scanned and enumerated - a process not needed when booting from the sothbridge SATA ports. The fastest boot time of a MacPro5,1 will always be for a SATA SSD installed on SATA bay one.

Another info frequently overlooked, the NVMe blades take even more time than the older AHCI ones for the enumeration process to be completed.

 

[GlynH]

Don't waste your time with a HS test.
It is a known fact that a Sata SSD boots quicker than an NVMe.

Don't ask me why, I read it somewhere in this thread I think.....

Oh I wasn't wasting my time...I was bored and had noting better to do! ?

Well although I have read many hundreds of posts in this thread I must have missed the one where it says that!

I would be interested in finding the logic behind that although it does illustrate my earlier point that theoretically faster on paper does not always mean faster in real life. I have been around cars & bikes long enough to know that is not always the case so I was applying the same logic to computers here.

To me the SSD with Mojave just felt a lot quicker although part of that might have been the placebo effect as I installed a Quadro K5000 and now get the grey Apple logo and progress bar to look at which might have made it appear quicker...the opposite of 'a watched kettle never boils' theory I suppose ?

Figured the stopwatch would settle that argument for me...

-=Glyn=-

PS. And I've just seen the reply from Alex...thanks for that. I figured it might have something to do with polling...

 

[VaZ]

Thanks. Yes, I knew RAID boot was not possible since 10.13. But little software runs best or at all on High Sierra - and the same would go for Highpoint's software I reckon.

No no no you mis-understood Alex there., RAID 0 boot IS possible under Mojave and even under Catalina with my tests. He just means Apple doesn't 'support' making it for booting since Mojave because of AFPS limitations and broken software updates via the Apple menu. NO PROBLEM clone your startup drive to a freshly made RAID 0 drive and Voila it'll boot √

Will this same technique work with multiple blades and multiple cheap pcie adapters? Im not certain.. Can someone weigh in who's tried this?

 

[Fastsavage]

Personally I would refrase it like e.g.:
I only spend 50,- to get a massive speed improvement . more than sufficient for 95% of the applications.

My thoughts exactly - good value for money and will suit for what I do

 

[Melbourne Park]

No no no you mis-understood Alex there., RAID 0 boot IS possible under Mojave and even under Catalina with my tests. He just means Apple doesn't 'support' making it for booting since Mojave because of AFPS limitations and broken software updates via the Apple menu. NO PROBLEM clone your startup drive to a freshly made RAID 0 drive and Voila it'll boot √

Wow, thanks.

IMO that would make a big $ difference. It almost ticks the KISS principle as well - cloning is not a big deal ... Superduper etc.

I've been procrastinating on what drives to buy - mostly due to my concern about a potential roaring fan in the SSD7101A-1. I've also been chasing prices. I had thought I'd buy three drives - 1 x 1 TB, and 2 x 2 TB, leaving one slot spare and keeping the 1 TB for the boot drive. People put the boot blade too in I think slot 3, as its said to be the best cooled slot ... that might have changed now though.

However just buying two similar drives make it much easier and lower cost too and also with greater growth simplicity and RAID choice.

My problem now though, is which 2 TB drive. The WD Black SN 750 would cost at best $Au 336 x 2 = $672 ($US477) or more, and they are in very short supply at the moment. The Samsung 970 Evo Plus 2 TB are very cheap at the moment. $Au 299 ($US 212) inc tax. They are said to run hot though ... but two would be cooler than 3 drives. Also, I think the latest versions may not run as hot as they used to - they have slowed down a lot in big file transfer speeds. Samsung changed controllers too I think, but they certainly increased the cache, but also, they changed the NAND so I think instead of the old tripple bit, they have likely now 4 per memory store (TLC => MLC). So ultimatley they are slower than they were for big file work.

I've seen a few recent tests showing that. The Evo Plus 970s' now with very large file transfer speeds, of files that are encrypted (which tests the real size of the file rather than the maker's encryption tech inside the blade) they now transfer not much more than half the speed of a WD SN 750 blade. I reckon now it would also mean they probably run cooler than they used to when transferring very large files! But maybe not... XPG 8200 blades also had similar changes to them. And other blades, the same changes have been happening. Its one reason why the prices have dropped.

The "best" PCIe-3 bus blade is IMO the P31 Hynix, mostly because it runs really cool (its fast too though). But its expensive and difficult to get. Few companies sell them. $Au442 for a 2TB drive. $Au884 for 2 2TB blades ($US314 each and they were specialed for $US213 for 2 TB at one time) is almost the price of 3 Evo Plus 2 TB blades. I figure that 3 x 2 TB blades would work less hard and cooler than 2 blades working flat out - 50% more cache. But a birdy in my ignorant ear says that since blades will work as fast as they can ... 3 will be hotter than 2! I'm now knocking head on the wall.

I am overthinking it clearly ... and I think a quieter fan in the SSD7101A-1 is $30. Or put in under my desk. My videos are not noise sensitive either.

I can get the Evo Plus straight away from a store I've bought lots from before ... I'm strongly leaning towards them now ...