Thunderbolt 4 & NVMe M.2 External Storage: Read/Write Speeds?

[raid0hunter]

This is the only place on the internet where I could find a discussion specifically about SSDs and RAID0s on Macs. So glad to find this thread!
I bought 2 Gen 4 NVMe SSDs and 2 TB4 enclosures. Connected them to separate TB ports on my iMac 2017. Individually, they reach about 2.7 GBps speed. Then I setup RAID0 on them. To my surprise, I still only get 2.7 GBps! I tried BlackMagic and Amorphous tools. They both show about 2.7-ish. Any idea what's throttling the speed?

 

[brandair]

I guess that iMac has only one Thunderbolt controller for all TB ports. If there are 4 TB ports try this unless already done: plug enclosure 1 into port 1 and enclosure 2 into port 3 or 4.

This is the only place on the internet where I could find a discussion specifically about SSDs and RAID0s on Macs. So glad to find this thread!
I bought 2 Gen 4 NVMe SSDs and 2 TB4 enclosures. Connected them to separate TB ports on my iMac 2017. Individually, they reach about 2.7 GBps speed. Then I setup RAID0 on them. To my surprise, I still only get 2.7 GBps! I tried BlackMagic and Amorphous tools. They both show about 2.7-ish. Any idea what's throttling the speed? View attachment 2183396

 

[raid0hunter]

Thank you @brandair! Man, this has been driving me nuts for a while. I think you solved the case. iMac 2017 has two TB ports, but only one shared controller. Not sure what the overall limit is, but it's the bottleneck! That's disappointing. I guess I'll rather use the drives separately instead of increasing the risk of failure with RAID0 with no benefit.

 

[MacStudioDude]

M1 Studio Max.

Acacis TB34 enclosure.

WD SN850X, 2 TB

 

[Silly John Fatty]

Just setup today my RAID 0:
- two (2) separate Acacias TBU401E 40Gbps NVMe m.2 SSD TB4 cases
- pair of 2TB WD_BLACK SN850X NVMe PCIe 4.0 SSD

Below please find my results vs. the internal Apple SSD:

View attachment 2172741


512GB Internal Apple SSD

View attachment 2172742

For additional reference: external (non-raid) LaCie SSD Pro (2TB) w/Seagate FireCuda

View attachment 2172743

What do you do to make this "Raid 0"? Do you just connect the two encasings to your Mac, or do you add them to another type of encasing, or something like this?

 

[steve123]

What do you do to make this "Raid 0"? Do you just connect the two encasings to your Mac, or do you add them to another type of encasing, or something like this?

You will require two Thunderbolt 3/4 SSD enclosures (for example, the ACASIS TBU405). Each enclosure is connected to the Mac using a Thunderbolt 3/4 cable. It is important that the ports selected are connected to different Thunderbolt controllers. The M2 Pro Mini has 4 Thunderbolt ports and 2 Thunderbolt controllers. So, there are two ports per controller. You can confirm which controllers the attached enclosures as connected to using System Information->Thunderbolt. You do not want to connect both drives to the same controller. Then use Disk Utility for configure the two external drives as RAID0.

 

[Silly John Fatty]

You will require two Thunderbolt 3/4 SSD enclosures (for example, the ACASIS TBU405). Each enclosure is connected to the Mac using a Thunderbolt 3/4 cable. It is important that the ports selected are connected to different Thunderbolt controllers. The M2 Pro Mini has 4 Thunderbolt ports and 2 Thunderbolt controllers. So, there are two ports per controller. You can confirm which controllers the attached enclosures as connected to using System Information->Thunderbolt. You do not want to connect both drives to the same controller. Then use Disk Utility for configure the two external drives as RAID0.

Thanks Steve!

I don't quite understand what those Thunderbolt controllers are. Is it basically the actual technology behind a Thunderbolt port? That's how I read it.

But what does it mean then when two ports use one controller? Also, if I use two different controllers for the RAID0 setup, can I still use the two left ports, and what things should I consider if I do so? Could it impact speeds for example of pose some problems?

And as for the RAID0: Basically it's merging multiple drives to one, in order to get more speed, if I get that right. If I merge even more of them, can I get more read/write speeds from them than from the internal hardware or is there a limit to this as well?

Thanks again

 

[steve123]

I don't quite understand what those Thunderbolt controllers are. Is it basically the actual technology behind a Thunderbolt port? That's how I read it.

Yes. This graphic illustrates the elements that comprise a controller in the shaded area. A Thunderbolt port will have additional elements such as a power delivery controller not shown here.

These resources might help:

https://www.thunderbolttechnology.net/sites/default/files/intel-thunderbolt4-announcement-press-deck.pdf

https://www.thunderbolttechnology.net/sites/default/files/18-241_Thunder7000Controller_Brief_FIN_HI.pdf

Intel's Maple Ridge (JHL8540) Thunderbolt 4 Controller Now Shipping

www.anandtech.com

But what does it mean then when two ports use one controller? Also, if I use two different controllers for the RAID0 setup, can I still use the two left ports, and what things should I consider if I do so? Could it impact speeds for example of pose some problems?

The illustration describes two ports connected to one controller. The two ports "share" the controller and therefore share the bandwidth of the controller. The M2 Pro mini has two of these controllers and 4 ports. If you connect two devices to the same controller, the two devices share the 40 Gbps bandwidth of one controller. If you connect one device to each controller, each device gets the full bandwidth of each controller = 2 x 40 Gbps = 80 Gbps. This is why a RAID0, when using separate controllers, is twice the speed as a single drive connected to one controller.

And as for the RAID0: Basically it's merging multiple drives to one, in order to get more speed, if I get that right. If I merge even more of them, can I get more read/write speeds from them than from the internal hardware or is there a limit to this as well?

No. The M2 Pro mini only has two controllers so configuring an array with more than two enclosures does not improve performance.

 

[Silly John Fatty]

Yes. This graphic illustrates the elements that comprise a controller in the shaded area. A Thunderbolt port will have additional elements such as a power delivery controller not shown here.

View attachment 2185755

These resources might help:

https://www.thunderbolttechnology.net/sites/default/files/intel-thunderbolt4-announcement-press-deck.pdf

https://www.thunderbolttechnology.net/sites/default/files/18-241_Thunder7000Controller_Brief_FIN_HI.pdf

Intel's Maple Ridge (JHL8540) Thunderbolt 4 Controller Now Shipping

www.anandtech.com

The illustration describes two ports connected to one controller. The two ports "share" the controller and therefore share the bandwidth of the controller. The M2 Pro mini has two of these controllers and 4 ports. If you connect two devices to the same controller, the two devices share the 40 Gbps bandwidth of one controller. If you connect one device to each controller, each device gets the full bandwidth of each controller = 2 x 40 Gbps = 80 Gbps. This is why a RAID0, when using separate controllers, is twice the speed as a single drive connected to one controller.


No. The M2 Pro mini only has two controllers so configuring an array with more than two enclosures does not improve performance.

Thanks, that was helpful! Too bad we can't get more than that. Having more thunderbolt controllers is really a thing now in my eyes, although I never thought about it before.

But isn't that faster than the internal drives? People say the Mini with 1+ TB has read/write speeds of 6000 MB/s, using both TB controllers would theoretically be 80000 MB/s?

 

[Silly John Fatty]

@EzisAA's reaction to my post tells me something is wrong in my calculation.

 

[Shamgar]

There are 8 bits (lower case b) in a byte (upper case B). Divide the 40Gb/s by 8. Then you have to account for various types of overhead. Notice that the single drive performance of the 40 Gb/s enclosures in this thread max out around 3 GB/s despite the enclosed drives having higher rated speeds. Double that (or slightly less for RAID control overhead) for two controllers and you might get back to around 6 GB/s.

 

[handheldgames]

There are 8 bits (lower case b) in a byte (upper case B). Divide the 40Gb/s by 8. Then you have to account for various types of overhead. Notice that the single drive performance of the 40 Gb/s enclosures in this thread max out around 3 GB/s despite the enclosed drives having higher rated speeds. Double that (or slightly less for RAID control overhead) for two controllers and you might get back to around 6 GB/s.

The chart below illustrates the 40 Gb/s Thunderbolt 4 data stream. As you can see, data bandwidth is capped at 22 Gb/s or 2750 MB/s in all configurations. Although this aligns closely with the maximum Thunderbolt 3 attached NVME SSD's reported on M1 Macs, specific configurations of NVME drives / enclosures have achieved read speeds that extend to 24 Gbs or 3000 MB/s.

If the full 40 GB/s bandwidth were available to data, 5000 MB/s would be possible.

22Gbps =

 

[steve123]

As you can see, data bandwidth is capped at 22 Gb/s

I understand TB4 allows PCIe tunnelling with rates up to 32 Gbps. The TB4 SSD enclosures that use the Intel TB4 chips use PCIe tunnelling.

 

[handheldgames]

I understand TB4 allows PCIe tunnelling with rates up to 32 Gbps. The TB4 SSD enclosures that use the Intel TB4 chips use PCIe tunnelling.

Is there a specification you can reference?

 

[steve123]

Is there a specification you can reference?

see slide 5

https://www.thunderbolttechnology.net/sites/default/files/intel-thunderbolt4-announcement-press-deck.pdf

 

[EzisAA]

@EzisAA's reaction to my post tells me something is wrong in my calculation.

Based on this single controller in real life is about ~2.6GB/s for data (Samsung 980 Pro 2TB witch Acasis TBU401E enclosure connected to MacBook Pro M1).
So used two controllers (RAID0 with two Thunderbolt enclosures) you will get about ~5.2GB/s.
But speed still very dependent from encluser type and SSD model.

 

[handheldgames]

Based on this single controller in real life is about ~2.6GB/s for data (Samsung 980 Pro 2TB witch Acasis TBU401E enclosure connected to MacBook Pro M1).
So used two controllers (RAID0 with two Thunderbolt enclosures) you will get about ~5.2GB/s.
But speed still very dependent from encluser type and SSD model.

With the approval of the ASM4242 chipset earlier this month, we should see a performance improvement over Intel's JHL7740.

 

[Elviss116]

any problem with ugreen RTL9210B ? after update mac os ventura 13.3.1 is very slow

 

[edubfromktown]

I just setup two Sabrent TB3 enclosures with KC3000's RAID 0 and results in Amorphous were fine by me!

Internal Studio SSD 3959 5416
Dual KC3K's RAID 0 3342 2940
Single KC3K drive 1650 1485 (also tested a buddy's Samsung T7 while ago 1780 1300)

I backup to a 14 TB WD EasyStore since RAID0 can drive right off of a cliff quite easily LOL

 

[hajime]

You will require two Thunderbolt 3/4 SSD enclosures (for example, the ACASIS TBU405). Each enclosure is connected to the Mac using a Thunderbolt 3/4 cable. It is important that the ports selected are connected to different Thunderbolt controllers. The M2 Pro Mini has 4 Thunderbolt ports and 2 Thunderbolt controllers. So, there are two ports per controller. You can confirm which controllers the attached enclosures as connected to using System Information->Thunderbolt. You do not want to connect both drives to the same controller. Then use Disk Utility for configure the two external drives as RAID0.

How many independent Thunderbolt controllers do the base model of M2 Mini and M2 Air 15" have?

 

[steve123]

How many independent Thunderbolt controllers do the base model of M2 Mini and M2 Air 15" have?

The base model M2 Mini only has one controller. The M2 Air's only have one controller.

 

[hajime]

Thank you

 

[Silly John Fatty]

Is it possible to damage your Mac if you have a external RAID system in which you get close to these 40 gb/s?

Seeing as how my Thunderbolt devices occasionally disconnect while the Studio is sleeping, I'd be terrified to do any kind of RAID across multiple enclosures long term. RAID within same enclosure seems fine so far. Has anybody who daily drives a multiple-enclosure RAID for more than a few months encountered issues with array stability?

What's the danger here @Shamgar? (For newbs) What's the worst case?

 

[Shamgar]

Is it possible to damage your Mac if you have a external RAID system in which you get close to these 40 gb/s?



What's the danger here @Shamgar? (For newbs) What's the worst case?

There is no risk of physical damage. The danger is data loss. If one enclosure in a RAID 0 multi-enclosure array loses connection while the volume is mounted it is quite possible for the volume to become damaged. This could be corrupted files where one chunk of the file is now out of sync with the rest of the file on the other drive, or total volume loss where the OS sees the different drives as out of sync and is unable to recover.

 

[Silly John Fatty]

There is no risk of physical damage. The danger is data loss. If one enclosure in a RAID 0 multi-enclosure array loses connection while the volume is mounted it is quite possible for the volume to become damaged. This could be corrupted files where one chunk of the file is now out of sync with the rest of the file on the other drive, or total volume loss where the OS sees the different drives as out of sync and is unable to recover.

Okay, I see what you mean now. I recently saw some sort of hub with a battery to which you'd connect your hardware, and in case of a power shutdown, your devices would have time to finish doing what they were doing.