Thunderbolt 3 M.2 NVMe SSD Enclosures

[joevt]

Thank you! I'm going to assume we are talking about NVMe devices connected via TB3. It seems testing multiple NVMe devices connected to a single external TB3 controller would be limited by that controller before they hit any limits of the integrated TB3 controller. But how do I connect multiple external TB3 controllers to a single TB3 port of the integrated TB3 controller?

For Ice Lake or Tiger Lake, I believe there's a Thunderbolt bus for the left side and a Thunderbolt bus for the right side of a laptop. Each side has two Thunderbolt ports. So you could connect one NVMe to one port and the other NVMe to the second port.

You can check that the ports belong to the same bus by looking at Thunderbolt buses in the System Information.app. A Thunderbolt bus is an individual Thunderbolt host controller.

I think on Apple Silicon Macs, each Thunderbolt port belongs to a separate Thunderbolt bus.

Besides testing two Thunderbolt ports of a Thunderbolt host, another interesting test would be to connect two NVMe to the same Thunderbolt 3 port. This can be done using a Thunderbolt 4 hub. Or you could put an NVMe in a Thunderbolt enclosure that has two Thunderbolt ports - such as any Thunderbolt PCIe expansion chassis. It would be interesting if this configuration could get more than 3200 MB/s but it will probably be limited to less than that.

For an integrated Thunderbolt controller, the total bandwidth of a Thunderbolt bus is > 4500 MB/s (limited by internal CPU stuff). A discrete Thunderbolt controller is limited to PCIe 3.0 x4 ≈ 3500 MB/s but I was not able to get much more than ≈2800 MB/s which is approximately the max expected speed of a single NVMe.

 

[Low Shrimp]

So you could connect one NVMe to one port and the other NVMe to the second port.

Ah, me and my narrow mind. I was thinking strictly in terms of port throughput. Since the internal controller has two PCIe x4 links it is possible to have the total bandwidth on the controller higher than a single PCIe x4 link allows.

Besides testing two Thunderbolt ports of a Thunderbolt host, another interesting test would be to connect two NVMe to the same Thunderbolt 3 port. This can be done using a Thunderbolt 4 hub. Or you could put an NVMe in a Thunderbolt enclosure that has two Thunderbolt ports - such as any Thunderbolt PCIe expansion chassis.

Thunderbolt 4 hub is an interesting suggestion. I wonder what chips are they based on. Expansion chassis would be connected to multiple thunderbolt ports on the host, no?

 

[joevt]

Since the internal controller has two PCIe x4 links it is possible to have the total bandwidth on the controller higher than a single PCIe x4 link allows.

Internal controller of the CPU? They don't have real PCIe links since they're built into the CPU. Who know how many lanes or what speed they use.

Thunderbolt 4 hub is an interesting suggestion. I wonder what chips are they based on.

Thunderbolt 4 hub uses Intel Goshen Ridge.
Current Thunderbolt 3 docks use Titan Ridge.
The first Thunderbolt 3 docks use Alpine Ridge.

Expansion chassis would be connected to multiple thunderbolt ports on the host, no?

That's not possible. Expansion chassis isn't much different than any other Thunderbolt device. It has one upstream Thunderbolt connection to the host and possibly one downstream Thunderbolt port. It is just a Thunderbolt controller where the PCIe lanes are exposed so that you can connect one or two or 4 PCIe devices to it. Or you can connect a PCIe switch and connect any number of devices. The Sonnet Echo Express III-D is a Thunderbolt PCIe expansion chassis that has a PCIe switch that provides 3 PCIe slots. The HighPoint 7505A is a PCIe card that has a PCIe switch that provides four M.2 slots. The CalDigit TS3+ is a Thunderbolt dock with 4 PCIe devices connected (Ethernet controller, 3 USB controllers).

 

[EncryptedUser]

I found a cheap Kingston NV2 2TB drive which I planned to use for storage and TM backups.
Did ran some tests for it in my one of my Acasis TBU401 on a Mac mini M1.

Read speed is decent but write takes a hit in the TBU401.

 

[Chancha]

@micheloupatrick
@Chancha

Does anybody of you 4TB guys have a chance to test if these drives can be formatted with a single 4TB (NTFS) GPT partition?

I read somehwere that most NVMe ensclosures ony support MBR (partition 2TB max.) not GPT.

What are your max.temps?

Sorry I only use these SSDs with Macs, so only ever tried APFS/HFS+.
But as with everything Windows, it sounds like there must be a work around.

As for temps, the ACASIS does not seem to passthrough SMART via Thunderbolt, so I am only measuring with my IR gun on the outer chassis; never see anything higher than 40 degrees C.

 

[joevt]

As for temps, the ACASIS does not seem to passthrough SMART via Thunderbolt, so I am only measuring with my IR gun on the outer chassis; never see anything higher than 40 degrees C.

I think SMART should work for PCI devices (NVMe, AHCI) which are usually those > 1500 MB/s.
SMART doesn't work for USB devices ( < 1100 MB/s) unless you install some extra software.
https://binaryfruit.com/drivedx/usb-drive-support

 

[Chancha]

I think SMART should work for PCI devices (NVMe, AHCI) which are usually those > 1500 MB/s.
SMART doesn't work for USB devices ( < 1100 MB/s) unless you install some extra software.
https://binaryfruit.com/drivedx/usb-drive-support

Is there a free Mac app that can easily check SMART temps then?

I have iStat Menus and the sensor list does not show the ACASIS (it does for some other enclosures). Then I also have Sensei.app where its drive health section says the mounted SSD's SMART data is not found.

I am unsure what is going on so I assume it is the enclosure not fully supporting SMART. Or that macOS has changed something with external drives.

 

[dia3olik]

So both enclosures perform very similarly (and very well).

Hey! ;-) Thanks for your tests, which enclosure dissipate heat the best? Have you measured temperatures using smart during long and intensive transfers? Thanks again!!!

 

[EncryptedUser]

Smartmontools works ok!

smartmontools

www.smartmontools.org

smartctl -x disk6
smartctl 7.3 2022-02-28 r5338 [Darwin 22.2.0 x86_64] (sf-7.3-1)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       KINGSTON SKC3000S1024G
Serial Number:                      50026B7685CCC62A
Firmware Version:                   EIFK31.6
PCI Vendor/Subsystem ID:            0x2646
IEEE OUI Identifier:                0x0026b7
Total NVM Capacity:                 1 024 209 543 168 [1,02 TB]
Unallocated NVM Capacity:           0
Controller ID:                      1
NVMe Version:                       1.4
Number of Namespaces:               1
Local Time is:                      Thu Jan 12 18:05:43 2023 CET
Firmware Updates (0x12):            1 Slot, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005d):     Comp DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Log Page Attributes (0x08):         Telmtry_Lg
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     84 Celsius
Critical Comp. Temp. Threshold:     89 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     8.80W       -        -    0  0  0  0        0       0
 1 +     7.10W       -        -    1  1  1  1        0       0
 2 +     5.20W       -        -    2  2  2  2        0       0
 3 -   0.0620W       -        -    3  3  3  3     2500    7500
 4 -   0.0620W       -        -    4  4  4  4     2500    7500

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        39 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    127 099 [65,0 GB]
Data Units Written:                 706 496 [361 GB]
Host Read Commands:                 1 587 347
Host Write Commands:                839 371
Controller Busy Time:               4
Power Cycles:                       81
Power On Hours:                     2 865
Unsafe Shutdowns:                   17
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 2:               49 Celsius

Error Information (NVMe Log 0x01, 16 of 63 entries)
No Errors Logged

 

[micheloupatrick]

Hey! ;-) Thanks for your tests, which enclosure dissipate heat the best? Have you measured temperatures using smart during long and intensive transfers? Thanks again!!!

I just did a real world test copying 1.27 TB worth of video files from a Raid 0 Firecuda 530 volume (inside an OWC Mercury Pro U.2 Dual) to an older Sabrent 4TB nvme drive (the dark blue one) inside the Acasis enclosure : the drives were all connected to a core i9 2020 iMac, which has a single thunderbolt bus so it's quite busy with 3 raid enclosures and the display connected to it.

Unfortunately iStat Menus couldn't read the SMART status from the Sabrent drive : it reported the temps from every drive in my system but this one...

Transferring 1.27 TB took exactly 20 minutes, or around 1.08 GB/sec. Transfer speed started at 1.3 GB/sec for the first minute before dropping to 1.05 GB/sec for the rest of the transfer. The enclosure got very hot to the touch after a few minutes. Temps for the Firecuda drives (source of the copy) went from 20°C / 68°F (idle) to 40°C / 104°F (just after the copy operation finished).

UPDATE : iStat Menus is reading the temp for the Sabrent / Acasis combo on the M1 Max MBP. I'll do some tests this weekend and report back here.

UPDATE 2 : the temp reported by iStat Menus is buggy, it never changes, even after 15 minutes of intensive writing... I tried both enclosures (Acasis and Orico), same behavior.

 

[macross18]

Hi, Do you guys know if 4th Gen NVME.M2 SSDs can work as an external with a 2018 MBP? Using an enclosure of course. Prices are quite attractive now.

 

[Mac-ra]

The 2018 has TB3 ports which are of course the primary connection type these enclosures support (along with TB4's backward compatibility and USB3 Gen 2). With an Intel CPU you shouldn’t see the noticeably slower write speeds from the "throttled" TB4 port on my M1 Pro MBP vs. the M1 Max. I don’t know this for sure since I don’t have access to an ‘18 MBP (perhaps someone else who does can weigh in) but tables on the Acasis site listing SSD speeds for various Mac models/CPU’s (including the ‘18 MBP) indicates that that’s the case.

OTOH not only will a PCIe Gen4 NVME SSD work but some will provide the fastest possible speeds in these vs. a Gen3 one (many Gen3's have max R/W or atleast write speed specs below that possible with both Gen3 and TB3/4) which is strange since these enclosures use a PCIe Gen3 NVME interface. I'm using the Gen4 WD Black SN770 in my Acasis enclosure and am getting pretty much the max expected speeds (i.e., with the M1 Pro’s ~400MB/s slower write speed vs. an M1 Max).

However not all Gen4 SSD's supposedly work well with these enclosures (like the Crucial P5 or Hynix Gold…not sure why nor have I tested these) so I'd stick with the sure-thing 980 Pro or the less-expensive/less-capable SN770 or Rocket 4 Plus (these three I did test), though the latest top-end but overkill-pricey 990 Pro, SN850x and KC3000 also supposedly work great too. There’s many recent posts here about this in this thread.

You probably won’t know exactly what performance you’ll get from your MBP with a given SSD until you try it but either way these enclosures are fabulous!

 

[macross18]

The 2018 has TB3 ports which are of course compatible with these enclosures. With an Intel CPU you shouldn’t see the noticeably slower write speeds from the "throttled" TB4 port on my M1 Pro MBP vs. the M1 Max. I don’t know this for sure since I don’t have access to an ‘18 MBP (perhaps someone else who does can weigh in) but tables on the Acasis site listing SSD speeds for various Mac models/CPU’s (including the ‘18 MBP) indicates that that’s the case.

OTOH not only will a PCIe Gen4 NVME SSD work but some will provide the fastest speeds in these vs. a Gen3 one (many Gen3's have max R/W or atleast write speed specs below that possible with both Gen3 and TB3/4) which is strange since these enclosures use a PCIe Gen3 NVME interface. I'm using the Gen4 WD Black SN770 in my Acasis enclosure and am getting pretty much the max expected speeds (i.e., with the M1 Pro’s ~400MB/s slower write speed vs. an M1 Max).

However not all Gen4 SSD's work well with these enclosures (like the Crucial P5 or Hynix Gold…not sure why) so I'd stick with the sure-thing 980 Pro or the less-expensive/less-capable SN770 or Rocket 4 Plus, though the latest top end but overkill-pricey 990 Pro and SN850x also supposedly work great too. There’s many recent posts here about this in this thread.

You probably won’t know exactly what performance you’ll get from your MBP with a given SSD until you try it but either way these enclosures are fabulous!

Thank you for the reply. More questions if you don't mind. There's so many post and pages to read and understand. More than half I don't 😆.

So far I'm using the Kingston KC2500- ~3000 write speed with a ugreen enclosure, it works well enough on my MBP'18. When I was purchasing the KC2500, I asked about the KC3000(gen4) and was told it may not work with my motherboard. The seller didn't know I was using it as an external and I was too preoccupied to give him the full story.
My pc guy brother said the controllers? on my computer USB-C port might not be compatible even though I'm only using it as an external. That was the reason I asked.

The new Kingston Fury Renegade at 7300/7000 MB/s looks very attractive and I wanted to be sure before purchasing. Thanks again

 

[Mac-ra]

See post #672 which shows great results using the KC3000. The strange thing is that I’ve seen information from some of the enclosure manufacturers that indicate that the Kingston Fury is not recommended (see the SSD table in this Amazon listing for a Siliking enclosure). Not sure why since your 3,000MBps write is way faster than one might hope for from a Gen3 drive that isn’t supposed to have write speeds that fast and is a bit faster than the ~2,800 theoretical max for TB3/4. I assume you’re not having random disconnects from yur MBP…another problem that might lead to “not recommended”?. Anyway give it a shot and report back (I assume you can return the SSD if it does not work well).

 

[macross18]

See post #672 which shows great results using the KC3000. The strange thing is that I’ve seen information from some of the enclosure manufacturers that indicate that the Kingston Fury is not recommended (see the SSD table in this Amazon listing for a Siliking enclosure). Not sure why since your 3,000MBps write is way faster than one might hope for from a Gen3 drive that isn’t supposed to have write speeds that fast and is a bit faster than the ~2,800 theoretical max for TB3/4. I assume you’re not having random disconnects from yur MBP…another problem that might lead to “not recommended”?. Anyway give it a shot and report back (I assume you can return the SSD if it does not work well).

The MBP 18 is my work computer. I have a Mac Studio at home so the Fury would technically be the slightly better upgrade for slightly more money which is fair.
Will definitely report back once I get it. Thanks for the tips, much appreciated.

 

[Mac-ra]

I found a cheap Kingston NV2 2TB drive which I planned to use for storage and TM backups.
Did ran some tests for it in my one of my Acasis TBU401 on a Mac mini M1.

Read speed is decent but write takes a hit in the TBU401.View attachment 2139449View attachment 2139448

Yeah that's kinda expected given the mid-range Gen3-like performance specs of the NV2. I've also seen conflicting results on whether the M1 Mini's TB4 ports are heavily write-throttled like that on the M1 Air so I wonder if that's a contributing factor or not...have you used the Acasis with a higher-performance Gen4 SSD (and, if so, what were the results)? Anyway, not sure why Kinsgston makes the NV2 other than to perhaps say that they've got the least expensive Gen4 SSD available and then under their breath "but don't look at the specs". No disrespect to your buying decision of course!

 

[EncryptedUser]

Yeah that's kinda expected given the mid-range Gen3-like performance specs of the NV2. I've also seen conflicting results on whether the M1 Mini's TB4 ports are heavily write-throttled like that on the M1 Air so I wonder if that's a contributing factor or not...have you used the Acasis with a higher-performance Gen4 SSD (and, if so, what were the results)? Anyway, not sure why Kinsgston makes the NV2 other than to perhaps say that they've got the least expensive Gen4 SSD available and then under their breath "but don't look at the specs". No disrespect to your buying decision of course!

Yes I knew the NV2 would be slower and I have no problem with that as I mainly bought it for acting TM at a very attractive price.
My KC3000 and SN750 are stellar performers on in the Acasis chassis.

 

[Mac-ra]

Yes I knew the NV2 would be slower and I have no problem with that as I mainly bought it for acting TM at a very attractive price.
My KC3000 and SN750 are stellar performers on in the Acasis chassis.

What max speeds do you see from the Acasis connected to yur M1 Mini w/ the KC3000 & SN750?

 

[EncryptedUser]

What max speeds do you see from the Acasis connected to yur M1 Mini w/ the KC3000 & SN750?

Here's my post about SN750 and KC3000

https://forums.macrumors.com/threads/thunderbolt-3-m-2-nvme-ssd-enclosures.2027925/page-27?post=31295498#post-31295498

 

[Mac-ra]

Here's my post about SN750 and KC3000

https://forums.macrumors.com/threads/thunderbolt-3-m-2-nvme-ssd-enclosures.2027925/page-27?post=31295498#post-31295498

Sorry, I shoulda looked first to see if you had already posted that. Those results are great and indicate that the M1 Mini’s TB4‘s show minimal if any speed throttling despite what some of the enclosure manufacturers‘ tables show especially for the M1 Air. So I wonder if anyone knows: 1) Is there a difference between the TB4 buses or motherboard connections on the M1 Mini vs. Air (or are the reports of heavily throttled Air TB4 ports now wrong)?, and 2) Why the more powerfiul M1 Pro MBP TB4 clearly have write speeds throttled by ~500MB/s (at least on my MBP)?

 

[steve123]

Sorry, I shoulda looked first to see if you had already posted that. Those results are great and indicate that the M1 Mini’s TB4‘s show minimal if any speed throttling despite what some of the enclosure manufacturers‘ tables show especially for the M1 Air. So I wonder if anyone knows: 1) Is there a difference between the TB4 buses or motherboard connections on the M1 Mini vs. Air (or are the reports of heavily throttled Air TB4 ports now wrong)?, and 2) Why the more powerfiul M1 Pro MBP TB4 clearly have write speeds throttled by ~500MB/s (at least on my MBP)?

I have been carefully reading the information in this thread. I have a 16" MBP w. M1 Max and am considering my options to add an external SSD.

Regarding your question (1) about the difference between Mini and Air TB4 performance, perhaps the difference might be due in part to macOS throttling the speed based on thermal differences between Mini and Air? That is, to keep the Air within its thermal limits, macOS may be throttling the TB4 port?

 

[Mac-ra]

I have been carefully reading the information in this thread. I have a 16" MBP w. M1 Max and am considering my options to add an external SSD.

Regarding your question (1) about the difference between Mini and Air TB4 performance, perhaps the difference might be due in part to macOS throttling the speed based on thermal differences between Mini and Air? That is, to keep the Air within its thermal limits, macOS may be throttling the TB4 port?

Just googled the issue and found mostly "who knows why?" so the throttling being due to thermal constraints is as good as anything else I've seen. I did read somewhere on a MacRumors forum (or maybe somewhere else) that it has to do with Apple wanting to limit the number of displays on the M1 Air & M1 Pro MBP though there was no explanation about why that affects external SSD's too. Adding to the mystery is that initial reports for the M2 Air is that its TB4 ports are as speedy as that on an M1 Max machine...perhaps the M2 Pro MBP will see the same boost though that still doesn't help me understand why my M1 Pro MBP TB4's are write-throttled by ~500MB/s or about 20% (and USB3.2 Gen2 drives are also both R & W throttled by 10-20%). So, who knows!

 

[steve123]

Have you noticed if the write-throttling on your MBP M1 Pro changes over time? If the machine is idle for a while, does it start out faster and get progressively slower?

 

[Mac-ra]

Have you noticed if the write-throttling on your MBP M1 Pro changes over time? If the machine is idle for a while, does it start out faster and get progressively slower?

Not that I've noticed. I did one session when I ran Black Magic (I let it go through ~12 R/W passes and the max speeds would jump up and down (though not by much), then loaded AmorphousDisk & ran that a few times with similarly +/- results consistent w/ BM (i.e., ~200MB/s faster), and then loaded & ran ATTO also with similar results...without seeing a progressively slower-speed trend. I ran BM at various other times, and never saw a write speed faster than about 2,350 & rarely saw it slower than ~2,250 which seemed to be a random occurrence rather than a downward trend. In all cases the enclosure case did get fairly hot though not surprisingly so (i.e., too hot to touch)...the SN770 is supposedly very efficient.

 

[masotime]

This looks promising:

ZikeDrive World's First and Fastest USB4 SSD Drive

USB4 40Gbps Enclosure • Max 3.8GB/s Read and 3.1GB/s Write Speed • NVMe PCIe Gen 4X4 SSD | Check out 'ZikeDrive World's First and Fastest USB4 SSD Drive' on Indiegogo.

www.indiegogo.com