HighPoint SSD7505 notes
Pictures:
Setup:
I have the card installed in a Netstor NA255A connected to a MacPro3,1 in slot 2. The NA255A has PCIe 3.0 x8 slots (similar in bandwidth to the PCIe 2.0 x16 of slot 2). PCIe 4.0 x4 is like PCIe 2.0 x16 so I should be able to get near full performance from PCIe 4.0 NVMe devices.
The MacPro3,1 has unmodified firmware.
There are two devices on the card besides the four NVMe devices.
The Catalina System Information.app PCI and NVMExpress tabs do not show the 16 GT/s link speed of the Sabrent Rocket 2TB 4.0. I wonder if Big Sur is the same?
pcitree.sh shows the following:
Code:
┬[0000:00]
├─00:00.0 # g1x4 [8086:4003] [0600] (rev 20) Host bridge : Intel Corporation 5400 Chipset Memory Controller Hub
├┬00:01.0-[01-17] # g2x16 [8086:4021] [0604] (rev 20) PCI bridge : Intel Corporation 5400 Chipset PCI Express Port 1
│└┬01:00.0-[02-17] # g3x16 > g2x16 [10b5:8747] [0604] (rev ca) PCI bridge : PLX Technology, Inc. PEX 8747 48-Lane, 5-Port PCI Express Gen 3 (8.0 GT/s) Switch
│ ├┬02:08.0-[03-08] # g3x8 > g3x4 [10b5:8747] [0604] (rev ca) PCI bridge : PLX Technology, Inc. PEX 8747 48-Lane, 5-Port PCI Express Gen 3 (8.0 GT/s) Switch
│ │└┬03:00.0-[04-08] # g3x4 [8086:15ea] [0604] (rev 06) PCI bridge : Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018]
│ │ ├┬04:00.0-[05] # g1x4 [8086:15ea] [0604] (rev 06) PCI bridge : Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018]
│ │ │└─05:00.0 # g1x4 [8086:15eb] [0880] (rev 06) System peripheral : Intel Corporation JHL7540 Thunderbolt 3 NHI [Titan Ridge 4C 2018]
│ │ ├┬04:01.0-[06] # g1x4 [8086:15ea] [0604] (rev 06) PCI bridge : Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018]
│ │ ├┬04:02.0-[07] # g1x4 [8086:15ea] [0604] (rev 06) PCI bridge : Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018]
│ │ │└─07:00.0 # g1x4 [8086:15ec] [0c03] (rev 06) USB controller : Intel Corporation JHL7540 Thunderbolt 3 USB Controller [Titan Ridge 4C 2018]
│ │ └┬04:04.0-[08] # g1x4 [8086:15ea] [0604] (rev 06) PCI bridge : Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018]
│ ├┬02:09.0-[09-15] # g3x8 [10b5:8747] [0604] (rev ca) PCI bridge : PLX Technology, Inc. PEX 8747 48-Lane, 5-Port PCI Express Gen 3 (8.0 GT/s) Switch
│ │└┬09:00.0-[0a-15] # g4x16 > g3x8 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ ├┬0a:00.0-[0b-10] # g4x16 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ │└┬0b:00.0-[0c-10] # g4x16 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ │ ├┬0c:10.0-[0d] # g4x4 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ │ │└─0d:00.0 # g4x4 [1987:5016] [0108] (rev 01) Non-Volatile memory controller : Phison Electronics Corporation E16 PCIe4 NVMe Controller
│ │ │ ├┬0c:14.0-[0e] # g4x4 > g3x4 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ │ │└─0e:00.0 # g3x4 [1cc1:8201] [0108] (rev 03) Non-Volatile memory controller : ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive
│ │ │ ├┬0c:18.0-[0f] # g4x4 > g3x4 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ │ │└─0f:00.0 # g3x4 [144d:a804] [0108] Non-Volatile memory controller : Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961
│ │ │ └┬0c:1c.0-[10] # g4x4 > g3x4 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ │ └─10:00.0 # g3x4 [144d:a802] [0108] (rev 01) Non-Volatile memory controller : Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951
│ │ ├┬0a:0c.0-[11-14] # g4x16 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ │└┬11:00.0-[12-14] # g4x16 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ │ ├┬12:14.0-[13] # g4x1 > g2x1 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ │ │└─13:00.0 # g2x2 > g2x1 [1103:7505] [0104] (rev 01) RAID bus controller : HighPoint Technologies, Inc. Device
│ │ │ └┬12:15.0-[14] # g4x1 > g1x0 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ └┬0a:1c.0-[15] # g4x16 [1000:c010] [0604] (rev b0) PCI bridge : Broadcom / LSI Device
│ │ └─15:00.0 # g4x16 [1000:c010] [0180] (rev b0) Mass storage controller : Broadcom / LSI Device
│ ├┬02:10.0-[16] # g3x8 [10b5:8747] [0604] (rev ca) PCI bridge : PLX Technology, Inc. PEX 8747 48-Lane, 5-Port PCI Express Gen 3 (8.0 GT/s) Switch
│ │├─16:00.0 # g3x16 > g3x8 [10de:17c2] [0300] (rev a1) VGA compatible controller : NVIDIA Corporation GM200 [GeForce GTX TITAN X]
│ │└─16:00.1 # g3x16 > g3x8 [10de:0fb0] [0403] (rev a1) Audio device : NVIDIA Corporation GM200 High Definition Audio
│ └┬02:11.0-[17] # g3x8 > g3x4 [10b5:8747] [0604] (rev ca) PCI bridge : PLX Technology, Inc. PEX 8747 48-Lane, 5-Port PCI Express Gen 3 (8.0 GT/s) Switch
│ └┬17:00.0-[XX] # g3x4 [ffff:ffff] [0604] PCI bridge : Illegal Vendor ID Device
The
PCI ID Repository needs to be updated with the info for some of these devices.
Bus 00, Device 01 is slot 2.
Bus 01 is the upstream bridge of the PEX8747 of the NA255A
Bus 02 is the four downstream bridge of the PEX8747 of the NA255A
Bus 03 is the upstream bridge of a GC-TITAN RIDGE
Bus 09 1000:c010 is the upstream bridge of the
PEX88048 (50 lanes, Gen 4) of the SSD7505
The PEX layout of the SSD7505 is different than what we usually see with a PEX - it has two levels of buses instead of the usual single level.
The first level (under the PEX upstream bridge at bus 09) is 16 lanes with three downstream bridges at bus 0A:
- upstream bridge at bus 0B, four downstream bridges at bus 0C, four NVMe endpoint devices: the four NVMe devices are connected under this bridge in the next level of buses. 1987:5016 is the Sabrent Rocket 2TB 4.0 which is running at gen 4 speed. All the other NVMe devices are running at gen 3 speed.
- upstream bridge at bus 11, two downstream bridges at bus 12, one endpoint device at bus 13: 1103:7505 HighPoint RAID bus controller. Is this an external device connected with a single PCIe 2.0 lane? This maybe where the Marvell 88SE9235 is connected which is a PCIe 2.0 x2 four port SATA 6Gbps device.
- end point device at bus 15: 1000:c010 Mass storage controller. Is this a PEX software device?
Bus 16 is a Nvidia Titan X (Maxwell)
Bus 17 is a hidden GC-ALPINE RIDGE. During this boot, I did not execute the command to unhide the Thunderbolt controller.
Booting:
Some HighPoint cards may boot without EFI NVMe driver (in EFI, it treats NVMe devices as SCSI) - I still need to test this with the HighPoint SSD7505 in my MacPro3,1 which doesn't have EFI NVMe driver. It has code in the PEX chip that makes the nvme devices appear as SCSI so they can be booted by the Startup Manager (hold option at boot) without an NVMe driver. The Startup Manager will set the EFI boot device path if you hold the control key and press return on an item. If there's a mystery item in the Startup Manager then doing that will help you discover what it is. I haven't tested the boot from fake SCSI to see if it completes booting. Of course, the EFI device path created by Startup Disk preferences would point to NVMe instead of SCSI, so that method may not boot without NVMe driver (which I could try adding using the Driver#### method). That's another thing to test. Third thing to test is the hardware RAID feature - the PEX chip has code to act as a RAID. Anyway, regardless of the EFI device driver (SCSI) used to start boot.efi, there should be a successful transition to kext device driver (NVMe). The transition waits for the proper system partition to appear as a kext device - the partition is matched by volume UUID I guess.
Using the
dumpallbootvars command, here's what the boot variable looks like when selecting a Preboot volume of an APFS container on an NVMe by pressing the control key in the Startup Manager (hold option at boot):
Code:
Boot0080: 1, "", "PciRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x0)/Pci(0x9,0x0)/Pci(0x0,0x0)/Pci(0x0,0x0)/Pci(0x0,0x0)/Pci(0x1C,0x0)/Pci(0x0,0x0)/Scsi(0x2,0x0)/HD(2,GPT,0686E618-76AB-46CD-AA68-508E81F9B395,0x64028,0x5DA13B80)/VenMedia(BE74FCF7-0B7C-49F3-9147-01F4042E6842,4FA09B303BC86E498E61CB17F0D6A2FA)/\5CF6AC3D-4597-439F-8501-E07149A248C5\System\Library\CoreServices\boot.efi"
The EFI device path is using the UEFI 2.x format. It will appear slightly differently in EFI 1.1 - here's the partition part in EFI 1.1 format:
Acpi(PNP0A03,0)/Pci(1|0)/Pci(0|0)/Pci(9|0)/Pci(0|0)/Pci(0|0)/Pci(0|0)/Pci(1C|0)/Pci(0|0)/Scsi(Pun2,Lun0)/HD(Part2,Sig0686E618-76AB-46CD-AA68-508E81F9B395)
In either case, you see that it is a Scsi device. I have not tried this with the NVMe efi driver installed into the
Driver#### nvram variable. Maybe this Preboot volume will appear as an NVMe, or maybe it will appear as both a Scsi and an NVMe.
The path is pointing to the NVMe device at bus 10, not at either of the controllers at bus 13 or 15. So how did a Scsi protocol get attached to an NVMe device?
In EFI, each of the four NVMe devices is a "HighPoint RocketNVMe Controller". Under the fourth at (Device 1C), is attached a "HPT VD0-# SCSI Disk Device" for each NVMe (where # is 0,1,2,3) and a "HPT RCM Device". Under each SCSI device is the partitions. These devices are all controlled by the "HighPoint RocketNVMe driver". There is a "Samsung Electronics NVMe Driver" (probably from the 960 Pro's PCI option rom? I need to dump it to find out) but it's not attached to a device so maybe it got overridden by the Highpoint driver. The device at bus 13 also has an option rom - maybe this is where the Highpoint driver is stored? (need to dump it to find out).
My MacPro3,1 currently cannot boot into Windows using legacy BIOS boot (I have not tried Windows EFI boot). It might be because of the PCI option roms of the Highpoint, or Titan X or something. I should see if I can make an EFI driver to bypass PCI legacy BIOS option roms (my current idea is to just set rom size and location to 0 for each PCI protocol device - it may work if the legacy booter does not read the rom from PCI).
Benchmarks:
Different benchmarks will give different numbers. sequential read usually gives the highest value.
1st highest is AmorphousDiskMark. If you want to see the max speed from your storage device, then click the SEQ1M QD8 button.
2nd is AJA System Test Lite.
3rd is Blackmagic Disk Speed Test.
I've just started using ATTO Disk Benchmark. I like how it shows how different transfer sizes changes performance. The max number is a little less than that from Amorphous Disk Mark. Below are pictures of benchmarks for a Sabrent 2TB Rocket NVMe PCIe 4.0 connected to my MacPro3,1 with a Highpoint SSD7505 (PCIe 4.0 x16 to four M.2 card):
ATTO Disk Benchmark also has the ability to test multiple disks at once even if they're not in a RAID (My HighPoint SSD7505 currently has 4 NVMe's attached - the max 6 GB/s shows the limit of my PCIe 2.0 slot):
Noise:
The fan is larger and therefore quieter than the one on the Amfeltec Squid gen 3 carrier.
Heat:
The Amfeltec didn't have a heat sink but this Highpoint does. I haven't measured temperatures in either case.
Software:
Untested. It has the usual web interface for creating hardware RAID. I need to test how it works and if it's bootable.