So I used you pcitree.sh script (ran it from an Ubuntu 19.04 environment) and these are the results. With the TB1 plugged in, I see the Titan Ridge JHL7540 thunderbolt controller as well as another DSL6540 Alpine Ridge controller under it, which disappeared after I unplugged the TB16 dock (leaving just the JHL7540). How does this work? Does the dock have another PCI-e TB controller in it?
This would be easier to answer if you pasted the text instead of using screenshots.
4:00:0 is the Titan Ridge upstream bridge. 05:xx are its downstream bridges. The GC-TITAN RIDGE support PCIe gen 3 speed but it's currently using gen 2 speed as expected for MacPro5,1.
05:00.0 is for its NHI at 06:00.0
05:02.0 is for its USB controller at 0b:00.0
05:01.0 is for port 1 (it has only 4 buses [07-0a] reserved for it - that's not good)
05:04.0 is for port 2 (it has only 1 bus reserved for it - that's not good)
Usually a firmware that support Thunderbolt will reserve more buses for the downstream Thunderbolt ports. But the MacPro5,1 firmware knows nothing about Thunderbolt. It's up to the OS to reserve buses. For Linux, maybe there's a setting that can be applied to a device path that says how many buses to reserve for that path when enumerating the PCI devices. For macOS, maybe a kext can tell macOS to reenumerate the PCI devices with extra reservations for the Thunderbolt ports (IOPCIConfigurator stuff).
I will eventually try to make a EFI driver that can reenumerate PCI devices for Thunderbolt.
Back to your pcitree output, when you connect the TB16 to the first port (05:01.0), there's only enough buses to enumerate the following:
07:00.0: The first Alpine Ridge upstream bridge. 08:xx are its downstream bridges.
There's not enough buses to enumerate the PCIe endpoint devices that are part of the Alpine Ridge or that are connected to the Alpine Ridge. From your screenshot in macOS, we expect another Thunderbolt controller to appear. Maybe you can see all the Thunderbolt devices with the
boltctl list -a or
tbtadm topology commands.
http://manpages.ubuntu.com/manpages/jammy/en/man1/boltctl.1.html
http://manpages.ubuntu.com/manpages/bionic/man1/tbtadm.1.html
If you restart the computer at this point, the firmware may be able to see the Alpine Ridge bridges and enumerate more downstream PCI devices?
For example, with unflashed Titan Ridge in MacPro3,1, we connect Thunderbolt devices in Windows and Windows will setup the Thunderbolt chain. Then we restart and the PCI devices in the chain will then be enumerated by the firmware and will be usable in Windows or Linux or macOS.
https://forums.macrumors.com/threads/testing-tb3-aic-with-mp-5-1.2143042/post-27612591
19.04 is old. Maybe 21.10 has better Thunderbolt support or more options. I have a few versions of Ubuntu installed, each taking about 20 GB. They all use the same swap partition. Need to research the linux kernel pci related options to see if something can improve Thunderbolt support.
https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html
Maybe
assign-busses?
Maybe some of the hot plug stuff
hpiosize,
hpmmioprefsize,
hpmemsize,
hpbussize. Does Linux consider Thunderbolt as hot plug? Maybe it depends on the HotPlug flag. This command lists devices that have the HotPlug flag:
sudo lspci -vvvnn | sed -nE '/^[0-9a-f].*/,/^$/ { /^([0-9a-f].*)/h; /HotPlug\+/ { g; p; }; }'
It may contain non-Thunderbolt slots. In that case, you want to apply the settings only to the Thunderbolt devices but the hp* settings don't have a <pci_dev> parameter? For Alpine Ridge, the USB controller might be marked as hot plug so you would want to ignore that device and make sure only the Thunderbolt ports get resources reserved for them. I suppose a proper Thunderbolt EFI driver would decide which downstream bridges correspond to a PCIe Down Adapter and only make reservations for those. I suppose the Linux kernel could be made to do that as well so it would be less dependant on firmware.
www.tonymacx86.com
