Does AppleSoc support IOMMU for directed IO?

[DearthnVader]

Does AppleSoC support PCI Passthough?

What Intel calls VT-d?

This link seems to imply that it does, but it's more talking about security/integrity of DMA on IOMMU, not really about using it for PCI Passthough.

Direct memory access protections for Mac computers

macOS supports direct memory access (DMA) for high throughput and also maintains a comprehensive set of DMA protections.

support.apple.com

 

[leman]

Does AppleSoC support PCI Passthough?

What Intel calls VT-d?

This link seems to imply that it does, but it's more talking about security/integrity of DMA on IOMMU, not really about using it for PCI Passthough.

Direct memory access protections for Mac computers

macOS supports direct memory access (DMA) for high throughput and also maintains a comprehensive set of DMA protections.

support.apple.com

You mean PCI passthrough for virtual machines? If so, no idea whether it’s supported by the chip, but Apples virtualization frameworks definitely dont support it.

P.S. DMA is something completely different. That’s the ability to access RAM without involving the CPU.

 

[DearthnVader]

You mean PCI passthrough for virtual machines? If so, no idea whether it’s supported by the chip, but Apples virtualization frameworks definitely dont support it.

P.S. DMA is something completely different. That’s the ability to access RAM without involving the CPU.

Well, Apples virtualization frameworks could add support at some point, if the hardware supports it, thus the question.

 

[joevt]

You mean PCI passthrough for virtual machines? If so, no idea whether it’s supported by the chip, but Apples virtualization frameworks definitely dont support it.

P.S. DMA is something completely different. That’s the ability to access RAM without involving the CPU.

Also, it might be a feature that the AsahiLinux people might want to enable, if it exists and they can figure out how.

But the only PCI that currently exists for Mac M-Series CPUs is through Thunderbolt. Has PCI passthrough been done on PCs for Thunderbolt devices?

 

[DearthnVader]

Has PCI passthrough been done on PCs for Thunderbolt devices?

Yes, I've read reports that it works.

Tho it comes down to the IOMMU groups, if the PCI-E( thouderbolt devices ) are grouped with other devices needed by the host. All the devices in a group must be passed to the guest.

 

[joevt]

Tho it comes down to the IOMMU groups, if the PCI-E( thouderbolt devices ) are grouped with other devices needed by the host. All the devices in a group must be passed to the guest.

I think each Thunderbolt port is a separate PCI segment.
That means multiple devices connected to different Thunderbolt ports can have the same PCI bus number.
I don''t know if that means the roots are in different IOMMU groups. Possibly not.

 

[DearthnVader]

I think each Thunderbolt port is a separate PCI segment.
That means multiple devices connected to different Thunderbolt ports can have the same PCI bus number.
I don''t know if that means the roots are in different IOMMU groups. Possibly not.

I found some documentation about Arm64 support for IOMMU and VFIO-PCI from 2016, so I think this stuff got upstreamed in the Linux kernel. Mainly, just as the Apple document I linked in post#1 it has to do with MSI's. So if AppleSoC supports VT-d for directed I/O( as Intel calls it ) I think there is a good chance the Linux kernel supports it and the VFIO-PCI module.

We'll have to wait and see if AsahiLinux can get support for the thunderbolt ports to work for M1/M2, but likely we could get an output of the groups of the PCI-E devices in AsahiLinux if we enable IOMMU at boot time even without Thunderbolt working.

I have the 13" M2 MBP, so maybe I can install AsahiLinux on an external drive, if I can boot from it, to try it out and see.

 

[leman]

I found some documentation about Arm64 support for IOMMU and VFIO-PCI from 2016, so I think this stuff got upstreamed in the Linux kernel. Mainly, just as the Apple document I linked in post#1 it has to do with MSI's. So if AppleSoC supports VT-d for directed I/O( as Intel calls it ) I think there is a good chance the Linux kernel supports it and the VFIO-PCI module.

We'll have to wait and see if AsahiLinux can get support for the thunderbolt ports to work for M1/M2, but likely we could get an output of the groups of the PCI-E devices in AsahiLinux if we enable IOMMU at boot time even without Thunderbolt working.

I have the 13" M2 MBP, so maybe I can install AsahiLinux on an external drive, if I can boot from it, to try it out and see.

I was under impression that Apples IOMMU was fully custom and does not adhere to ARM specifications.

 

[DearthnVader]

I was under impression that Apples IOMMU was fully custom and does not adhere to ARM specifications.

That could be the case, but what I read about VFIO-PCI and AppleSoC, in the way they treat interrupts is consistent with Arm64( MSI's ).

Tho, given what you have said, there may well be some work to do in the Linux kernel for VT-d to work with AppleSoC, if the hardware can even support PCI-Passthough.

 

[ondioline]

On Apple Silicon each device has its own IOMMU, it’s not the same as Intel. In MacOS you have to interface with the devices through DriverKit:

IODMACommand | Apple Developer Documentation

An object that converts memory references to I/O bus addresses.

developer.apple.com

I’m sure “passthrough” is possible but it’s not implemented in HVF natively.

 

[Gerdi]

Does AppleSoC support PCI Passthough?

What Intel calls VT-d?

This link seems to imply that it does, but it's more talking about security/integrity of DMA on IOMMU, not really about using it for PCI Passthough.

Direct memory access protections for Mac computers

macOS supports direct memory access (DMA) for high throughput and also maintains a comprehensive set of DMA protections.

support.apple.com

I am wondering, what you are wanting to achieve here from use-case perspective. Typically a pass-through device is not available to the host anymore, in addition you need a driver for the device on the guest side - so i am wondering about the use-case.

 

[leman]

I am wondering, what you are wanting to achieve here from use-case perspective. Typically a pass-through device is not available to the host anymore, in addition you need a driver for the device on the guest side - so i am wondering about the use-case.

An obvious one would be the use of external GPUs. If one could for example use CUDA from a Linux VM with an Nvidia GPU, Apple Silicon would become much more attractive to many people doing or using ML.

 

[DearthnVader]

I am wondering, what you are wanting to achieve here from use-case perspective. Typically a pass-through device is not available to the host anymore, in addition you need a driver for the device on the guest side - so i am wondering about the use-case.

What does anyone do with virtual machines?

We run software we want to run, and some of that software relies an PCI-E hardware.

 

[LinkRS]

What does anyone do with virtual machines?

We run software we want to run, and some of that software relies an PCI-E hardware.

Howdy DearthnVader,

When folks talk about PCIe passthrough, it normally is in relation to passing through to a Virtual Machine. A common usecase for this, is to install a second dedicated GPU into a spare PCIe slot, and assigning it to a VM, giving it near native GPU performance. If you are not trying to pass-through to a VM, there would be no reason to need that feature. A hypothetical Mac that had both PCIe slots AND Apple Silicon, would not need to use PCIe passthrough, as macOS would allow native access to any software running.

However, your response leads me to a question, what do you want to do with PCIe passthrough, that is not related to a Virtual Machine? Thanks!

Rich S.

 

[DearthnVader]

Howdy DearthnVader,

When folks talk about PCIe passthrough, it normally is in relation to passing through to a Virtual Machine. A common usecase for this, is to install a second dedicated GPU into a spare PCIe slot, and assigning it to a VM, giving it near native GPU performance. If you are not trying to pass-through to a VM, there would be no reason to need that feature. A hypothetical Mac that had both PCIe slots AND Apple Silicon, would not need to use PCIe passthrough, as macOS would allow native access to any software running.

However, your response leads me to a question, what do you want to do with PCIe passthrough, that is not related to a Virtual Machine? Thanks!

Rich S.

I want to use it WITH may virtual machines, of course.

 

[DearthnVader]

So to answer the question, yes, it does seem my 2022 MacBook Pro 13" M2 does support IOMMU for PCI Passthough.

Sadly there only seems to be one PCI device in the system. I would have thought the USB and Thunderbolt controllers would have been PCI devices, but they don't seem to be:

 ./groups.sh       
IOMMU Group 0 lspci: -s: Invalid slot number
IOMMU Group 0 lspci: -s: Invalid slot number
IOMMU Group 1 lspci: -s: Invalid slot number
IOMMU Group 2 lspci: -s: Invalid slot number
IOMMU Group 3 lspci: -s: Invalid slot number
IOMMU Group 4 01:00.0 Network controller [0280]: Broadcom Inc. and subsidiaries BRCM4378 Wireless Network Adapter [14e4:4425] (rev 05)
IOMMU Group 4 01:00.1 Network controller [0280]: Broadcom Inc. and subsidiaries Device [14e4:5f69] (rev 05)
jam@jam-mac:~ -$ lspci -nnk
00:00.0 PCI bridge [0604]: Apple Inc. Device [106b:100c] (rev 01)
        Kernel driver in use: pcieport
01:00.0 Network controller [0280]: Broadcom Inc. and subsidiaries BRCM4378 Wireless Network Adapter [14e4:4425] (rev 05)
        Subsystem: Apple Inc. Device [106b:4378]
        Kernel driver in use: brcmfmac
        Kernel modules: brcmfmac
01:00.1 Network controller [0280]: Broadcom Inc. and subsidiaries Device [14e4:5f69] (rev 05)
        Subsystem: Apple Inc. Device [106b:4378]
        Kernel driver in use: hci_bcm4377
        Kernel modules: hci_bcm4377
jam@jam-mac:~ -$ lspci
00:00.0 PCI bridge: Apple Inc. Device 100c (rev 01)
01:00.0 Network controller: Broadcom Inc. and subsidiaries BRCM4378 Wireless Network Adapter (rev 05)
01:00.1 Network controller: Broadcom Inc. and subsidiaries Device 5f69 (rev 05)

Linux on the M2 doesn't support Thunderbolt yet, so I don't know if an external Thunderbolt to PCI-E would show up as a PCI device????

 

[mr_roboto]

So to answer the question, yes, it does seem my 2022 MacBook Pro 13" M2 does support IOMMU for PCI Passthough.

Sadly there only seems to be one PCI device in the system. I would have thought the USB and Thunderbolt controllers would have been PCI devices, but they don't seem to be:

Nothing sad about it, IMO. The M2 is highly integrated: almost every peripheral is part of the same system-on-chip. Internal SoC peripherals like USB controllers and so forth connect to Apple's on-chip interconnect, which is much higher performance than PCIe.

I should have replied to this topic before, but Apple SoCs have substantially more IOMMU capability than Intel systems in that Intel uses one IOMMU to serve the whole system, while Apple Silicon has dedicated IOMMUs per subsystem. Also, when the Apple page talked about security, you can read that as being the same thing as VM passthrough. Fundamentally, security and VM passthrough are two faces of the same coin: providing guarantees that an IO device can't read or write memory it shouldn't be able to. (For VM passthrough, you also need to set up regular MMU page tables to give the guest direct access to hardware registers, but that isn't an IOMMU function.)

Linux on the M2 doesn't support Thunderbolt yet, so I don't know if an external Thunderbolt to PCI-E would show up as a PCI device????

It would.

 

[joevt]

So to answer the question, yes, it does seem my 2022 MacBook Pro 13" M2 does support IOMMU for PCI Passthough.

Sadly there only seems to be one PCI device in the system. I would have thought the USB and Thunderbolt controllers would have been PCI devices, but they don't seem to be:

 ./groups.sh       
IOMMU Group 0 lspci: -s: Invalid slot number
IOMMU Group 0 lspci: -s: Invalid slot number
IOMMU Group 1 lspci: -s: Invalid slot number
IOMMU Group 2 lspci: -s: Invalid slot number
IOMMU Group 3 lspci: -s: Invalid slot number
IOMMU Group 4 01:00.0 Network controller [0280]: Broadcom Inc. and subsidiaries BRCM4378 Wireless Network Adapter [14e4:4425] (rev 05)
IOMMU Group 4 01:00.1 Network controller [0280]: Broadcom Inc. and subsidiaries Device [14e4:5f69] (rev 05)
jam@jam-mac:~ -$ lspci -nnk
00:00.0 PCI bridge [0604]: Apple Inc. Device [106b:100c] (rev 01)
        Kernel driver in use: pcieport
01:00.0 Network controller [0280]: Broadcom Inc. and subsidiaries BRCM4378 Wireless Network Adapter [14e4:4425] (rev 05)
        Subsystem: Apple Inc. Device [106b:4378]
        Kernel driver in use: brcmfmac
        Kernel modules: brcmfmac
01:00.1 Network controller [0280]: Broadcom Inc. and subsidiaries Device [14e4:5f69] (rev 05)
        Subsystem: Apple Inc. Device [106b:4378]
        Kernel driver in use: hci_bcm4377
        Kernel modules: hci_bcm4377
jam@jam-mac:~ -$ lspci
00:00.0 PCI bridge: Apple Inc. Device 100c (rev 01)
01:00.0 Network controller: Broadcom Inc. and subsidiaries BRCM4378 Wireless Network Adapter (rev 05)
01:00.1 Network controller: Broadcom Inc. and subsidiaries Device 5f69 (rev 05)

Linux on the M2 doesn't support Thunderbolt yet, so I don't know if an external Thunderbolt to PCI-E would show up as a PCI device????

You're running lspci in Asahi Linux?

It looks like you're showing devices from only one segment. M1 and M2 Macs have separate PCI segments for each Thunderbolt port. Like a Power Mac 9600 has two separate PCI buses.

What does lspci -nntv show?

Also, you can try my pcitree.sh script in linux. https://gist.github.com/joevt/e3cd4ff08aae06279134969c98ca3ab7

Do you have any Thunderbolt devices connected?

Use ioreg -fliw0 in macOS. Then parse the results with my ParseIORegPCILinkStatus.sh script.
https://forums.macrumors.com/threads/issues-with-usb-ports-on-m1-max.2355896/post-31367729

lspci won't work for Apple Silicon until a DirectHW.kext is created for it or maybe Apple added an interface to AppleARMPE in AppleARMPlatform.kext like they did to AppleACPIPlatformExpert in AppleACPIPlatform.kext? I haven't checked. I have a fork of pciutils but I haven't made any changes to it for Apple Silicon.

 

[DearthnVader]

Do you have any Thunderbolt devices connected?

No, I have none at this time. I was thinking of ordering the Akitio Node, but waiting until Asahi supports Thunderbolt.

You're running lspci in Asahi Linux?

Yes, sadly I can't build or install Qemu with packman at this time. Seems to be broken in Asahi at this time.

Use ioreg -fliw0 in macOS. Then parse the results with my ParseIORegPCILinkStatus.sh script.
https://forums.macrumors.com/threads/issues-with-usb-ports-on-m1-max.2355896/post-31367729

pci.txt
    | |   |   +-o pci-bridge0@0  <class IORegistryEntry:IOService:IOPCIDevice, id 0x1000002c7, registered, matched, active, busy 0 (18106 ms), retain 12>
    | |   |     |   "name" = <"pci-bridge0">
    | |   |     |   "compatible" = <"apcie-bridge">
    | |   |     |   "vendor-id" = 106b
    | |   |     |   "device-id" = 100c
    | |   |     |   "pcidebug" = 00:00.0-[01]
    | |   |     |   "IOPCIExpressCapabilities" = 66 (0x0042): Version:2 (Root Port of PCI Express Root Complex) Interrupt:0
    | |   |     |   "IOPCIExpressLinkCapabilities" = 7567380 (0x00737814): 16.0 GT/s x1 L1Supported L0s:(4,∞)µs L1:[32,64]µs ActiveReporting BandwidthNotifications ASPM Port:0
    | |   |     |   "IOPCIExpressLinkStatus" = 4113 (0x1011): 2.5 GT/s x1 SlotClocked
    | |   |       +-o wlan@0  <class IORegistryEntry:IOService:IOPCIDevice, id 0x100000753, registered, matched, active, busy 0 (18140 ms), retain 198>
    | |   |       | |   "name" = <"wlan">
    | |   |       | |   "compatible" = <"wlan-pcie,bcm4378","wlan-pcie,bcm">
    | |   |       | |   "vendor-id" = 14e4
    | |   |       | |   "subsystem-vendor-id" = 106b
    | |   |       | |   "pcidebug" = 01:00.0
    | |   |       | |   "IOPCIExpressCapabilities" = 2 (0x0002): Version:2 (PCI Express Endpoint) Interrupt:0
    | |   |       | |   "IOPCIExpressLinkCapabilities" = 4651026 (0x0046f812): 5.0 GT/s x1 L1Supported L0s:(4,∞)µs L1:[16,32)µs AllowReferenceClockRemoval ASPM Port:0
    | |   |       | |   "IOPCIExpressLinkStatus" = 12306 (0x3012): 5.0 GT/s x1 SlotClocked Active
    | |   |       +-o bluetooth-pcie@0,1  <class IORegistryEntry:IOService:IOPCIDevice, id 0x100000754, registered, matched, active, busy 0 (1 ms), retain 15>
    | |   |         |   "name" = <"bluetooth-pcie">
    | |   |         |   "compatible" = <"wlan-pcie,bcm4378","wlan-pcie,bcm">
    | |   |         |   "vendor-id" = 14e4
    | |   |         |   "subsystem-vendor-id" = 106b
    | |   |         |   "pcidebug" = 01:00.1
    | |   |         |   "IOPCIExpressCapabilities" = 2 (0x0002): Version:2 (PCI Express Endpoint) Interrupt:0
    | |   |         |   "IOPCIExpressLinkCapabilities" = 4642834 (0x0046d812): 5.0 GT/s x1 L1Supported L0s:[1,2)µs L1:[16,32)µs AllowReferenceClockRemoval ASPM Port:0
    | |   |         |   "IOPCIExpressLinkStatus" = 12306 (0x3012): 5.0 GT/s x1 SlotClocked Active
    | |   |   +-o pcic0-bridge@0  <class IORegistryEntry:IOService:IOPCIDevice, id 0x1000006df, registered, matched, active, busy 0 (0 ms), retain 12>
    | |   |     |   "name" = <"pcic0-bridge">
    | |   |     |   "compatible" = <"pciec-bridge">
    | |   |     |   "vendor-id" = 106b
    | |   |     |   "device-id" = 1012
    | |   |     |   "pcidebug" = 00:00.0-[01-80]
    | |   |     |   "IOPCIExpressCapabilities" = 66 (0x0042): Version:2 (Root Port of PCI Express Root Complex) Interrupt:0
    | |   |     |   "IOPCIExpressLinkCapabilities" = 7551233 (0x00733901): 2.5 GT/s x16 L1Supported L0s:[256,512)ns L1:[32,64]µs ActiveReporting BandwidthNotifications ASPM Port:0
    | |   |     |   "IOPCIExpressLinkStatus" = 4113 (0x1011): 2.5 GT/s x1 SlotClocked
    | |   |   +-o pcic1-bridge@0  <class IORegistryEntry:IOService:IOPCIDevice, id 0x1000005d9, registered, matched, active, busy 0 (1 ms), retain 12>
    | |   |     |   "name" = <"pcic1-bridge">
    | |   |     |   "compatible" = <"pciec-bridge">
    | |   |     |   "vendor-id" = 106b
    | |   |     |   "device-id" = 1012
    | |   |     |   "pcidebug" = 00:00.0-[01-80]
    | |   |     |   "IOPCIExpressCapabilities" = 66 (0x0042): Version:2 (Root Port of PCI Express Root Complex) Interrupt:0
    | |   |     |   "IOPCIExpressLinkCapabilities" = 7551233 (0x00733901): 2.5 GT/s x16 L1Supported L0s:[256,512)ns L1:[32,64]µs ActiveReporting BandwidthNotifications ASPM Port:0
    | |   |     |   "IOPCIExpressLinkStatus" = 4113 (0x1011): 2.5 GT/s x1 SlotClocked

 

[DearthnVader]

What does lspci -nntv show?

Also, you can try my pcitree.sh script in linux. https://gist.github.com/joevt/e3cd4ff08aae06279134969c98ca3ab7

 

[DearthnVader]

What we appear to be missing in Linux is the two PCI bridges with the device ID of 0x1012.

I'd assume these are for the ThunderBolt ports.

 

[joevt]

No, I have none at this time. I was thinking of ordering the Akitio Node, but waiting until Asahi supports Thunderbolt.

It would be interesting to see if the output of lspci would differ with Thunderbolt devices connected. We already know ioreg works fine.

Yes, sadly I can't build or install Qemu with packman at this time. Seems to be broken in Asahi at this time.

I think Qemu would show a separate list of pci devices (virtual devices) which would not map to the real pci devices.

pci.txt
    | |   |   +-o pci-bridge0@0  <class IORegistryEntry:IOService:IOPCIDevice, id 0x1000002c7, registered, matched, active, busy 0 (18106 ms), retain 12>
    | |   |     |   "name" = <"pci-bridge0">
    | |   |     |   "compatible" = <"apcie-bridge">
    | |   |     |   "vendor-id" = 106b
    | |   |     |   "device-id" = 100c
    | |   |     |   "pcidebug" = 00:00.0-[01]
    | |   |     |   "IOPCIExpressCapabilities" = 66 (0x0042): Version:2 (Root Port of PCI Express Root Complex) Interrupt:0
    | |   |     |   "IOPCIExpressLinkCapabilities" = 7567380 (0x00737814): 16.0 GT/s x1 L1Supported L0s:(4,∞)µs L1:[32,64]µs ActiveReporting BandwidthNotifications ASPM Port:0
    | |   |     |   "IOPCIExpressLinkStatus" = 4113 (0x1011): 2.5 GT/s x1 SlotClocked
    | |   |       +-o wlan@0  <class IORegistryEntry:IOService:IOPCIDevice, id 0x100000753, registered, matched, active, busy 0 (18140 ms), retain 198>
    | |   |       | |   "name" = <"wlan">
    | |   |       | |   "compatible" = <"wlan-pcie,bcm4378","wlan-pcie,bcm">
    | |   |       | |   "vendor-id" = 14e4
    | |   |       | |   "subsystem-vendor-id" = 106b
    | |   |       | |   "pcidebug" = 01:00.0
    | |   |       | |   "IOPCIExpressCapabilities" = 2 (0x0002): Version:2 (PCI Express Endpoint) Interrupt:0
    | |   |       | |   "IOPCIExpressLinkCapabilities" = 4651026 (0x0046f812): 5.0 GT/s x1 L1Supported L0s:(4,∞)µs L1:[16,32)µs AllowReferenceClockRemoval ASPM Port:0
    | |   |       | |   "IOPCIExpressLinkStatus" = 12306 (0x3012): 5.0 GT/s x1 SlotClocked Active
    | |   |       +-o bluetooth-pcie@0,1  <class IORegistryEntry:IOService:IOPCIDevice, id 0x100000754, registered, matched, active, busy 0 (1 ms), retain 15>
    | |   |         |   "name" = <"bluetooth-pcie">
    | |   |         |   "compatible" = <"wlan-pcie,bcm4378","wlan-pcie,bcm">
    | |   |         |   "vendor-id" = 14e4
    | |   |         |   "subsystem-vendor-id" = 106b
    | |   |         |   "pcidebug" = 01:00.1
    | |   |         |   "IOPCIExpressCapabilities" = 2 (0x0002): Version:2 (PCI Express Endpoint) Interrupt:0
    | |   |         |   "IOPCIExpressLinkCapabilities" = 4642834 (0x0046d812): 5.0 GT/s x1 L1Supported L0s:[1,2)µs L1:[16,32)µs AllowReferenceClockRemoval ASPM Port:0
    | |   |         |   "IOPCIExpressLinkStatus" = 12306 (0x3012): 5.0 GT/s x1 SlotClocked Active
    | |   |   +-o pcic0-bridge@0  <class IORegistryEntry:IOService:IOPCIDevice, id 0x1000006df, registered, matched, active, busy 0 (0 ms), retain 12>
    | |   |     |   "name" = <"pcic0-bridge">
    | |   |     |   "compatible" = <"pciec-bridge">
    | |   |     |   "vendor-id" = 106b
    | |   |     |   "device-id" = 1012
    | |   |     |   "pcidebug" = 00:00.0-[01-80]
    | |   |     |   "IOPCIExpressCapabilities" = 66 (0x0042): Version:2 (Root Port of PCI Express Root Complex) Interrupt:0
    | |   |     |   "IOPCIExpressLinkCapabilities" = 7551233 (0x00733901): 2.5 GT/s x16 L1Supported L0s:[256,512)ns L1:[32,64]µs ActiveReporting BandwidthNotifications ASPM Port:0
    | |   |     |   "IOPCIExpressLinkStatus" = 4113 (0x1011): 2.5 GT/s x1 SlotClocked
    | |   |   +-o pcic1-bridge@0  <class IORegistryEntry:IOService:IOPCIDevice, id 0x1000005d9, registered, matched, active, busy 0 (1 ms), retain 12>
    | |   |     |   "name" = <"pcic1-bridge">
    | |   |     |   "compatible" = <"pciec-bridge">
    | |   |     |   "vendor-id" = 106b
    | |   |     |   "device-id" = 1012
    | |   |     |   "pcidebug" = 00:00.0-[01-80]
    | |   |     |   "IOPCIExpressCapabilities" = 66 (0x0042): Version:2 (Root Port of PCI Express Root Complex) Interrupt:0
    | |   |     |   "IOPCIExpressLinkCapabilities" = 7551233 (0x00733901): 2.5 GT/s x16 L1Supported L0s:[256,512)ns L1:[32,64]µs ActiveReporting BandwidthNotifications ASPM Port:0
    | |   |     |   "IOPCIExpressLinkStatus" = 4113 (0x1011): 2.5 GT/s x1 SlotClocked

ioreg shows three separate PCI segments. The three root ports all have a PCI address of 00:00.0 which isn't possible if they were on the same PCI segment. For some reason, the Thunderbolt root ports only reserve 128 bus numbers [0x01-0x80] but that's more than an Intel PC or Mac would reserve, since the Thunderbolt controller in that case is a PCI device that needs to live with other PCI devices.

The bridges show a 2.5 GT/s x1 link but you can ignore that since that is a link between the bridge and the CPU which is not a real PCIe link. It is the same for Integrated Thunderbolt controllers in Intel CPUs. Even in discrete Thunderbolt controllers, the links inside the discrete Thunderbolt controller are not real PCIe and report 2.5 GT/s even though they perform faster than that.

What we appear to be missing in Linux is the two PCI bridges with the device ID of 0x1012.

I'd assume these are for the ThunderBolt ports.

If Apple Silicon boot firmware enumerates Thunderbolt PCI devices (so you can boot from Thunderbolt NVMe), then maybe Asahi linux can see them even if it doesn't have it's own Thunderbolt enumeration capability.

 

[DearthnVader]

I think Qemu would show a separate list of pci devices (virtual devices) which would not map to the real pci devices.

Not sure what you mean?

I wanted to test Qemu with PCI Passthough. I should be able to pass the wifi and bluetooth devices in group 4 to a virtual machine just to ensure it works, but I suppose I could just see if vfio-pci will link to the devices.

 

[deconstruct60]

Not sure what you mean?

I wanted to test Qemu with PCI Passthough. I should be able to pass the wifi and bluetooth devices in group 4 to a virtual machine just to ensure it works, but I suppose I could just see if vfio-pci will link to the devices.

Can't "pass through" if the base OS is using it. Qemu can't really do that as it is just an app sitting on top of the base macOS foundation. Pragmatically, you need to figure out raw hardware passthrough at boot time ( so host OS never takes formal control of it).

The only work around is if can 'eject' the device and remove it from a running hostOS. Not so sure that is going to happen for the builtin Wifi/Bluetooth for a Mac device. An USB plug in (ejectable) Wifi/Bluetooth dongle ( on a discrete USB controller provisioned port if guest OS is looking for generic hardware drivers. ) perhaps.

 

[DearthnVader]

Can't "pass through" if the base OS is using it. Qemu can't really do that as it is just an app sitting on top of the base macOS foundation. Pragmatically, you need to figure out raw hardware passthrough at boot time ( so host OS never takes formal control of it).

The only work around is if can 'eject' the device and remove it from a running hostOS. Not so sure that is going to happen for the builtin Wifi/Bluetooth for a Mac device. An USB plug in (ejectable) Wifi/Bluetooth dongle ( on a discrete USB controller provisioned port if guest OS is looking for generic hardware drivers. ) perhaps.

I wasn't speaking of the macOS as the host, I was speaking of Asahi as the host OS.

I know how PCI Passthrough works in Linux.