MP 1,1-5,1 Mac OS 11.3 has broken support for older Mac Pros

[VitaminK]

macOS Big Sur 11.3 Beta 3 has broken Mac Pro's early-2009 to mid-2012 ability to boot reliably into macOS.

It's still possible to boot 11.3/11.4, but the process requires multiple restarts and a connection has been made between PCIe devices, especially storage related devices and USB controller cards, causing panics at boot. While most people report NVMe crashes, the issue is not NVMe related and PCIe SATA cards or AHCI PCIe M.2 devices, including Apple 12+16 AHCI PCIe models, also have the same crashes. It's probably an unknown race condition somewhere early in the kernel that is causing the crashes.

Some Mac Pros are less affected with the 11.3.1/11.4/11.5bx crashes and seems that having fewer PCIe devices installed improve the rate of successful boots. Other non-storage related PCIe devices also seem to contribute to the crashes like the older AirPort Extreme models - at this moment the best success rate of an AirPort Extreme model seems to be with the BCM943602CDPAX AirPort Extreme + bluetooth 4.1 combo card, but this card have unrelated issues when you boot Mojave from BigSur.

Please check if your NVRAM volume garbage collection is working before trying to test 11.3.1/11.4/11.5bx, if you have a Mac Pro with the VSS stores full, you could brick it with the constant crashes at boot - several people already have, be warned.

OpenCore on the Mac Pro

There is a extremely simple way (simple here as in not need to know how the NVRAM works/check free space indicators/validate checksums and etc) to check if the garbage collection failed and the need to reflash with the clean dump: Dump the BootROM with ROMTool Open the dump with the most...

forums.macrumors.com

The crashes still continue to happen with the current 11.4 (20F71) release version and with the developers beta release, 11.5b2 (20G5033c).

To the users that do not have issues while booting 11.3, test at least 10 reboots before reporting successful then please list any PCIe cards, PCIe storage and the model of the AirPort Extreme you may have installed.


For detailed panic logs see here:

Mac Pro 11.3 Beta NVME issues · Issue #110 · dortania/OpenCore-Legacy-Patcher

Describe the bug Im not sure this will fall within the scope of this patcher, Apple's changes in IONvmefamily has broken support with Mac Pros 5,1/4,1/3,1 the bug started with Beta 3 of 11.3 has co...

github.com

Note: MacPro4,1 and MacPro5,1 are not the only machines affected. These symptoms can be replicated less frequently by @khronokernel on MacPro3,1, iMac8,1, iMac11,2, MacBook7,1 and MacBookPro8,3 as well as by ASentientBot on a Penryn Hackintosh. Therefore this race condition is not limited by the CPU architecture and instead by the CPU's speed and number of PCI devices present for device setup

Big Sur 11.2.3 direct download link:​

Big Sur 11.2.3 direct download from Apple servers: http://swcdn.apple.com/content/down...i7fezrmvu4vuab80m0e8a5ll/InstallAssistant.pkg

 

[Syncretic]

Can you (or someone) post a copy of the panic report? It's always helpful to understand the details of what's happening, even if the problem appears to be insoluble.

 

[Steve_Jones]

I think your title is a bit overly dramatic. Not all 5,1’s have NVMe drives.

 

[VitaminK]

Maybe a tad, but I wouldn't want anyone walking into the issues ill had today. The issue isn't limited to NVME drivers, more a combination of PCI cards causing freezing during boot noticeable in verbose mode, the issue has been all over the place for me.

 

[goMac]

Technically macOS 11 has never supported older Mac Pros. If you’re running it on a 5,1 or older things are probably going to break...

5,1s are no longer in their test set. I wouldn’t count on them fixing anything. They have no reason to fix it unless it’s also affecting 7,1s.

 

[startergo]

Technically macOS 11 has never supported older Mac Pros. If you’re running it on a 5,1 or older things are probably going to break...

5,1s are no longer in their test set. I wouldn’t count on them fixing anything. They have no reason to fix it unless it’s also affecting 7,1s.

Up to 11.2 the cMP was perfectly running BS on a NVME PCIE drive. The only thing needed was -no_compat_check boot argument. I can confirm the issue with the NVME drivers. I will try to update the NVME driver inside the firmware from the MP6,1 firmware.

 

[cdf]

This issue was discussed and ultimately closed here:

Mac Pro 11.3 Beta NVME issues · Issue #110 · dortania/OpenCore-Legacy-Patcher

Describe the bug Im not sure this will fall within the scope of this patcher, Apple's changes in IONvmefamily has broken support with Mac Pros 5,1/4,1/3,1 the bug started with Beta 3 of 11.3 has co...

github.com

 

[startergo]

This issue was discussed and ultimately closed here:

Mac Pro 11.3 Beta NVME issues · Issue #110 · dortania/OpenCore-Legacy-Patcher

Describe the bug Im not sure this will fall within the scope of this patcher, Apple's changes in IONvmefamily has broken support with Mac Pros 5,1/4,1/3,1 the bug started with Beta 3 of 11.3 has co...

github.com

It was closed as it was considered a hardware fault, which I don't think is the case here.

 

[cdf]

Indeed. Although I haven't investigated the issue, it does seem rather widespread.

 

[tsialex]

Indeed. Although I haven't investigated the issue, it does seem rather widespread.

Weird that APFSJumpStart EFI module inside the MP5,1 EFI firmware is not loading the correct/updated APFS EFI module from the Big Sur EFI partition…

 

[cdf]

Weird that APFSJumpStart EFI module inside the MP5,1 EFI firmware is not loading the correct/updated APFS EFI module from the Big Sur EFI partition…

Is that what's going on here? Perhaps enabling EnableJumpstart in OC can get the correct module to load...

 

[Flocarino]

How about a PCIe NMVE blade in a 2019 Imac 4k which is supported by BS, will it cause a crash or......

 

[Macschrauber]

11.3 Beta even crashes on a AHCI Apple Blade, btw

Also I noticed that using a Kepler GPU without UGA Bootscreen Rom does not crash when the Apple AHCI Blade was in.

So that's a second data point for PCIe Bus Problems.

A Buddy reported that using a USB3 PCIe Card also crashed 11.3 Beta 4 and 5.

So maybe it's not NVMe but a problem with talking to the PCI Bus at all?

 

[VitaminK]

11.3 beta 5 will boot from an NVME, ill had more success with booting after an initial restart from a working Mac OS or let the system kernel panic restart decrease the chance of a halt from assert failed,

 

[startergo]

panic(cpu 20 caller 0xffffff800d9bee36): Kernel trap at 0xffffff801000f109, type 14=page fault, registers:
CR0: 0x000000008001003b, CR2: 0x0000000000000000, CR3: 0x0000000011c35000, CR4: 0x00000000000226e0
RAX: 0x0000000000000100, RBX: 0x0000000000000000, RCX: 0x0000000000000000, RDX: 0x0000000000000009
RSP: 0xffffffa212c1bd80, RBP: 0xffffffa212c1bdb0, RSI: 0x0000000905243fd3, RDI: 0xffffff94206c4800
R8:  0xffffff9420796640, R9:  0xffffff8754905580, R10: 0x0000000000000001, R11: 0xffffff942092d280
R12: 0xffffff942092d280, R13: 0xffffff801000dfee, R14: 0x0000000905243fd3, R15: 0xffffff94206c4800
RFL: 0x0000000000010246, RIP: 0xffffff801000f109, CS:  0x0000000000000008, SS:  0x0000000000000000
Fault CR2: 0x0000000000000000, Error code: 0x0000000000000000, Fault CPU: 0x14, PL: 0, VF: 1

Backtrace (CPU 20), Frame : Return Address
0xffffffa212c1b7a0 : 0xffffff800d887f3d mach_kernel : _handle_debugger_trap + 0x3fd
0xffffffa212c1b7f0 : 0xffffff800d9ce493 mach_kernel : _kdp_i386_trap + 0x143
0xffffffa212c1b830 : 0xffffff800d9bea6a mach_kernel : _kernel_trap + 0x55a
0xffffffa212c1b880 : 0xffffff800d82ca2f mach_kernel : _return_from_trap + 0xff
0xffffffa212c1b8a0 : 0xffffff800d88775d mach_kernel : _DebuggerTrapWithState + 0xad
0xffffffa212c1b9c0 : 0xffffff800d887a53 mach_kernel : _panic_trap_to_debugger + 0x273
0xffffffa212c1ba30 : 0xffffff800e095c9a mach_kernel : _panic + 0x54
0xffffffa212c1baa0 : 0xffffff800d9bee36 mach_kernel : _sync_iss_to_iks + 0x2c6
0xffffffa212c1bc20 : 0xffffff800d9beb1d mach_kernel : _kernel_trap + 0x60d
0xffffffa212c1bc70 : 0xffffff800d82ca2f mach_kernel : _return_from_trap + 0xff
0xffffffa212c1bc90 : 0xffffff801000f109 com.apple.iokit.IONVMeFamily : __ZN16IONVMeController18GetOveragedRequestEy + 0x27
0xffffffa212c1bdb0 : 0xffffff801000e04f com.apple.iokit.IONVMeFamily : __ZN16IONVMeController15CheckForTimeoutEP18IOTimerEventSource + 0x61
0xffffffa212c1bde0 : 0xffffff800dfeddb5 mach_kernel : __ZN18IOTimerEventSource15timeoutSignaledEPvS0_ + 0xa5
0xffffffa212c1be50 : 0xffffff800dfedcb6 mach_kernel : __ZN18IOTimerEventSource17timeoutAndReleaseEPvS0_ + 0xc6
0xffffffa212c1be80 : 0xffffff800d8ce315 mach_kernel : _thread_call_delayed_timer + 0x4a5
0xffffffa212c1bef0 : 0xffffff800d8cf284 mach_kernel : _thread_call_delayed_timer + 0x1414
0xffffffa212c1bfa0 : 0xffffff800d82c13e mach_kernel : _call_continuation + 0x2e
      Kernel Extensions in backtrace:
         com.apple.iokit.IONVMeFamily(2.1)[204D1783-899F-353F-84CB-D7F6862AE265]@0xffffff8010001000->0xffffff801002afff
            dependency: com.apple.driver.AppleEFINVRAM(2.1)[055F51C0-3FE9-31C1-8CC8-2ADC456E3C28]@0xffffff800ecf1000->0xffffff800ecfafff
            dependency: com.apple.driver.AppleMobileFileIntegrity(1.0.5)[08C786C4-D093-3AE9-A378-7A716759967F]@0xffffff800eeab000->0xffffff800eec0fff
            dependency: com.apple.iokit.IOPCIFamily(2.9)[368F5229-F604-3F4F-B877-05DAB5072243]@0xffffff80102c0000->0xffffff80102e8fff
            dependency: com.apple.iokit.IOReportFamily(47)[21571942-1055-35FE-935F-544ADB4A43A5]@0xffffff80102f7000->0xffffff80102f9fff
            dependency: com.apple.iokit.IOStorageFamily(2.1)[063C627D-49F0-356D-9B37-EE3D587BECC2]@0xffffff80103c3000->0xffffff80103d4fff

Process name corresponding to current thread: kernel_task
Boot args: -v -no_compat_check keepsyms=1 debug=0x144 no32exec=0

Mac OS version:
Not yet set

Kernel version:
Darwin Kernel Version 20.4.0: Tue Feb 23 23:43:36 PST 2021; root:xnu-7195.100.354.131.1~3/RELEASE_X86_64
Kernel UUID: 638E017A-1AC0-30AB-BB38-C4C34BEF8D3A
KernelCache slide: 0x000000000d600000
KernelCache base:  0xffffff800d800000
Kernel slide:      0x000000000d610000
Kernel text base:  0xffffff800d810000
__HIB  text base: 0xffffff800d700000
System model name: MacPro5,1 (Mac-7BA5B2D9E42DDD94)
System shutdown begun: NO
Panic diags file unavailable, panic occurred prior to initialization
Hibernation exit count: 0

System uptime in nanoseconds: 39862752695
Last Sleep:           absolute           base_tsc          base_nano
  Uptime  : 0x000000094ade257a
  Sleep   : 0x0000000000000000 0x0000000000000000 0x0000000000000000
  Wake    : 0x0000000000000000 0x0000003216a3d58c 0x0000000000000000

Above is the panic log and AppleNvme assert failed just before the panic:

I also tried updating the latest NVME and APFS jump start drivers from MP61.scap

 

[VitaminK]

Which PCI cards our you using?

 

[startergo]

Which PCI cards our you using?

Media name : OWC Aura P12 480GB Media

 

[MisterAndrew]

It seems weird to me to complain about a compatibility issue with a non-supported system. Big Sur isn’t suppose to work on 5,1 and older. You need a 6,1 or newer for Big Sur.

 

[startergo]

Big Sur isn’t suppose to work on 5,1 and older.

The accurate statement is :
is not designed with respect to cMP and older. As it was demonstrated many times before the only thing, which was needed to run Big Sur was _no_compat_check boot argument. Big Sur does not support USB1 and some older wifi cards. Other than that everything worked fine.

 

[MisterAndrew]

The accurate statement is :
is not designed with respect to cMP and older. As it was demonstrated many times before the only thing, which was needed to run Big Sur was _no_compat_check boot argument. Big Sur does not support USB1 and some older wifi cards. Other than that everything worked fine.

Maybe it would run fine, but I'm sure there are more complicated compatibility issues than that.

 

[VitaminK]

Isn’t a maybe.. If a computer has a compatible CPU typically Mac OS will boot older unsupported Mac’s especially.

As our Mac’s have native hardware Mac OS is looking like SMC, an EC , and correct DSDT & firmware. Very rarely Apple has fully dropped an unsupported Mac small parts stay for awhile like audio layouts, USB mapping , Ethernet , power management , Sata , Cd/DVD burning. Apple so far has dropped WiFi/Bluetooth, original Non metal GPUs from Mac Pro 5,1..

 

[startergo]

To get back BS working I had to create a new BS 11.2.3 partition, then from within BS install it to the unbootable partition. From the bootpicker with bootable USB installer it said it cannot "downgrade" BS. Hopefully that will be fixed in the latest major BS release.
Also Safari for later beta cannot run on earlier released version (if reinstalled), so I had to manually copy the contents of the Safari package from the freshly installed Big Sur partition to the old Safari package and
/Library/Apple/System/Library/CoreServices/SafariSupport.bundle

 

[startergo]

What is interesting is that the Assert failed is not the reason for the kernel panic. Here is the Bootlog_Kernel of Catalina:

2021-03-27 07:15:10.67 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.68 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.68 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.71 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.71 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.71 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.71 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.98 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:11.05 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:11.05 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:11.06 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:11.06 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:11.06 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64

 

[startergo]

This may be a way to disable loading of IONvmeFamily:
[Guide] HackrNVMeFamily co-existence with IONVMeFamily using class-code spoof | tonymacx86.com

 

[VitaminK]

I don’t believe the issue has todo with an NVME any longer but PCI cards them self become an issue. Apple has focused heavily in 11.3 improving 7,1 Mac Pros these changes could be the issue during boot but after Beta 5 our Pros still have full compatibility.

for reference I use a Sonnet NVME, Titan ridge, Sonnet Asmedia 1142, and Vega FE