MP 7,1 Mac Pro (2019) sudden shutdowns/ sleep issues

[machenryr]

I too have had a few unexpected shutdowns. 2019 7,1 16 core 4TB 32 GB.

 

[thexash]

For data collection’s sake: I only had one crash caused by Viscosity’s kext while the Mac was sleeping. I uninstalled that and I’m using Tunnelblick now.

I know for a fact its Viscosity because I installed it, put the computer to sleep an hour later and it crashed when I tried to turn it back on it rebooted and the error report mentioned a kernel panic in a Viscosity kext.

Tunnelblick doesnt seem to need a kext at all which is much better anyway.

 

[daveedjackson]

Just had my first random shutdown overnight. Was doing a backup to an external drive. Not using anything other than drag and drop. Came to it this morning, and the system was off.
Kernel Panic:


panic(cpu 0 caller 0xffffff80112288b7): "AppleAHCIDiskQueueManager::setPowerState(0xffffff80d5751d80 : 0xffffff7f96478958, 2 -> 3) timed out after 101141 ms"@/BuildRoot/Library/Caches/com.apple.xbs/Sources/xnu/xnu-6153.81.5/iokit/Kernel/IOServicePM.cpp:5302
Backtrace (CPU 0), Frame : Return Address
0xffffffbad384bb40 : 0xffffff8010b3bb2b
0xffffffbad384bb90 : 0xffffff8010c734d5
0xffffffbad384bbd0 : 0xffffff8010c64f4e
0xffffffbad384bc20 : 0xffffff8010ae2a40
0xffffffbad384bc40 : 0xffffff8010b3b217
0xffffffbad384bd40 : 0xffffff8010b3b5fb
0xffffffbad384bd90 : 0xffffff80112d2aa9
0xffffffbad384be00 : 0xffffff80112288b7
0xffffffbad384be50 : 0xffffff8011228479
0xffffffbad384be60 : 0xffffff801123f82e
0xffffffbad384bea0 : 0xffffff8011227228
0xffffffbad384bec0 : 0xffffff8010b7d7e5
0xffffffbad384bf40 : 0xffffff8010b7d311
0xffffffbad384bfa0 : 0xffffff8010ae213e

BSD process name corresponding to current thread: kernel_task
Boot args: chunklist-security-epoch=0 -chunklist-no-rev2-dev

Mac OS version:
19D76

Kernel version:
Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64
Kernel UUID: A8DDE75C-CD97-3C37-B35D-1070CC50D2CE
Kernel slide: 0x0000000010800000
Kernel text base: 0xffffff8010a00000
__HIB text base: 0xffffff8010900000
System model name: MacPro7,1 (Mac-27AD2F918AE68F61)
System shutdown begun: NO

System uptime in nanoseconds: 216339733159507
last loaded kext at 150960714167945: |IOAHCISerialATAPI 268 (addr 0xffffff7f94bb2000, size 49152)
last unloaded kext at 151059052284118: >!UAudio 320.49 (addr 0xffffff7f9640b000, size 434176)
loaded kexts:
com.getdropbox.dropbox.kext 1.10.3
|SCSITaskUserClient 422.0.2
>!ATopCaseHIDEventDriver 3430.1
>AudioAUUC 1.70
>!AUpstreamUserClient 3.6.8
>!AMCCSControl 1.13
@kext.AMDFramebuffer 3.0.5
@kext.AMDRadeonX4000 3.0.5
@kext.AMDRadeonServiceManager 3.0.5
>!AGraphicsDevicePolicy 4.7.2
@fileutil 20.036.15
@AGDCPluginDisplayMetrics 4.7.2
>!AHV 1
|IOUserEthernet 1.0.1
|IO!BSerialManager 7.0.3f5
>AGPM 111.4.2
>!APlatformEnabler 2.7.0d0
>X86PlatformShim 1.0.0
>pmtelemetry 1
@Dont_Steal_Mac_OS_X 7.0.0
@kext.AMD9500!C 3.0.5
>BridgeAudioCommunication 6.66
>!AThunderboltIP 3.1.3
>!ABridgeAudio!C 6.66
>!AGFXHDA 100.1.424
>!A!IPCHPMC 2.0.1
>!AAVEBridge 6.1
>!A!ISlowAdaptiveClocking 4.0.0
>!A!IMCEReporter 115
@filesystems.autofs 3.0
>BCMWLANFirmware4355.Hashstore 1
>BCMWLANFirmware4364.Hashstore 1
>BCMWLANFirmware4377.Hashstore 1
@filesystems.hfs.kext 522.0.9
@BootCache 40
@!AFSCompression.!AFSCompressionTypeDataless 1.0.0d1
@!AFSCompression.!AFSCompressionTypeZlib 1.0.0
>!AVirtIO 1.0
>!ABCMWLANBusInterfacePCIe 1
@filesystems.apfs 1412.81.1
@private.KextAudit 1.0
>!AAHCIPort 341.0.2
>!AACPIButtons 6.1
>!ASMBIOS 2.1
>!AACPIEC 6.1
>!AAPIC 1.7
$!AImage4 1
@nke.applicationfirewall 303
$TMSafetyNet 8
@!ASystemPolicy 2.0.0
|EndpointSecurity 1
|IOAHCISerialATAPI 268
|IOAHCIBlock!S 316.80.1
>!AThunderboltPCIUpAdapter 2.5.4
>!AThunderboltDPOutAdapter 6.2.5
>usb.!UHub 1.2
>!AHIDKeyboard 209
>!AMultitouchDriver 3430.1
>!AInputDeviceSupport 3430.1
>!AHS!BDriver 3430.1
>IO!BHIDDriver 7.0.3f5
>!ASMBus!C 1.0.18d1
@kext.AMDRadeonX4700HWLibs 1.0
|IOAccelerator!F2 438.3.1
@kext.AMDRadeonX4000HWServices 3.0.5
>!AGraphicsControl 4.7.2
|IOAVB!F 800.17
>!ASSE 1.0
|IO!BHost!CUARTTransport 7.0.3f5
|IO!BHost!CTransport 7.0.3f5
@!AGPUWrangler 4.7.2
>usb.cdc.acm 5.0.0
>usb.serial 6.0.0
|IONDRVSupport 569.4
>!A16X50Serial 3.2
@kext.AMDSupport 3.0.5
@!AGraphicsDeviceControl 4.7.2
|IOGraphics!F 569.4
>X86PlatformPlugin 1.0.0
>IOPlatformPlugin!F 6.0.0d8
|IOSlowAdaptiveClocking!F 1.0.0
@plugin.IOgPTPPlugin 810.1
|IOEthernetAVB!C 1.1.0
@kext.triggers 1.0
>usb.cdc.ncm 5.0.0
>usb.cdc 5.0.0
>usb.networking 5.0.0
>usb.!UHostCompositeDevice 1.2
|IOSurface 269.6
@filesystems.hfs.encodings.kext 1
|IOAudio!F 300.2
@vecLib.kext 1.2.0
>!ABCMWLANCore 1.0.0
>IOImageLoader 1.0.0
|IOSerial!F 11
|IO80211!FV2 1200.12.2b1
>corecapture 1.0.4
|IOSkywalk!F 1
>!AEthernetAquantiaAqtion 1.0.64
>mDNSOffloadUserClient 1.0.1b8
>!AThunderboltPCIDownAdapter 2.5.4
>!AThunderboltDPInAdapter 6.2.5
>!AThunderboltDPAdapter!F 6.2.5
>!AHPM 3.4.4
>!A!ILpssI2C!C 3.0.60
>!A!ILpssDmac 3.0.60
>!AThunderboltNHI 5.8.6
|IOThunderbolt!F 7.6.0
|IOUSB!F 900.4.2
>usb.!UVHCIBCE 1.2
>usb.!UVHCI 1.2
>usb.!UVHCICommonBCE 1.0
>usb.!UVHCICommon 1.0
>!AEffaceableNOR 1.0
|IOBufferCopy!C 1.1.0
|IOBufferCopyEngine!F 1
|IONVMe!F 2.1.0
>!A!ILpssI2C 3.0.60
|IOAHCI!F 290.0.1
>usb.!UXHCIPCI 1.2
>usb.!UXHCI 1.2
>!AEFINVRAM 2.1
>!AEFIRuntime 2.1
>!ASMCRTC 1.0
|IOSMBus!F 1.1
|IOHID!F 2.0.0
$quarantine 4
$sandbox 300.0
@Kext.!AMatch 1.0.0d1
>!AKeyStore 2
>!UTDM 489.80.2
|IOSCSIBlockCommandsDevice 422.0.2
>!ACredentialManager 1.0
>KernelRelayHost 1
>!ASEPManager 1.0.1
>IOSlaveProcessor 1
>!AFDEKeyStore 28.30
>!AEffaceable!S 1.0
>!AMobileFileIntegrity 1.0.5
@kext.CoreTrust 1
|CoreAnalytics!F 1
|IOTimeSync!F 810.1
|IONetworking!F 3.4
>DiskImages 493.0.0
|IO!B!F 7.0.3f5
|IO!BPacketLogger 7.0.3f5
|IOUSBMass!SDriver 157.40.7
|IOSCSIArchitectureModel!F 422.0.2
|IO!S!F 2.1
|IOUSBHost!F 1.2
>usb.!UCommon 1.0
>!UHostMergeProperties 1.2
>!ABusPower!C 1.0
|IOReport!F 47
>!AACPIPlatform 6.1
>!ASMC 3.1.9
>watchdog 1
|IOPCI!F 2.9
|IOACPI!F 1.4
@kec.pthread 1
@kec.corecrypto 1.0
@kec.Libm 1

 

[MikeabMac]

I have a 12c /98GB Ram/ Pro Vega II 7,1 which I got last feb 7 , 2020 ....had 1 sudden shutdown and failed to wakeup on sleep.... checked console which showed kernel panics cause by Norton Security (latest version) ... removed Norton Security and smooth sailing ever since. I have installed a 2TB Samsung Evo Plus on an ASUS hyper M.2 PCIE adapter and no problems..

 

[th0masp]

Why are these systems turning off though? Is there a default setting to shut down instead of restart in the vent of a kernel panic?

A computer that just shuts off randomly would freak me out.

 

[MikeabMac]

Why are these systems turning off though? Is there a default setting to shut down instead of restart in the vent of a kernel panic?

A computer that just shuts off randomly would freak me out.

Kernel Panics causes the system to become unresponsive and if the OS fails to recover it automatically restarts the computer/shutdown...I don't know if this is the default Catalina behavior but I haven't encountered this on my very old MacBook Pro running OS X Lion ( I know its ancient)

 

[th0masp]

Never seen this behaviour either in my time on the Mac, mine restart when a panic occurs.

Anyway I wonder how many of those having issues with their MPs are running potentially unsupported third party cards/drivers? Have to say that it is disappointing to hear about these problems. I'd demand absolute reliability from a machine of this kind. Random bluescreens are so turn-of-the-century.

 

[astrorider]

Why are these systems turning off though? Is there a default setting to shut down instead of restart in the vent of a kernel panic?

A computer that just shuts off randomly would freak me out.

I believe they do restart but if Filevault is enabled and no password is entered (and auto-login is disabled), after some amount of time it shuts down.

 

[daveedjackson]

Never seen this behaviour either in my time on the Mac, mine restart when a panic occurs.

Anyway I wonder how many of those having issues with their MPs are running potentially unsupported third party cards/drivers? Have to say that it is disappointing to hear about these problems. I'd demand absolute reliability from a machine of this kind. Random bluescreens are so turn-of-the-century.

nope, nothing additional here at all. Was literally just doing a drive backup onto an external drive.

 

[bxs]

nope, nothing additional here at all. Was literally just doing a drive backup onto an external drive.

Bad memory maybe? Have you added 3rd party RAM ?

 

[Theophilos]

The kernel panic while sleeping issue is a well-documented problem with Catalina. If you look at the thread discussing the High Point 7101A, you’ll see that I and several others are suffering from this issue, which may or may not be related to that card. There are plenty of people who had the same issue without the 7101A.

 

[daveedjackson]

Bad memory maybe? Have you added 3rd party RAM ?

A day after it arrived mid January: first kernel today... over a month later. And it’s been in constant use.

 

[Romanesco]

I have also observed that on every Mac I ran/ run Google Drive File Stream for longer period of times, the Mac freezes. I disabled autostart on mine and only use it sporadically when needed -- no issues since.

 

[handheldgames]

Considering the cost the system should be flawless. You paid for a Ferrari and got a Fiat. Aka. Fix it Again Tony!

 

[Bernardl]

I have been experiencing various issue with my high spec Mac Pro 2019 but 2 are particularly annoying:
- these random crashes related to sleep
- frequent disconnection of my TB3 Promise Raid array

Like most here, it's a 100% Apple genuine machine (16 cores, 384 GB RAM, Duo GPU) no peripherals added inside.

Very disappointed with this behavior. I would already be if the machine had cost 1/3 of what I paid for this machine.

The one thing I was expecting above all from the Pro was perfect stability. Otherwise I go Mackintosh and pay 3 times less.

Cheers,
Bernard

 

[jjhny]

What OS version are you running on the Mac Pro?

Also I did this (I'm still on 10.15.3) -> boot into recovery mode (cmd-r at startup) - open the 'startup security utility' then enable 'no security' and 'allow booting from external media' That works from me with my external OWC enclosure.

I also disabled Sip and Gatekeeper.

Apple's 'security' is a mess, and really isn't helping. I just use little snitch and don't allow safari to open downloads. I don't need their toddler version of security. I guess if I only ran Apple's toddler apps it would be okay. Ha!

 

[Bernardl]

I am on 10.15.5, but I have not noticed any difference.

Thanks for the tip, I will try that.

 

[Bakafish]

They do not restart, and it is super frustrating as I often remote access the machine first thing in the morning and having to go down several flights of stairs to physically power it up defeats the whole purpose. It isn't 'bad ram' or 3rd party hw/drivers, it is the OS. Disabling sleep has completely resolved the issue. I am confident that Apple will eventually fix the issue in the OS, I gave my theories as to why we won't see a fix for a while.

The Ferrari analogy is a poor one IMHO, and the slurs against Fiat are pretty childish. There is a basically painless workaround (if you can't afford some extra electricity, you can't afford this machine.) Ferrari's are not the pinnacle of reliability or flawless delivery. They are also not typically daily drivers (many of us depend on our Pro's from when we wake up to when we go to bed, at least I do.) These are expensive machines designed for craftsman to get things done that would be much harder to do with a more common tool. If you are unable to recognize the value, it isn't for you, but it doesn't mean that your opinion is universal.

We are early adopters, and should expect some teething problems with the most complex and sophisticated Mac in many years. It's hardly the first high end Mac to have issues. I've hit many of them personally, squeaking PSU's on my G5, mold lines in my Cube, flaking paint and hinge issues on my (beloved) TiBook, many others that I've long forgotten, these things always happen. They always get fixed eventually. Apple has never screwed me as a customer for the 37 years I've been buying their stuff. If you can't deal with the bleeding edge, don't buy the newest ****.

I think it is safe to assume there are engineering decisions that they made that will not make sense to us for some time, the T2 chip that is the likely culprit for these issues is one of those. It basically has its own low level OS running in parallel with MacOS, and I suspect we will be super happy about what that means when the switch to ARM happens.

Engineering is about tradeoffs. It sucks that the Pro was not in development sooner, and that it wasn't released as a flawless gem, but I'm much happier it is in my house and not still in Apple QA waiting for release. And maybe if you don't actually have one, consider if you are really qualified to comment.

 

[ZombiePhysicist]

I think part of the problem is also Catalina. It's just a rough OS. In part I suspect the killing of 32bit libraries have lots of gremlin problems for software that was updated but might have some vestigial stuff that received botchy patches to get things to work.

 

[Bakafish]

Catalina is not a monolithic thing that can be blamed. The fact that it is the only OS that works on the Pro makes such an argument impossible to refute. The issue in question is kernel panics when the CPU is in, or transitioning from a sleep state. The T2 chip takes over much of the system functionality at that time, when "power nap" is enabled it goes as far as actively servicing processes. So you have 3 competing subsystems doing a dance handing off control to each other. The low level BIOS, the T2 chip and the Intel CPU. This is all controlled at the Kernel level, hence the Kernel panics. Apple is not the only vendor to have issues managing CPU sleep states seamlessly, but the T2 highly complicates things.

It has nothing to do with old 32bit libraries or their removal, sorry.

My theory is that Apple is primarily doing development on a completely separate branch of their Kernel in order to add a fourth member to the dance, namely for our Mac Pro's an Axx ARM architecture CPU running on a PCI or MPX card. This will require changes that are not easily backported to the current architecture, and would potentially expose this transition. I presume they are not focusing on our bug because spending time on a low severity issue on a kernel branch that will soon be abandoned, when they are pushing all out to transition their architecture would be a misuse of limited resources.

They must know about the issue, if it was easy to fix (to be fair, kernel level stuff rarely is) it would have been fixed already. They are getting the crash reports. They use these machines themselves.

 

[ZombiePhysicist]

Catalina is not a monolithic thing that can be blamed. The fact that it is the only OS that works on the Pro makes such an argument impossible to refute. The issue in question is kernel panics when the CPU is in, or transitioning from a sleep state. The T2 chip takes over much of the system functionality at that time, when "power nap" is enabled it goes as far as actively servicing processes. So you have 3 competing subsystems doing a dance handing off control to each other. The low level BIOS, the T2 chip and the Intel CPU. This is all controlled at the Kernel level, hence the Kernel panics. Apple is not the only vendor to have issues managing CPU sleep states seamlessly, but the T2 highly complicates things.

It has nothing to do with old 32bit libraries or their removal, sorry.

My theory is that Apple is primarily doing development on a completely separate branch of their Kernel in order to add a fourth member to the dance, namely for our Mac Pro's an Axx ARM architecture CPU running on a PCI or MPX card. This will require changes that are not easily backported to the current architecture, and would potentially expose this transition. I presume they are not focusing on our bug because spending time on a low severity issue on a kernel branch that will soon be abandoned, when they are pushing all out to transition their architecture would be a misuse of limited resources.

They must know about the issue, if it was easy to fix (to be fair, kernel level stuff rarely is) it would have been fixed already. They are getting the crash reports. They use these machines themselves.

Um just because it's monolithic does not insulate it from blame, and contrary to your assertion, I blame it as do many others, your desire for "monolithicness' to immunize it from blame notwithstanding, as many other machines also have problems with it.

 

[UMHurricanes34]

I have also observed that on every Mac I ran/ run Google Drive File Stream for longer period of times, the Mac freezes. I disabled autostart on mine and only use it sporadically when needed -- no issues since.

Any recent developments with this issue for you? We have 7 of these towers in use at our production facility and have experienced similar issues with a recent addition (coming into the office with the Mac Pro unresponsive). The difference for me is that the LED light is still on on top of the machine, and it requires a hard restart to get it back up and operational.

Running the latest build of Big Sur as we don't want to migrate to Monterey for our production house.