M1 Kernel Panic using certain apps - finally able to replicate, asking for help to confirm...

[johnalan]

Hello,

For months I've struggled with Kernel Panics on reboot, even on the latest Big Sur, 11.2 / 11.2.1


That is, when I try to reboot my Mac, it takes 5/6 minutes and when it restarts I see a message saying a Kernel Panic occurred.
I posted about it here.

This issue has been ongoing for myself and others for nearly a year.


I've really struggled with it being intermittent but, now, on my M1 Mac I can consistently reproduce it using a number of ways, and it's weird.
All of these apps are installed via home-brew -

1) Using ruby and then gem installed via home-brew, i.e. executing /opt/homebrew/opt/ruby/bin/ruby, then /opt/homebrew/opt/ruby/bin/gem, then rebooting - see KP.

2) Executing "thef*ck" (https://github.com/nvbn/thef*ck) - see KP.

3) Executing "pip3 list" installed via homebrew, i.e. "/opt/homebrew/bin/pip3 list" - see KP.



I would greatly appreciate if someone could confirm this for me, I've confirmed it on the two M1 Macs I own.

It's odd but these apps are interfering with some system process or XNU process and causing watchdogd to timeout on reboot.

Please note, system ruby and pip3 are unaffected, it must be homebrew installed.


Pls let me know if you can replicate?

Thanks,
John

 

[Gnattu]

I can confirm the brew installed pip3 will trigger the panic.
What's more I found that the panic seems gone when SIP is disabled.

Super weird, but reproducible.

 

[johnalan]

Thank you!

 

[johnalan]

I can confirm the brew installed pip3 will trigger the panic.
What's more I found that the panic seems gone when SIP is disabled.

Super weird, but reproducible.

So this must be a kernel bug right?

 

[Gnattu]

So this must be a kernel bug right?

Maybe not kernel, but a watchdogd bug.

 

[mi7chy]

Using Homebrew 3.0.1 AS version and both #2 and #3 individually or back to back don't cause KP upon restart.

For #1, which gem did you install, brew-gem or gitlab-gem? 'ruby -v', 'ruby hello.rb' and brew-gem 'gem -v' from /opt/homebrew/opt/ruby don't KP on restart either.

 

[johnalan]

Using Homebrew 3.0.1 AS version and both #2 and #3 individually or back to back don't cause KP upon restart.

For #1, which gem did you install, brew-gem or gitlab-gem? 'ruby -v', 'ruby hello.rb' and brew-gem 'gem -v' from /opt/homebrew/opt/ruby don't KP on restart either.

Interesting. Are you on a mini or MacBook? This issue I can replicate on 3x MacBook but not on a friends mini.

 

[mi7chy]

Interesting. Are you on a mini or MacBook? This issue I can replicate on 3x MacBook but not on a friends mini.

Macbook Air M1 16GB. Have you tried the new Apple Silicon version of Homebrew?

 

[johnalan]

Macbook Air M1 16GB. Have you tried the new Apple Silicon version of Homebrew?

Wow ok,

Can you check your `brew config`?

HOMEBREW_VERSION: 3.0.0
ORIGIN: https://github.com/Homebrew/brew
HEAD: ebc7cc36a0e029d58fac8cebd1b5ff583b3fd95b
Last commit: 6 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 8d6440b2c0904aebb68fb79a4f6af2cab6270ccd
Core tap last commit: 22 hours ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 12.0 build 1200
Git: 2.30.1 => /opt/homebrew/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 11.2.1-arm64
CLT: N/A
Xcode: 12.4
Rosetta 2: false

My setup is:

ZSH from brew, using iTerm2, using ohMyZSH, and powerlevel10k...

But I can make a new user, boot into safe mode, and STILL have this KP happen.

Someone in this thread also replicated, it's bizarre, I hate that it happens tho, makes my system feel unstable.

Looking forward to seeing if we can see any obvious stuff here, what's your setup?

 

[mi7chy]

Can you check your `brew config`?

Everything default except for addition of homebrew and your mentioned formulae.

HOMEBREW_VERSION: 3.0.1
ORIGIN: https://github.com/Homebrew/brew
HEAD: c951be8d3c7b339c7e759b7c40aec859e09a70a5
Last commit: 19 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: d8f2ab2e9620208dfee599b01b83a7f4be08110f
Core tap last commit: 3 hours ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 12.0 build 1200
Git: 2.24.3 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 11.2.1-arm64
CLT: 12.4.0.0.1.1610135815
Xcode: N/A
Rosetta 2: false

 

[johnalan]

Maybe not kernel, but a watchdogd bug.

hey Gnattu, as you’re able to replicate can I ask your setup?

iterm? Power line? Using anything weird or wonderful?

 

[Gnattu]

hey Gnattu, as you’re able to replicate can I ask your setup?

iterm? Power line? Using anything weird or wonderful?

MacBook Pro, using system terminal. I don't think I have anything special for pip.

 

[johnalan]

MacBook Pro, using system terminal. I don't think I have anything special for pip.

Wow OK, I'm really thinking this must be something that's macOS based rather than a collection of small things that add up to cause problems.

I wonder why @mi7chy can't replicate.

 

[haralds]

I can confirm the brew installed pip3 will trigger the panic.
What's more I found that the panic seems gone when SIP is disabled.

Super weird, but reproducible.

No trouble with pip3 installed with brew 3.0 on 11.3beta. My list is NOT empty.

HOMEBREW_VERSION: 3.0.1
ORIGIN: https://github.com/Homebrew/brew
HEAD: c951be8d3c7b339c7e759b7c40aec859e09a70a5
Last commit: 2 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 6daeed17754a826ea1e42f9841fb23bebb911439
Core tap last commit: 9 hours ago
Core tap branch: master

HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 12.0 build 1200
Git: 2.24.3 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 11.3-arm64
CLT: N/A
Xcode: 12.4
Rosetta 2: false

 

[johnalan]

No trouble with pip3 installed with brew 3.0 on 11.3beta. My list is NOT empty.

HOMEBREW_VERSION: 3.0.1
ORIGIN: https://github.com/Homebrew/brew
HEAD: c951be8d3c7b339c7e759b7c40aec859e09a70a5
Last commit: 2 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 6daeed17754a826ea1e42f9841fb23bebb911439
Core tap last commit: 9 hours ago
Core tap branch: master

HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 12.0 build 1200
Git: 2.24.3 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 11.3-arm64
CLT: N/A
Xcode: 12.4
Rosetta 2: false

Are you saying 11.3 fixes this issue?
You on a MBP?

 

[dmccloud]

My config is slightly different than everyone else's. At the same time, I haven't had a single kernel panic on this Mac:

HOMEBREW_VERSION: 3.0.1
ORIGIN: https://github.com/Homebrew/brew
HEAD: c951be8d3c7b339c7e759b7c40aec859e09a70a5
Last commit: 3 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: bd3bc1c7feeec5677c9c4e9b6874a5780bcc5ff8
Core tap last commit: 5 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_REPOSITORY: /usr/local/Homebrew
HOMEBREW_CELLAR: /usr/local/Cellar
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: :0
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 12.0 build 1200
Git: 2.24.3 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 11.3-arm64
CLT: 12.4.0.0.1.1610135815
Xcode: 12.5 => /Applications/Xcode-beta.app/Contents/Developer
Rosetta 2: false

 

[johnalan]

Can anyone else replicate? I'm trying to see the delta between problematic installs and those without issue...

 

[johnalan]

SOLVED.

My replication steps only work when my terminal emulator (iTerm/Terminal) was in added in "Developer Tools" section of Security & Privacy settings in macOS.

Removed them and I could not longer replicate.

This looks to be a bug in codesign/watchdogd/adhoc signatures. Where you remove the terminal from dev tools, you can see the signature check is performed and all is well.

If you have terminal in the "developer tools" category, certain tools (like the ones I describe in my post, ruby/pip), cause the OS to panic on reboot, as something ends up hanging.

Described further here: https://sigpipe.macromates.com/2020/macos-catalina-slow-by-design/

 

[jdb8167]

SOLVED.

My replication steps only work when my terminal emulator (iTerm/Terminal) was in added in "Developer Tools" section of Security & Privacy settings in macOS.

Removed them and I could not longer replicate.

This looks to be a bug in codesign/watchdogd/adhoc signatures. Where you remove the terminal from dev tools, you can see the signature check is performed and all is well.

If you have terminal in the "developer tools" category, certain tools (like the ones I describe in my post, ruby/pip), cause the OS to panic on reboot, as something ends up hanging.

Described further here: https://sigpipe.macromates.com/2020/macos-catalina-slow-by-design/

The problem with removing the terminal from dev tools is now every command line tool needs to be signed. That can be a pain. I haven’t had any kernel panics that I can attribute to being in the dev tools section of privacy. I hope Apple can track down your panic issue and fix it.

 

[johnalan]

The problem with removing the terminal from dev tools is now every command line tool needs to be signed. That can be a pain. I haven’t had any kernel panics that I can attribute to being in the dev tools section of privacy. I hope Apple can track down your panic issue and fix it.

So actually it doesn’t have to be signed.

it’s running an OSCP check, but signature isn’t required. It’s more a Gatekeeper check for know malware.
Execution slow down is only on first run then cached.

So not a big deal.

 

[johnalan]

I can confirm the brew installed pip3 will trigger the panic.
What's more I found that the panic seems gone when SIP is disabled.

Super weird, but reproducible.

I bet you have your terminal added as a "Developer Tool" under System Preferences -> Security & Privacy

 

[Gnattu]

I bet you have your terminal added as a "Developer Tool" under System Preferences -> Security & Privacy

Exactly, when I removed that the crash does not occur. Thank you very much.

 

[mattgif]

I'll be the first unfortunate to volunteer that this fix does NOT work for me. I'm also an iTerm2 user, but it was never listed in the Developer Tools list. I get watchdog kernel panics almost every time the laptop goes to sleep for more than 4 hours.