Several Kernel Panic Reboots over the last several months

[honeycombz]

Hello, I've had a couple kernel panic reboots late at night when I'm not around. Wondering what your take is on the following log?

Anonymous UUID:       FC984446-6540-B8CF-246C-52E6A090E649

Fri Jan 29 21:24:33 2016
Panic(CPU 2): Unresponsive processor (this CPU did not acknowledge interrupts) TLB state:0x0
RAX: 0x00000000000000ff, RBX: 0x000020a88d6cb266, RCX: 0x0000000000000001, RDX: 0x00000000000020a8
RSP: 0xffffff81cb3f3758, RBP: 0xffffff81cb3f37b0, RSI: 0x0000000000000002, RDI: 0xffffff805183a10a
R8:  0xffffff81c33cd078, R9:  0x0000000000000000, R10: 0xffffff81cb3f35c0, R11: 0xffffffffffffffff
R12: 0xffffff805183a10a, R13: 0xffffff805183a108, R14: 0xffffff805183a110, R15: 0x0000000000000001
RFL: 0x0000000000000086, RIP: 0xffffff80194d1571, CS:  0x0000000000000008, SS:  0x0000000000000010
Backtrace (CPU 2), Frame : Return Address
0xffffff81c343ef50 : 0xffffff80194e2cc1 
0xffffff81c343ef80 : 0xffffff80194dc05f 
0xffffff81c343efd0 : 0xffffff80194f4649 
0xffffff81cb3f37b0 : 0xffffff7f9bbc261c 
0xffffff81cb3f3990 : 0xffffff7f9bbcbb63 
0xffffff81cb3f39e0 : 0xffffff80195fd1c7 
0xffffff81cb3f3a60 : 0xffffff80195f6265 
0xffffff81cb3f3c00 : 0xffffff80195f248a 
0xffffff81cb3f3d80 : 0xffffff80195ea70c 
0xffffff81cb3f3f50 : 0xffffff8019841a23 
0xffffff81cb3f3fb0 : 0xffffff80194f4c86 
      Kernel Extensions in backtrace:
         com.apple.filesystems.afpfs(11.1)[DCC45CDD-E950-364F-8221-1A914106BC59]@0xffffff7f9bba6000->0xffffff7f9bbfafff
            dependency: com.apple.security.SecureRemotePassword(1.0)[BC0BAB92-C8C9-3959-AA42-272AF578DFD9]@0xffffff7f9bb95000

BSD process name corresponding to current thread: fseventsd
Boot args: kext-dev-mode=1

Mac OS version:
13F1603

Kernel version:
Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64
Kernel UUID: 8B1A8FD1-2344-36C0-A7F5-D9D485A995FA
Kernel slide:     0x0000000019200000
Kernel text base: 0xffffff8019400000
System model name: MacPro3,1 (Mac-F42C88C8)

System uptime in nanoseconds: 35909037171412
last loaded kext at 1379940106989: com.apple.filesystems.afpfs  11.1 (addr 0xffffff7f9bba6000, size 348160)
last unloaded kext at 854400981908: com.apple.driver.AppleUSBCDC    4.2.1b5 (addr 0xffffff7f9bafc000, size 16384)
loaded kexts:
com.vara.driver.VaraAudio   1.0.3
com.Cycling74.driver.Soundflower    1.6.7
com.Logitech.Unifying.HID Driver    1.3.1
com.Logitech.Control Center.HID Driver  3.9.1
com.apple.filesystems.afpfs 11.1
com.apple.nke.asp-tcp   8.0.1
com.apple.filesystems.smbfs 2.0.3
com.apple.driver.Oxford_Semi    3.4.1
com.apple.filesystems.autofs    3.0
com.apple.iokit.IOBluetoothSerialManager    4.2.7f5
com.apple.driver.AppleHDA   2.6.3f7
com.apple.driver.AppleUpstreamUserClient    3.5.13
com.apple.driver.AppleMCCSControl   1.2.5
com.apple.kext.AMDFramebuffer   1.2.4
com.apple.AMDRadeonX3000    1.2.4
com.apple.kext.AMD5000Controller    1.2.4
com.apple.driver.ACPI_SMC_PlatformPlugin    1.0.0
com.apple.driver.AppleUSBDisplays   360.8.14
com.apple.driver.AudioAUUC  1.60
com.apple.iokit.CSRBluetoothHostControllerUSBTransport  4.2.7f5
com.apple.driver.AppleMCEDriver 1.1.9
com.apple.iokit.IOUserEthernet  1.0.0d1
com.apple.Dont_Steal_Mac_OS_X   7.0.0
com.apple.driver.AppleHWAccess  1
com.apple.driver.AppleLPC   1.7.0
com.apple.iokit.SCSITaskUserClient  3.6.7
com.apple.driver.XsanFilter 404
com.apple.driver.AppleUSBHub    683.4.0
com.apple.driver.AppleIntel8254XEthernet    3.1.4b1
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.AppleFSCompression.AppleFSCompressionTypeLZVN 1.0.0d1
com.apple.iokit.IOAHCIBlockStorage  2.6.0
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1
com.apple.BootCache 35
com.apple.driver.AppleFWOHCI    5.0.2
com.apple.driver.AirPortBrcm43224   700.36.24
com.apple.driver.AppleUSBEHCI   660.4.0
com.apple.driver.AppleIntelPIIXATA  2.5.1
com.apple.driver.AppleAHCIPort  3.0.5
com.apple.driver.AppleUSBUHCI   656.4.1
com.apple.driver.AppleACPIButtons   2.0
com.apple.driver.AppleHPET  1.8
com.apple.driver.AppleRTC   2.0
com.apple.driver.AppleSMBIOS    2.1
com.apple.driver.AppleACPIEC    2.0
com.apple.driver.AppleAPIC  1.7
com.apple.driver.AppleIntelCPUPowerManagementClient 217.92.1
com.apple.nke.applicationfirewall   153
com.apple.security.quarantine   3
com.apple.driver.AppleIntelCPUPowerManagement   217.92.1
com.apple.security.SecureRemotePassword 1.0
com.apple.iokit.IOSCSIReducedBlockCommandsDevice    3.6.7
com.apple.iokit.IOFireWireSerialBusProtocolTransport    2.5.1
com.apple.iokit.IOFireWireSBP2  4.2.6
com.apple.kext.triggers 1.0
com.apple.iokit.IOSerialFamily  10.0.7
com.apple.driver.DspFuncLib 2.6.3f7
com.apple.vecLib.kext   1.0.0
com.apple.driver.AppleSMBusController   1.0.12d1
com.apple.iokit.IONDRVSupport   2.4.1
com.apple.iokit.IOAcceleratorFamily 98.26
com.apple.kext.AMDSupport   1.2.4
com.apple.AppleGraphicsDeviceControl    3.6.22
com.apple.driver.AppleUSBAudio  2.9.5f8
com.apple.driver.IOPlatformPluginLegacy 1.0.0
com.apple.iokit.IOBluetoothHostControllerUSBTransport   4.2.7f5
com.apple.iokit.IOFireWireIP    2.2.6
com.apple.driver.AppleHDAController 2.6.3f7
com.apple.iokit.IOHDAFamily 2.6.3f7
com.apple.iokit.IOAudioFamily   1.9.7fc2
com.apple.kext.OSvKernDSPLib    1.14
com.apple.iokit.IOSurface   91.3
com.apple.iokit.IOGraphicsFamily    2.4.1
com.apple.driver.AppleSMC   3.1.8
com.apple.driver.IOPlatformPluginFamily 5.7.1d6
com.apple.driver.AppleUSBHIDKeyboard    170.15
com.apple.driver.AppleHIDKeyboard   170.15
com.apple.driver.AppleUSBMergeNub   650.4.0
com.apple.driver.IOBluetoothHIDDriver   4.2.7f5
com.apple.iokit.IOBluetoothFamily   4.2.7f5
com.apple.iokit.IOUSBHIDDriver  660.4.0
com.apple.driver.AppleUSBComposite  656.4.1
com.apple.iokit.IOSCSIMultimediaCommandsDevice  3.6.7
com.apple.iokit.IOBDStorageFamily   1.7
com.apple.iokit.IODVDStorageFamily  1.7.1
com.apple.iokit.IOCDStorageFamily   1.7.1
com.apple.iokit.IOATAPIProtocolTransport    3.5.0
com.apple.iokit.IOSCSIArchitectureModelFamily   3.6.7
com.apple.iokit.IOUSBUserClient 660.4.2
com.apple.iokit.IOFireWireFamily    4.5.5
com.apple.iokit.IO80211Family   640.36
com.apple.iokit.IONetworkingFamily  3.2
com.apple.iokit.IOATAFamily 2.5.2
com.apple.iokit.IOAHCIFamily    2.6.5
com.apple.iokit.IOUSBFamily 686.4.1
com.apple.driver.AppleEFINVRAM  2.0
com.apple.iokit.IOHIDFamily 2.0.0
com.apple.driver.AppleEFIRuntime    2.0
com.apple.iokit.IOSMBusFamily   1.1
com.apple.security.sandbox  278.11.2
com.apple.kext.AppleMatch   1.0.0d1
com.apple.security.TMSafetyNet  7
com.apple.driver.AppleKeyStore  2
com.apple.driver.DiskImages 373
com.apple.iokit.IOStorageFamily 1.9
com.apple.iokit.IOReportFamily  23
com.apple.driver.AppleFDEKeyStore   28.30
com.apple.driver.AppleACPIPlatform  2.0
com.apple.iokit.IOPCIFamily 2.9
com.apple.iokit.IOACPIFamily    1.4
com.apple.kec.pthread   1
com.apple.kec.corecrypto    1.0
panic(cpu 0 caller 0xffffff80194c0bab): "TLB invalidation IPI timeout: " "CPU(s) failed to respond to interrupts, unresponsive CPU bitmap: 0x4, NMIPI acks: orig: 0x0, now: 0x1"@/SourceCache/xnu/xnu-2422.115.14/osfmk/x86_64/pmap.c:2615
Backtrace (CPU 0), Frame : Return Address
0xffffff81cb0f3bb0 : 0xffffff8019423139 
0xffffff81cb0f3c30 : 0xffffff80194c0bab 
0xffffff81cb0f3cc0 : 0xffffff80194c5b63 
0xffffff81cb0f3da0 : 0xffffff80194c6657 
0xffffff81cb0f3e00 : 0xffffff801947e77a 
0xffffff81cb0f3f10 : 0xffffff8019482a4f 
0xffffff81cb0f3f40 : 0xffffff801941d307 
0xffffff81cb0f3f70 : 0xffffff80194ca13d 
0xffffff81cb0f3fb0 : 0xffffff80194f4ca6 

BSD process name corresponding to current thread: fseventsd
Model: MacPro3,1, BootROM MP31.006C.B05, 8 processors, Quad-Core Intel Xeon, 2.8 GHz, 14 GB, SMC 1.25f4
Graphics: ATI Radeon HD 5770, ATI Radeon HD 5770, PCIe, 1024 MB
Memory Module: DIMM Riser B/DIMM 1, 2 GB, DDR2 FB-DIMM, 800 MHz, 0x02BA, 0x202020202020202020202020202020202020
Memory Module: DIMM Riser B/DIMM 2, 2 GB, DDR2 FB-DIMM, 800 MHz, 0x02BA, 0x202020202020202020202020202020202020
Memory Module: DIMM Riser A/DIMM 1, 2 GB, DDR2 FB-DIMM, 800 MHz, 0x0000, 0x000000463732353642363145363830304600
Memory Module: DIMM Riser A/DIMM 2, 2 GB, DDR2 FB-DIMM, 800 MHz, 0x0000, 0x000000463732353642363145363830304600
Memory Module: DIMM Riser B/DIMM 3, 2 GB, DDR2 FB-DIMM, 800 MHz, 0x02BA, 0x202020202020202020202020202020202020
Memory Module: DIMM Riser B/DIMM 4, 2 GB, DDR2 FB-DIMM, 800 MHz, 0x02BA, 0x202020202020202020202020202020202020
Memory Module: DIMM Riser A/DIMM 3, 1 GB, DDR2 FB-DIMM, 800 MHz, 0x0000, 0x000000463732323842363145363830304600
Memory Module: DIMM Riser A/DIMM 4, 1 GB, DDR2 FB-DIMM, 800 MHz, 0x0000, 0x000000463732323842363145363830304600
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x88), Broadcom BCM43xx 1.0 (5.10.131.36.16)
Bluetooth: Version 5.2.7f4 15858, 3 services, 22 devices, 1 incoming serial ports
Network Service: Ethernet 1, Ethernet, en0
PCI Card: ATI Radeon HD 5770, Display Controller, Slot-1
PCI Card: pci1b21,612, AHCI Controller, Slot-2
PCI Card: ATI Radeon HD 5770, ATY,HoolockParent, Slot-1
Serial ATA Device: WDC WD1002FAEX-00Y9A0, 1 TB
Serial ATA Device: WDC WD5000AAKS-00V6A0, 500.11 GB
Serial ATA Device: WDC WD1002FAEX-00Z3A0, 1 TB
Serial ATA Device: WDC WD1002FAEX-00Z3A0, 1 TB
Serial ATA Device: Crucial_CT500MX200SSD1, 500.11 GB
Parallel ATA Device: PIONEER DVD-RW  DVR-112D
USB Device: Hub
USB Device: Keyboard Hub
USB Device: USB Receiver
USB Device: Apple Keyboard
USB Device: MegeCam
USB Device: Apple Cinema HD Display
USB Device: Bluetooth USB Host Controller
FireWire Device: built-in_hub, Up to 800 Mb/sec
FireWire Device: unknown_device, Unknown
FireWire Device: unknown_device, Unknown
Thunderbolt Bus:

 

[Bytehoven]

Do you have your Hardware Test DVD?

You can find many Apple OEM disc images here: https://github.com/upekkha/AppleHardwareTest

You just need to confirm which one will boot properly.

 

[honeycombz]

No, I don't. I'll download the appropriate disc image and try to run it from a USB disk today.

 

[honeycombz]

The Apple Hardware Test didn't show anything.

 

[ScottishCaptain]

Hello, I've had a couple kernel panic reboots late at night when I'm not around. Wondering what your take is on the following log?

1) One of the CPUs has failed (or is failing)
2) The logic board is failing
3) Your RAM is failing
4) Some kind of software is gumming up the CPU

Something is causing the second CPU to stop responding to... well, everything.

I would recommend that you try running the machine with a clean installation of OS X (and nothing else), and see if it still panics. If it does, I'd try swapping out the RAM. If you don't have any spare RAM kicking around, I suppose you could remove half of it, see if the machine still panics, and if it does swap that RAM with the other half and see if the problem persists.

Beyond that, unfortunately I'd say that either the logic board or CPU itself is bad. Personally, I'm leaning towards either some kind of bizarre software fault that is causing the CPU to lockup, or the logic board/CPU is bad. Usually memory related issues manifest as some other kind of problem (machine check exception, ECC errors, or other random crashes and/or kernel panics), so I don't think it's going to be your RAM. The kernel panic is pretty specific though (unless your other panics are different), so there's not many things it could be.

-SC