Secure Boot is a red-herring, people worries a lot about it and forget that the real problem that is NVRAM fragmentation and garbage collection not working leading to corruption/bricks.
I've sent a PM with instructions.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
MP 1,1-5,1 MacPro5,1: BootROM thread | 144.0.0.0.0
- Thread starter tsialex
- WikiPost WikiPost
- Start date
- Sort by reaction score
- Status
I've sent a PM with instructions.
Got an interesting BootROM dump today from @permanentmacdabbler, where garbage collection totally and completely failed.
This is the nvram -xp log - redacted any personal data - from the System Information report:
Pretty clean, no? Nothing wrong, just the expected variables. Now look at the clusterfu_k inside the first VSS store of the NVRAM volume, redacted any personal/private data:
It's a 20H Mac Pro, so an early-2009 dual CPU Mac Pro and should have one MemoryConfig for each memory slot (dual CPU, so, 8 slots), but with the failed garbage collection, now the NVRAM volume has 26 MemoryConfig variables and the 1st store now has only 8577 bytes available, not enough for the garbage collection do it's job.
If the owner changed a DIMM for example, the main VSS store would almost sure overrun and brick the Mac Pro.
Btw, this one is an excellent example of the discrepancy between what you see when accessing the NVRAM data and what is really stored inside the SPI flash memory. Looking at just the nvram -xp you'd never imagine that NVRAM volume is on the brink of corruption.
This is the nvram -xp log - redacted any personal data - from the System Information report:
Code:
NVRAM contents:
Source: /usr/sbin/nvram -xp
Size: 2 KB (1.654 bytes)
Last Modified: 11/02/21 09:39
Recent Contents: <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>SystemAudioVolume</key>
<data>
QA==
</data>
<key>SystemAudioVolumeDB</key>
<data>
AA==
</data>
<key>bluetoothActiveControllerInfo</key>
<data>
FYKsBQAAAAARWgAmSps7dA==
</data>
<key>bluetoothInternalControllerInfo</key>
<data>
FYKsBQAAEVoAJkqbO3Q=
</data>
<key>boot-args</key>
<string>-no_compat_check</string>
<key>csr-active-config</key>
<data>
EAAAAA==
</data>
<key>efi-boot-device</key>
<data>
PGFycmF5PjxkaWN0PjxrZXk+SU9NYXRjaDwva2V5PjxkaWN0PjxrZXk+SU9Qcm92aWRl
ckNsYXNzPC9rZXk+PHN0cmluZz5JT01lZGlhPC9zdHJpbmc+PGtleT5JT1Byb3BlcnR5
TWF0Y2g8L2tleT48ZGljdD48a2V5PlVVSUQ8L2tleT48c3RyaW5nPjgwNUIwOEY0LTlB
QTAtNDU3Qy1CQjRCLUY5QkJBM0VBMUIzNjwvc3RyaW5nPjwvZGljdD48L2RpY3Q+PGtl
eT5CTExhc3RCU0ROYW1lPC9rZXk+PHN0cmluZz5kaXNrNXMxPC9zdHJpbmc+PC9kaWN0
PjxkaWN0PjxrZXk+SU9FRklEZXZpY2VQYXRoVHlwZTwva2V5PjxzdHJpbmc+TWVkaWFG
aWxlUGF0aDwvc3RyaW5nPjxrZXk+UGF0aDwva2V5PjxzdHJpbmc+XEVGSVxCT09UXEJP
T1R4NjQuZWZpPC9zdHJpbmc+PC9kaWN0PjwvYXJyYXk+AA==
</data>
<key>efi-boot-device-data</key>
<data>
AgEMANBBAwoAAAAAAQEGAAIfAxIKAAMAAAAAAAQBKgABAAAAKAAAAAAAAAAAQAYAAAAA
APQIW4CgmnxFu0v5u6PqGzYCAgQEMABcAEUARgBJAFwAQgBPAE8AVABcAEIATwBPAFQA
eAA2ADQALgBlAGYAaQAAAH//BAA=
</data>
<key>fmm-computer-name</key>
<data>
xxxxbWFz4oCZcyBNYWMgxxxx
</data>
<key>prev-lang:kbd</key>
<data>
ZW4tR0I6Mg==
</data>
</dict>
</plist>Pretty clean, no? Nothing wrong, just the expected variables. Now look at the clusterfu_k inside the first VSS store of the NVRAM volume, redacted any personal/private data:
Code:
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
0 0x0 UEFI PI Firmware Volume, volume size: 524288, header size: 1, revision: 0, EFI Firmware File System, GUID: 7A9354D9-0468-444A-CE81-0BF617D890DF
24972 0x618C CRC32 polynomial table, little endian
35787 0x8BCB mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit
243907 0x3B8C3 BIOS version: MP51.88Z.F000.B00.1904121248
524288 0x80000 UEFI PI Firmware Volume, volume size: 524288, header size: 1, revision: 0, EFI Firmware File System, GUID: 7A9354D9-0468-444A-CE81-0BF617D890DF
549260 0x8618C CRC32 polynomial table, little endian
560075 0x88BCB mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit
768195 0xBB8C3 BIOS version: MP51.88Z.F000.B00.1904121248
1048576 0x100000 UEFI PI Firmware Volume, volume size: 16384, header size: 1, revision: 0, EFI Firmware File System, GUID: 7A9354D9-0468-444A-CE81-0BF617D890DF
1064960 0x104000 UEFI PI Firmware Volume, volume size: 49152, header size: 1, revision: 0, GUID: 153D2197-29BD-44DC-59AC-887F70E41A6B
1065216 0x104100 Intel x86 or x64 microcode, sig 0x000106a5, pf_mask 0x03, 2018-05-11, rev 0x001d, size 12288
1077504 0x107100 Intel x86 or x64 microcode, sig 0x000206c0, pf_mask 0x13, 2009-08-20, rev 0x-ffea, size 8192
1085696 0x109100 Intel x86 or x64 microcode, sig 0x000206c2, pf_mask 0x03, 2018-05-08, rev 0x001f, size 11264
1114112 0x110000 UEFI PI Firmware Volume, volume size: 16384, header size: 1, revision: 0, EFI Firmware File System, GUID: 7A9354D9-0468-444A-CE81-0BF617D890DF
1130496 0x114000 UEFI PI Firmware Volume, volume size: 49152, header size: 1, revision: 0, GUID: 153D2197-29BD-44DC-59AC-887F70E41A6B
1130752 0x114100 Intel x86 or x64 microcode, sig 0x000106a5, pf_mask 0x03, 2018-05-11, rev 0x001d, size 12288
1143040 0x117100 Intel x86 or x64 microcode, sig 0x000206c0, pf_mask 0x13, 2009-08-20, rev 0x-ffea, size 8192
1151232 0x119100 Intel x86 or x64 microcode, sig 0x000206c2, pf_mask 0x03, 2018-05-08, rev 0x001f, size 11264
1179648 0x120000 UEFI PI Firmware Volume, volume size: 196608, header size: 1, revision: 0, Variable Storage, GUID: FFF12B8D-7696-4C8B-85A9-2747075B4F50
1179688 0x120028 NVRAM start of the 1st VSS store
1179766 0x120076 NVRAM MemoryConfig type: (j)
1181777 0x120851 NVRAM bluetoothActiveControllerInfo
1183982 0x1210EE NVRAM MemoryConfig type: (i)
1186770 0x121BD2 NVRAM MemoryConfig type: (g)
1188818 0x1223D2 NVRAM MemoryConfig type: (h)
1190866 0x122BD2 NVRAM MemoryConfig type: (i)
1193586 0x123672 NVRAM MemoryConfig type: (g)
1195634 0x123E72 NVRAM MemoryConfig type: (h)
1197682 0x124672 NVRAM MemoryConfig type: (i)
1201089 0x1253C1 NVRAM MemoryConfig type: (g)
1203137 0x125BC1 NVRAM MemoryConfig type: (h)
1205185 0x1263C1 NVRAM MemoryConfig type: (i)
1208316 0x126FFC NVRAM MemoryConfig type: (g)
1210364 0x1277FC NVRAM MemoryConfig type: (h)
1212412 0x127FFC NVRAM MemoryConfig type: (i)
1215550 0x128C3E NVRAM MemoryConfig type: (g)
1217598 0x12943E NVRAM MemoryConfig type: (h)
1219646 0x129C3E NVRAM MemoryConfig type: (i)
1222857 0x12A8C9 NVRAM MemoryConfig type: (g)
1224905 0x12B0C9 NVRAM MemoryConfig type: (h)
1226953 0x12B8C9 NVRAM MemoryConfig type: (i)
1229942 0x12C476 NVRAM MemoryConfig type: (g)
1231990 0x12CC76 NVRAM MemoryConfig type: (h)
1234038 0x12D476 NVRAM MemoryConfig type: (i)
1245255 0x130047 NVRAM start of the 2nd VSS store
1245302 0x130076 NVRAM MemoryConfig type: (j)
1247313 0x130851 NVRAM bluetoothActiveControllerInfo
1249518 0x1310EE NVRAM MemoryConfig type: (i)
1252306 0x131BD2 NVRAM MemoryConfig type: (g)
1343518 0x14801E HardwareID Base_xx: 19
1343538 0x148032 bzip2 compressed data, block size = 100k
1345188 0x1486A4 HardwareID 11-digits SSN: CK9xxxxx20H
1345205 0x1486B5 HardwareID 3-digit HWC model: 20H
1376256 0x150000 UEFI PI Firmware Volume, volume size: 2686976, header size: 1, revision: 0, EFI Firmware File System, GUID: 7A9354D9-0468-444A-CE81-0BF617D890DF
1416827 0x159E7B BIOS version: MP51.88Z.F000.B00.1904121248
1614976 0x18A480 Apple NVMe EFI Module
4063232 0x3E0000 UEFI PI Firmware Volume, volume size: 65536, header size: 1, revision: 0, GUID: E3B980A9-5FE3-48E5-929B-2798385A9027
4128768 0x3F0000 UEFI PI Firmware Volume, volume size: 65536, header size: 0, revision: 0, Apple Boot Volume, GUID: 04ADEEAD-61FF-4D31-BAB6-64F8BF901F5A
4128867 0x3F0063 BootBlock version: AAPLEFI1.88Z.0005.I00.1010071430
4194000 0x3FFED0 HardwareID MLB/LBSN: J59xxxxxx1LTC, BuildDate: 09xxxx09xxxxIt's a 20H Mac Pro, so an early-2009 dual CPU Mac Pro and should have one MemoryConfig for each memory slot (dual CPU, so, 8 slots), but with the failed garbage collection, now the NVRAM volume has 26 MemoryConfig variables and the 1st store now has only 8577 bytes available, not enough for the garbage collection do it's job.
If the owner changed a DIMM for example, the main VSS store would almost sure overrun and brick the Mac Pro.
Btw, this one is an excellent example of the discrepancy between what you see when accessing the NVRAM data and what is really stored inside the SPI flash memory. Looking at just the nvram -xp you'd never imagine that NVRAM volume is on the brink of corruption.
Last edited:
Lol I thought something was wrong with it and luckily you got to it before it was too late. Perfect word to describe how I feel about that Mac it has been a clusterf_(K to upgrade!
Last edited:
I am about to receive my first cMP4,1. It is a single 4-core w3520 cpu, gt120 graphics, 10.11.6 el capitan, 12gb ram, still a 4,1. I would like to flash it to 5,1 and 144. Are there any steps I should take or tools I should use to save copies of what's inside it now before attempting any upgrades, to be somehow prepared in case it gets bricked?
You should run romtool first to have a backup of your rom contents.
if anything goes wrong it is much more easier to reconstruct the rom.
if anything goes wrong it is much more easier to reconstruct the rom.
@tsialex
I have been following the OpenCore thread recent posts around NVRAM issues, so am hoping I can post here for some guidance and feedback.
My system is as per my signature - cMP 4,1>5,1 dual CPU (12-Core) X5680's with two 16GB ram sticks in each tray (in slots 1 and 2, and 5 and 6). I am running RefindPlus/OpenCore Chainloader 0.6.5, and macOS 10.14.6 with 2021-01 Security Update (not the latest one).
I disabled SIP and restarted into my native Mojave drive (not using RefindPlus/OpenCore Chainloader) and dumped my ROM using RomTool and got this result:
Just in case you can't see it, the Full size is 3165. Ouch - that looks perilous.
I then did 5 x NVRAM resets (Cmd+Opt+P+R), then booted back into Recovery and disabled SIP again, and booted back into my native Mojave drive (not using RefindPlus/OpenCore Chainloader) and dumped my ROM again using RomTool and got this result:
And this second time I get a whopping Full size value of 33532.
What does this tel you? Anything to worry about?
If you don't mind @tsialex, do you mind PM me info about the service you provide. I have two cMP 4,1>5,1 12-Core and am considering protecting myself moving forward seeing I'm using RefindPlus/OpenCore Chainloader.
I have been following the OpenCore thread recent posts around NVRAM issues, so am hoping I can post here for some guidance and feedback.
My system is as per my signature - cMP 4,1>5,1 dual CPU (12-Core) X5680's with two 16GB ram sticks in each tray (in slots 1 and 2, and 5 and 6). I am running RefindPlus/OpenCore Chainloader 0.6.5, and macOS 10.14.6 with 2021-01 Security Update (not the latest one).
I disabled SIP and restarted into my native Mojave drive (not using RefindPlus/OpenCore Chainloader) and dumped my ROM using RomTool and got this result:
Just in case you can't see it, the Full size is 3165. Ouch - that looks perilous.
I then did 5 x NVRAM resets (Cmd+Opt+P+R), then booted back into Recovery and disabled SIP again, and booted back into my native Mojave drive (not using RefindPlus/OpenCore Chainloader) and dumped my ROM again using RomTool and got this result:
And this second time I get a whopping Full size value of 33532.
What does this tel you? Anything to worry about?
If you don't mind @tsialex, do you mind PM me info about the service you provide. I have two cMP 4,1>5,1 12-Core and am considering protecting myself moving forward seeing I'm using RefindPlus/OpenCore Chainloader.
Checking the available space for the 1st VSS store of the NVRAM is just one of the things that you have to look to know if the BootROM image is healthy. While is very useful to see if you gonna have a brick soon, unfortunately it's the only one that is accessible for non firmware engineers and everything else involves lot's of knowledge.
I bet that your Mac Pro has other cruft inside the NVRAM, with just 4 DIMMs being installed, the free space needs to be around 45000 to 40000 range.
I've sent a PM, follow the instructions and please send me the dumps before the 4-times NVRAM reset and not "cleaned" one.
I wouldn't worry about the size of the VSS free space too much.
This is pretty normal to be small when you use your MacPro on a daily basis with OpenCore.
A few reboots here some boots into Windows there, some tests with OC and the first VSS store is full. Nothing to worry about. Just do your NVRAM resets every few weeks and your VSS store is empty again.
As long as your binwalk looks good and your UEFI check is not totally out of place there's no need to be in panic of bricking your Mac just because your VSS free space is not big enough.
And: flashing a fresh cleaned bootrom will not prevent your Mac's VSS store from running full again. If you keep using your Mac like always your free space will fill up just as quickly as before.
Your advice here is valid only for perfectly working circular log, not when the it's lose track of the head/tail positions.
Nope, even with a perfectly working garbage collection, the NVRAM accumulates lot's of cruft over the years that the garbage collection can't erase in any way, like ASD and AHT reports/PanicLogs/MDM variables/FirmwarePassword variables and several others. What is stored inside the SPI is very different from what you see with nvram -xp, that will only show a limited scope of all variables stored.
A never booted image solves that once and for all, since it's completely clean. When you ever get a fully cleaned 1st VSS store reseting the NVRAM?
Btw, binwalk knows nothing about the NVRAM contents, if you don't have a signature file with everything you need to track, binwalk is utterly useless and will only show SecureBoot der/pks/dbs inside the NVRAM and ignore the dozens/maybe hundreds of different variables stored inside the NVRAM.
Newbie question, when I'm doing the NVRAM reset, should I be doing the following:
- Pull all discs out, leaving only the clean Mojave disc (used for OC configuration)
- Power on
- Do the 3x (5x?) NVRAM reset
- log into Mojave cleanly (and do a ROMTOOL backup while I'm there)
- Power off
- Attach OC disc
- boot back into Mojave
- (re)bless OC
- then reattach all other discs.
- Power up.
Your recipe is valid, but why all the song and dance with the multiple BootROM image backups? I understand doing that if you want to know when to flash back your "golden" BootROM dump tracking the free space available at the 1st VSS store, but then you do the BootROM dump before reseting the NVRAM.
You will never fully clean the 1st store of the NVRAM just with the NVRAM reset procedure (it's until you hear the 4th chime, btw), you can try a few times to see the most space you can get, but doing it as a regular procedure is useless.
Ok I'll scrap that back up bit
I have my "original" backup I did before I jumped into OC and that's the only one I made.
Did you ever checked that it's even a decent one?Ok I'll scrap that back up bit
I have my "original" backup I did before I jumped into OC and that's the only one I made.
If like me you’re not well versed in what to look for, you may want to get a proper reconstruction made. If not you may be thinking you have a ‘backup’ rom that’s clean when it may be like mine and already on the verge of bricking. OpenCore may have been a red herring for me as a lot of the issues I presumed were down to OpenCore were down to corrupt VSS store issue.
Yep, yours was a hybrid (early-2009 cross-flashed to MP5,1 firmware) and that's a brick paradise.
Almost 12 years of usage plus all the OpenCore tests you did has taken it's toll, reseting the NVRAM was getting you nowhere.
I know there’s so many things now working smoothly since the reconstruction that I had previously presumed were due to old age like the Bluetooth mouse not freezing constantly. I once lost my whole iCloud keychain / logins after installing High Sierra which I now think was probably related lol as the keychain is working fine in Mojave. ?
Last edited:
I guess you got me wrong here ---
I just wanted to point out that a small left free space of the first VSS store isn't any clue for a not correctly working garbage collection/bootrom.
I tested the following:
1. set up a clean bootrom (no VSS stores present)
2. flashed it to my cMP running Mojave with OC 0.6.6, rebooted -> first VSS store free space full size was about 58000
3. rebooted several times just chosing different startvolumes in OC bootpicker (ctrl enter) -> first VSS store free space full size was down to only 15000 after few reboots
4. resetted NVRAM 3x -> VSS store free space full size was about 58000 again
So, my conclusion is that flashing a reconstructed rom does not protect the VSS free space from shrinking quickly again.
But I have to agree that the only secure solution seems to be in fact flashing a clean bootrom now and then if you want to be absolutely sure your cMP never got bricked.
Yes, you are right that the VSS store will be filled soon and that a working garbage collection will get you back to a decent amount of free space when fully reseting the NVRAM, the problem is that people are finding that the once the head/tails of the circular log are not working correctly anymore, you won't regain space reseting the NVRAM and your Mac Pro 1st VSS store is always with less than a third of the total space.
Since I've showed how to check the available space on the OC thread, two other people discovered that the 2nd VSS store header was already corrupt, so now we are up to 3 in the last 10 days.
I have to admit when I saw your post on the OC thread and checked my own Mac's VSS available space I was initially shocked because it was down to about 4200, the second VSS showed 33000 of free space left.
And that even though I was forced to flash a fresh cleaned bootrom just 3 weeks ago due to the installation process of Windows 10 which left some bad certificate traces of course.
Haha, I was relieved after I did the NVRAM dance which cleared the whole second VSS store and released another 58000 of free space of the first VSS.
I have only done some reboots into different OSs and some minor OC config tests within these 3 weeks.
I guess OpenCore can be a dangerous killer for a cMP with a corrupt garbage collection.
Hi all
I ran into an issue with OS 0.6.6 and Big Sur today. Cold starting my mac, ends up in black screen, no dektop at all. It seems that the OS has fully loaded as a USB adapter I have seems to be working. The only thing that works is to unplug/plug the DP cable from the monitor or the GPU or force shutdown and then restart. Any ideas?
Thanks!
I ran into an issue with OS 0.6.6 and Big Sur today. Cold starting my mac, ends up in black screen, no dektop at all. It seems that the OS has fully loaded as a USB adapter I have seems to be working. The only thing that works is to unplug/plug the DP cable from the monitor or the GPU or force shutdown and then restart. Any ideas?
Thanks!
The idea that OpenCore is dangerous seems to have spread like a wild fire since post #6,990 on the OC thread. This needs clarification: OC facilitates day-to-day use of the machine, such as installation, updates, and in particular, multi-booting. It is these "normal" activities that lead to eventual issues.
OC is not the problem, it's just showing that one more time the BootROM is the Achilles heel of MP5,1.
In reality, MP4,, some earlier mid-2010, SPI flash memories are dropping like flies since the expiring date of writes are over long ago, that's why you see so much Software or Security Update X/Windows/OC bricked my Mac threads.
Being blunt here, every week a newbie or two opens a new thread about a MP4,1>5,1 or MP5,1 brick, usually after some hardware upgrades/NVRAM resets/lot's of software experimentation and an enormous number of NVRAM write cycles in a short time. It's becoming extremely boring having to explain again the same things over and over.
Start dumping the BootROM, check if you are low with available space inside the 1st VSS store. See this post below:
OpenCore on the Mac Pro
There is a extremely simple way (simple here as in not need to know how the NVRAM works/check free space indicators/validate checksums and etc) to check if the garbage collection failed and the need to reflash with the clean dump: Dump the BootROM with ROMTool Open the dump with the most...
forums.macrumors.com
Thanks. I did dump the BootRom and the space is 32998. The second VSS is empty (65448). I reinstalled 11.2.1 on top of the already existing installation and so far I cannot reproduce this issue. I;ll be certain about it tomorrow.Start dumping the BootROM, check if you are low with available space inside the 1st VSS store. See this post below:
OpenCore on the Mac Pro
There is a extremely simple way (simple here as in not need to know how the NVRAM works/check free space indicators/validate checksums and etc) to check if the garbage collection failed and the need to reflash with the clean dump: Dump the BootROM with ROMTool Open the dump with the most...
Thanks
PS I'll be able to send you the files for your diagnosis sometime tomorrow.
Last edited: