MP 1,1-5,1 [GUIDE] Bypass FileVault blacklist on 5,1 in Mojave -- on existing installation

[COV.DEV]

I know there are some methods to get FileVault to work but none that I recall are simple for someone who is already running macOS, or they require access to another Mac to do it.

Apple added simple board-id checks to the Security prefpane file and the command line utility to prevent enabling FileVault (which is why the 3,1 is able to enable FV when running Mojave).

You can modify the binary to bypass this check, this will allow those that had to disable FV to upgrade to Mojave to be able to re-enable it while on Mojave presuming they have a GPU that supports the correct EFI boot screens, and now with MVC RX 580 cards available, I'd suspect that the number of people able to do this will increase.


There are two methods available to bypass the blacklist.
1. Modify CLI tool
2. Modify Security PreferencePane

Personally, I'd suspect the best idea is to modify only the CLI tool for purposes of enabling FV and then manage the rest from the prefpane.

Option 1 (safest, shouldn't need SIP disabled as we aren't replacing any system files):
Copy the CLI tool over to your desktop for hex editing.

Mac-Pro:~ ludacrisvp$ cp /usr/bin/fdesetup ~/Desktop/

Open up your favorite hex editor, I prefer HexFiend, and search for the 5,1 board-id (Mac-F221BEC8) and then replace it with another ID, in my example I'm swapping in a 1,1 board-id (Mac-F4208DC8). I do suspect that you really could just change the ID to an invalid one of the same length and it would have the same effect (for example using Mac-F221BEC2 would probably work too as that is not a real board-id).
View attachment 854705

Some IDs for reference:

Mac-F4208DC8 - 1,1
Mac-F42C88C8 - 3,1
Mac-F221BEC8 - 5,1

Save the file, then you need to code sign it and you can actually run it from right there without replacing the original binary.

Mac-Pro:~ ludacrisvp$ sudo codesign -f -s - ~/Desktop/fdesetup
/Users/ludacrisvp/Desktop/fdesetup: replacing existing signature

Then execute the binary file to enable FileVault.

Mac-Pro:~ ludacrisvp$ sudo ~/Desktop/fdesetup enable

As the check is only processed during the attempt to enable FileVault, you really don't need to keep the modified binary and you can do the rest of the FileVault management from the Security PrefPane.

==========================================================
Option 2:

This one will probably require SIP to be disabled as you will need to replace the system file with your modified file.
I'm not going to cover that in this.

Copy the Security binary out of the preference pane to your desktop for modification.

Mac-Pro:~ ludacrisvp$ cp /System/Library/PreferencePanes/Security.prefPane/Contents/MacOS/Security ~/Desktop/Security

Make a backup of this one though as we have to replace it.

Mac-Pro:~ ludacrisvp$ cp /System/Library/PreferencePanes/Security.prefPane/Contents/MacOS/Security ~/Desktop/Security.orig

Same process here, there is a single match for the board ID, this time I used the 3,1 Board-ID
Here is the board-id for the 5,1 in the PrefPane for security.
View attachment 854706

Replace the signature on the binary:

Mac-Pro:~ ludacrisvp$ sudo codesign -f -s - ~/Desktop/Security
/Users/ludacrisvp/Desktop/Security: replacing existing signature

Replace the system file with your modified and signed binary:

Mac-Pro:~ ludacrisvp$ sudo cp ~/Desktop/Security /System/Library/PreferencePanes/Security.prefPane/Contents/MacOS/Security

Then launch system preferences and go to security and you should be free to enable FileVault like any normal Mac user.

Not sure if Catalina has the same block in place, as the 5,1 isn't supported on Catalina, if it does have this block as part of code carryover then I'd suspect the same modification could be made.

While copying with sudo cp ~/Desktop/Security /System/Library/PreferencePanes/Security.prefPane/Contents/MacOS/Security, I was denied copying as: cp: /System/Library/PreferencePanes/Security.prefPane/Contents/MacOS/Security: Operation not permitted. Any suggestion? Thanks!

 

[Ludacrisvp]

While copying with sudo cp ~/Desktop/Security /System/Library/PreferencePanes/Security.prefPane/Contents/MacOS/Security, I was denied copying as: cp: /System/Library/PreferencePanes/Security.prefPane/Contents/MacOS/Security: Operation not permitted. Any suggestion? Thanks!

If you are on Catalina then you'd have to get the System partition mounted with read/write access most likely.
Also, make sure you don't have System Prefs open at the time.

 

[COV.DEV]

While copying with sudo cp ~/Desktop/Security /System/Library/PreferencePanes/Security.prefPane/Contents/MacOS/Security, I was denied copying as: cp: /System/Library/PreferencePanes/Security.prefPane/Contents/MacOS/Security: Operation not permitted. Any suggestion? Thanks!

I managed to make it work. Thanks. But the new issue is that after FileVault encryption completed and everything looked good. Played around an hour and then restarted Mac. It just doesn't startup. Keeps in post loop. Any idea? Appreciated.

 

[Northernchip]

FYI I changed my SSD drive to a m2.NVME drive and added 128GB of memory. Option 2 still works.

 

[mckyvlle]

Joining the Mojave party (very) late, and indeed option 2 still works on a fresh install of 10.14.6.

 

[jmwmixing]

I managed to make it work. Thanks. But the new issue is that after FileVault encryption completed and everything looked good. Played around an hour and then restarted Mac. It just doesn't startup. Keeps in post loop. Any idea? Appreciated.

I seem to be having the same issue here, have you since found a solution?

UPDATE:

This method (number 2) for some reason decided to make my system drive become labeled as not bootable under Get Info on the drive. Tried using an external hard drive with a fresh OSX install to boot off of instead and that too wasn't able to boot on the machine. I would get two startup chimes and it would hang. A buddy of mine recommended trying to pop in the original GPU for the machine and that got me booting again. Able to login no problem, and now I am turning filevault off. Hopefully when I swap in my rx580 again with filevault off the drive will be read as bootable again.

Would love to know if anyone had any insight as to what could be causing this? Not in an IT related field so my knowledge is limited. For reference, this option worked to enable filevault up until I rebooted the machine.


UPDATE 2:

Turned off filevault, swapped the rx580 back in and the rig is now booting. Still unsure of what exactly went awry but happy the machine is running again.

 

[Macschrauber]

It happens I needed to add another user to the preboot list. Did that in security panel and it wont showed up.

sudo diskutil apfs updatePreboot /

fixed it

 

[Macschrauber]

Did 2020-004 Security Update from 18G5033 without a flaw. Made triple backup before but it was not needed.

 

[saltarob]

The first step is to install OpenCore. There is a guide for that. Then you can edit the config.plist. To enable BoardProduct spoofing:

Hi cdf,

thank you for the great tip. I followed your instructions and could switch on the FileVault on Mac Pro 5,1 Catalina 10.15.7.

But now I get a black screen after choosing the system in the Opencore Bootpicker. I followed the instructions in the Dortania Guide for FileVault.

Attached is the OpenCore Debug Log. Can somebody help please? Thank you so much.

 

[cdf]

Hi cdf,

thank you for the great tip. I followed your instructions and could switch on the FileVault on Mac Pro 5,1 Catalina 10.15.7.

But now I get a black screen after choosing the system in the Opencore Bootpicker. I followed the instructions in the Dortania Guide for FileVault.

Because of the hybridization (BoardProduct spoofing), you may need agdpmod (as described in Part 2 of the wiki) to enable all video inputs (although simply switching inputs may provide a temporary solution). On the other hand, now that you have enabled FileVault, you can turn off the hybridization if you don't need it for anything else. Also, be careful with the Dortania guide, because it may suggest a configuration that is undesirable for the Mac Pro...

 

[saltarob]

Because of the hybridization (BoardProduct spoofing), you may need agdpmod (as described in Part 2 of the wiki) to enable all video inputs (although simply switching inputs may provide a temporary solution). On the other hand, now that you have enabled FileVault, you can turn off the hybridization if you don't need it for anything else. Also, be careful with the Dortania guide, because it may suggest a configuration that is undesirable for the Mac Pro...

Thank you so much for your quick answer. I am using the latest config from h9826790, from his post #1314. I thought the boot-args would take care of the agdpmod:

<key>boot-args</key>
                <string>agdpmod=pikera shikigva=80 mbasd=1 -wegtree -no_compat_check no32exec=0</string>

Additionally I added also your agdpmod with the PciRoot from my machine, from the gfxutil:

<key>DeviceProperties</key>
    <dict>
        <key>Add</key>
        <dict>
            <key>PciRoot(0x0)/Pci(0x3,0x0)/Pci(0x0,0x0)/Pci(0x0,0x0)/Pci(0x0,0x0)</key>
            <dict>
                <key>shikigva</key>
                <data>UA==</data>
                <key>agdpmod</key>
                <data>cGlrZXJhAA==</data>
                <key>rebuild-device-tree</key>
                <data>AA==</data>
            </dict>
        </dict>

I also tried it with the hybridization switched off. But with both versions the screen stays black after the BootPicker.

Maybe something is wrong with the Dortania guide. Following are the changes I made according the Dortania guide:

Misc -> Boot

<key>PollAppleHotKeys</key>
<true/>

NVRAM -> Add -> 4D1EDE05-38C7-4A6A-9CC6-4BCCA8B38C14 (was already there)

UEFI -> Input

<key>KeySupport</key>
<true/>

UEFI -> Output

<key>ProvideConsoleGop</key>
<true/>

UEFI -> ProtocolOverrides

<key>FirmwareVolume</key>
<true/>
<key>HashServices</key>
<true/>
<key>AppleSmcIo</key>
<true/>

UEFI -> Quirks

<key>RequestBootVarRouting</key>
<true/>
<key>ExitBootServicesDelay</key>
<integer>3000</integer>

I think you mentioned in a post that you are also using FileVault? Which set up are you using?
Thank you.

 

[cdf]

I think you mentioned in a post that you are also using FileVault? Which set up are you using?

Yes, I'm using FileVault successfully. My setup corresponds exactly to the one detailed in the wiki (with the black boot screen option) and the advanced configuration.

Note that the Dortania guide is not aimed at real Macs.

 

[saltarob]

Yes, I'm using FileVault successfully. My setup corresponds exactly to the one detailed in the wiki (with the black boot screen option) and the advanced configuration.

Note that the Dortania guide is not aimed at real Macs.

Ah okay Now I understand. Sorry. I am new to the subject and the forum. So many guides, so many posts, videos, and other forums. I didn't understand whom to follow.

I started again, following only your wiki, and now everything is fine. Perfect! Thank you so much for your great work.

May I ask you another question? Did you also apply the other security features? Vault, ScanPolicy and Apple Secure Boot. Do these features work with Mac Pro 5,1?

Thank you.

 

[cdf]

May I ask you another question? Did you also apply the other security features? Vault, ScanPolicy and Apple Secure Boot. Do these features work with Mac Pro 5,1?

I'm using SecureBootModel but haven't yet looked into the other security features.

 

[saltarob]

I'm using SecureBootModel but haven't yet looked into the other security features.

Thank you. I made some tests with Vault, ScanPolicy and Apple Secure Boot. I followed the Dortania guides. I seems to work well.

I tried SecureBootModel with Default which made me able to boot the original Apple systems but not a Dosedude system. So it seems to work somehow. The Startup Security Utility seems to be bypassed. It reports only a non-compatible firmware.

Then I tried SecureBootModel with j137 and ApECID with a 64-bit integer. With these settings I could only boot my original Catalina install after the "bootefi personalize blessing" described in the guide.
So also here no real verification but it seems to work.

The Vault also works fine and it's actually very easy with the sign.command. But to disable it you have to replace also the OpenCore.efi file with the original one. That is not mentioned in the Dortania guide.

ScanPolicy works as described.

 

[Macschrauber]

Side note:
Mojave with FileVault survived Security Update 2020-007