MP 1,1-5,1 Mac Pro 3,1 NVMe support + Upgrade Guide + Questions

[joevt]

This has been done before by @abdyfranco 's Next Loader and even more elegantly by @joevt by using the driver#### NVRAM variables. I just want to document my experiences.

I first tied to install the mouSSE SSE4.2 emulator so I could use my flashed Radeon HD 7770 card, but I failed to get the kext to load. I was however able to boot into the NVMe drive in safe mode by disabling compatibility check in com.apple.Boot.plist. (Safe mode disables all boot-args, including -no_compat_check.)

For installing kexts, I use my KextUtil.sh script.

I then switched over to my unflashed Nvidia GTX 770 reference card. High Sierra runs and sleeps perfectly, with no kexts or patches installed. The problem is in getting the unmodified Mac Pro 3,1 to boot a APFS volume from a NVMe drive.

I had rEFInd installed on my Mac. I added apfs.efi and NvmExpressDxe.efi driver files into the rEFInd drivers_x64 folder. The Apple native boot picker does not show APFS or NVMe drives, but once in rEFInd they become visible. The standard rEFInd distribution however requires a boot screen.

I use apfs.efi from Driver####. The Apple boot picker does show APFS volumes. I'm not sure how well nvme works from Driver#### though. You can bless the Recovery volume of APFS so it will appear. You can bless the Preboot volume also. I think blessing the System volume just blesses the Preboot volume. The folder/file to be blessed is the one containing the boot.efi file. For all three volumes, you can add a .VolumeIcon.icns (use El Capitan to create it to make sure it has a it32 icon because later macOS versions don't create the it32 by default or you may need to use a utility to add the it32 icon to the icns - use Preview.app to see what icon types the icns contains). Since the System volume can't be blessed directly, it may appear as EFI Boot but at least the volume icon can tell you what it's pointing to. The other two can be given unique .disk_label files. I use my DiskUtil.sh script to create the disk labels (they can have multiple lines).

Regarding nvme booting, I have a HighPoint SSD7505 (four M.2 slots, gen 4). It has code in the PEX chip that makes the nvme devices appear as SCSI so they can be booted by the Startup Manager (hold option at boot) without an NVMe driver. The Startup Manager will set the EFI boot device path if you hold the control key and press return on an item. If there's a mystery item in the Startup Manager then that doing that will help you discover what it is. I haven't tested the boot from fake SCSI to see if it completes booting. Of course, the EFI device path created by Startup Disk preferences would point to NVMe instead of SCSI, so that method may not boot without NVMe driver. That's another thing to test. Third thing to test is the hardware RAID feature - the PEX chip has code to act as a RAID. Anyway, regardless of the EFI device driver (SCSI) used to start boot.efi, there should be a successful transition to kext device driver (NVME). The transition waits for the proper system partition to appear as a kext device - the partition is matched by volume UUID I guess.

To use my unflashed GTX 770 I upgraded to RefindPlus, which should provide a boot screen with EFI GOP cards. My GTX 770 however did not show any video before macOS booted.

RefindPlus has recently added code to get more GPUs to show boot screen (at least RefindPlus boot screen). For example, I can get RefindPlus to appear on a GTX Titan X (Maxwell). It has a GOP driver that doesn't load unless the EFI version is 2.1 or later. My MacPro3,1 EFI version is 1.1. RefindPlus fakes the version so the GOP will load. The RefindPlus log should have the info. Output of dh -d -v > dh_d_v.txt from EFI Shell may be useful to show what drivers exist. You can also find the location of the rom and dump the contents to a text file.

Nvidia Kepler cards work in Mojave and above, but they do not seem to provide boot screens with bootloaders like RefindPlus or OpenCore. Luckily there is a Mac EFI file for the GTX 770 which worked perfectly on my "Founders Edition" card.

I have the Mac Edition EVGA Nvidia GTX 680 which has UGA driver so it works. But the goal is to be able to use non-UGA graphics GOP cards like AMD and newer Nvidia. RefindPlus and OpenCore have UGAPassThrough which is a UGA on GOP driver. I would also like to try a GOP on UGA driver since there is a EFI screen shot driver that requires GOP that I would like to be able to use to get a screen shot of the Startup Manager.

Currently, there's an issue where the first stage boot process doesn't appear on my GOP card. RefindPlus is visible on my GOP card, but the first stage boot appears on my UGA card. So that's another issue I would like to get fixed.

At first I tried to install High Sierra before installing the AFPS and NVMe drivers for rEFInd. I thought this would not be an issue, as the High Sierra installer can see both NVMe and APFS drives. The installer refused to install in a APFS-formatted drive. This is reasonable, as the end result would be an unbootable system, but I wondered how Apple implemented this. Normally the kernel should know nothing of the pre-boot environment. All of it disappears once boot.efi calls ExitBootServices(). There must be some undocumented mechanism for passing information from EFI to macOS.

So the apfs.efi driver sets a flag that says this Mac can boot apfs? Maybe it's an nvram thing. Or maybe a feature flag?
OpenCore has a list of firmware feature flags at https://github.com/acidanthera/Open...nclude/Apple/IndustryStandard/AppleFeatures.h
Does loading the apfs.efi driver change the value of this flag? It doesn't seem to on my MacPro3,1.
I have a script called gfxutil.sh that has commands to show nvram variables. One of the commands is showfirmwarefeatures

At first I tried installing mouSSE from the Catalina patcher USB drive. It copied the kext to /Library/Extensions/ but the kext was not loaded. This was half expected as cache rebuild is incompatible between Catalina and older macOS versions. I then tried to manually flush the cache with sudo kextcache -i / but received complaints about file permissions. I then followed these instructions for installing 3rd party kexts in /Library/Extensions. The kext was accepted, but MouSSEstats claimes the driver was not loaded.

I use my KextUtil.sh script. It will check for kext Info.plist, bundle identifier, unload existing kext, delete existing kext, copy the kext, fix the permissions and ownership, move the copy to /Library/Extensions, and load it.

Booting into live Linux disabled the rEFInd boot entry and put the Mac into a weird boot order loop instead of booting back to rEFInd and my NVMe High Sierra installation:

• Restarting from Linux boots it into El Capitan.
• Restarting from El Capitan puts it into rEFInd
• rEFInd waits 20 seconds and boot into High Sierra on a NVMe drive.
• Restarting from High Sierra puts it into Linux.

On every restart the Mac flashes the folder icon with a question mark. The boot order has now become random, with a preference on Linux. The problem persists, even after I reinstalled rEFInd.

I checked the boot order with efibootmgr in Linux. This is the output. rEFInd is installed on the same hard drive as Linux. El Capitan is on a SSD in the optical bay. I cannot see anything missing. rEFInd is the first item, as it should be.

petri@petri-MacPro:~$ efibootmgr -v
BootCurrent: 0000
Timeout: 5 seconds
BootOrder: 0001,0000,0080
Boot0000* ubuntu    HD(1,GPT,44c4643d-d1e7-453e-9d50-866c7787ec9a,0x800,0x100000)/File(\EFI\ubuntu\shimx64.efi)
Boot0001* rEFInd Boot Manager    HD(1,GPT,44c4643d-d1e7-453e-9d50-866c7787ec9a,0x800,0x100000)/File(\EFI\refind\refind_x64.efi)
Boot0080* Mac OS X    PciRoot(0x0)/Pci(0x1f,0x2)/Sata(4,0,0)/HD(2,GPT,3b9502a7-9747-4d32-9c34-aed695e0a632,0x64028,0xddfac40)
Boot0081* Mac OS X    PciRoot(0x0)/Pci(0x1f,0x2)/Sata(3,0,0)/HD(2,GPT,a10fadc5-4c84-49d6-8642-a6677afad36d,0x64028,0x25294b40)
BootFFFF*     PciRoot(0x0)/Pci(0x1f,0x2)/Sata(2,0,0)/HD(1,GPT,44c4643d-d1e7-453e-9d50-866c7787ec9a,0x800,0x100000)/File(\EFI\BOOT\BOOTX64.efi)

My gfxutil.sh script has a command dumpallbootvars that will show the same info in macOS. It also shows the Driver#### variables. There's also commands to change the info.

One thing interesting about your ubuntu and rEFInd boot variables is that they are relative to an HD instead of absolute (from PciRoot). I guess if that works then it means you can move the drive to other locations and it will still boot. If that doesn't work then you should make them absolute.
There are a few ways to change the default boot:

1. Startup Disk preferences panel
2. Startup Manager (hold option at boot) - select an item and press Control-Return to make it the default
3. bless command in macOS
   1. For FAT or EFI: bless --mount /Volumes/xxx --file /Volumes/xxx/yyy/zzz.efi --setboot
   2. For HFS+ or APFS: bless --folder /Volumes/xxx/yyy --file /Volumes/xxx/yyy/zzz.efi
   3. To create a disk label: bless --folder /Volumes/xxx/yyy --label yourdisklabel
4. setbootorder and setbootvar from gfxutil.sh script
5. Boot Camp control panel in Windows
6. EasyUEFI.exe in Windows
7. etc.

A fundamental problem with rEFInd on a Mac is that the Apple Boot Picker application will only show one boot target for each physical drive, or at most partition.

The Apple Boot picker has the problem of one boot target per drive - for booting EFI, it is either the blessed file (for HFS+ or APFS) or the file with a specific name and location such as /EFI/BOOT/bootx64.efi for the EFI partition. rEFInd does not have that problem.

The Linux drive now has two boot targets on the EFI partition of the Linux drive. How do you chose which one to boot?

rEFInd can choose from multiple EFI files on the same partition. It scans for EFI files in certain locations, not just files named bootx64.efi. There are flags in the refind.conf file to control what rEFInd scans. If there's no flag that makes it scan a particular location, you can add a manual entry (manual stanza) with the file path (and volume name if you want it to be specific to a single volume). Read the rEFInd documentation.

This is what happens if you try to boot an unpatched High Sierra installation in Safe Mode. Drivers are not loaded and boot args are ignored. To get past this you need the plist patch.

You mean add the flag to Kernel Flags in the com.apple.Boot.plist file?
This command shows the flags in the com.apple.Boot.plist file of all the mounted file systems:
grep "string" /Volumes/*/Library/Preferences/SystemConfiguration/com.apple.Boot.plist | sed -E '/(.*plist): *(.*)/s//\2 \1/; /<string>mach_kernel<\/string>/d' | sort

I wanted to get into safe mode to install kext manually and to rebuild the kext cache. To get into Safe Mode I pressed F2 in rEFInd and selected Safe Mode. I was able to do the second part of the installation in safe mode, but once once I entered my Apple ID password Apple locked my account. I do not know if it was because of the safe mode or because it was the 10th time I had installed High Sierra the same day.

Maybe disconnecting internet until installation is done would help. Going into safe mode to install a kext should not be necessary.

Having the the NVMe drive is the Mac Pro and shutting down El Capitan leads to a kernel panic. The shutdown fails at the last moment and the Mac reboots. I have sent the report to Apple at least ten times. I do not know if they care what happens in El Capitan. There is nothing unusual about having a NVMe drive in a Mac Pro 3,1 even if Apple does not support any macOS version that supports it. Linux and Windows 10 have no problems.

I have not tried NVMe in 10.11 El Capitan. Maybe remove the IONVMEFamily.kext if it is causing kernel panic? It is not compatible with third party NVMe drives that use 512K block size until 10.13 High Sierra. Or you could try patching it. https://github.com/RehabMan/patch-nvme

 

[Dayo]

With the drivers included this way, macOS apfs volumes can now boot and can also appear and be selectable in the Startup Manager

The apfs.efi file can differ between Mac OS versions and might want to use APFSDriverLoader.efi instead as this should load the appropriate apfs.efi file from each Mac OS instance as required.

Currently, there's an issue where the first stage boot process doesn't appear on my GOP card. RefindPlus is visible on my GOP card, but the first stage boot appears on my UGA card. So that's another issue I would like to get fixed.

THIS ISSUE HAS SINCE BEEN ADDRESSED IN REFINDPLUS WITH JOEVT'S HELP.
THE DRIVER IS NO LONGER REQUIRED AND USAGE IS ACTIVELY DISCOURAGED.​

Adding the OsxAptioDrv3 driver to RefindPlus will restore the Stage 1 loading screen on a cMP 3,1.

Caveats:

1. Does not work with GPUs (mainly Nvidia) where GOP needs to the reloaded from RAM.
2. Do not rename the file, or more accurately, ensure that the "OsxAptioDrv" part of the name is maintained.
   
   • RefindPlus will disable any driver with a name containing "OsxAptioDrv" when Condition 1 is met so that such GPUs continue booting albeit without the Stage 1 Loading Screen.
     
     • You will just get a grey screen for Stage 1. Stage 2 will show as normal
   • If not disabled, the driver will cause a crash under Condition 1.
   • The driver is not required on cMP 5,1. Not Tested on cMP 4,1.

 

[joevt]

The apfs.efi file can differ between Mac OS versions and might want to use APFSDriverLoader.efi instead as this should load the appropriate apfs.efi file from each Mac OS instance as required.


Adding the OsxAptioDrv3 driver to RefindPlus will restore the Stage 1 loading screen on a cMP 3,1.

Caveats:

1. Does not work with GPUs (mainly Nvidia) where GOP needs to the reloaded from RAM.
2. Do not rename the file, or more accurately, ensure that the "OsxAptioDrv" part of the name is maintained.
   • RefindPlus will disable any driver with a name containing "OsxAptioDrv" when Condition 1 is met so that such GPUs continue booting albeit without the Stage 1 Loading Screen.
   • If not disabled, the driver will cause a crash under Condition 1.

Your post's OsxAptioDrv3 driver link is broken. Strange that a memory fix driver for UEFI 2.1 Hackinthosh has a useful affect on old EFI 1.1 Mac Pro. Maybe it's doing more than memory fixing. Should have a look at the code to see what all it's doing...

 

[Dayo]

Your post's OsxAptioDrv3 driver link is broken. Strange that a memory fix driver for UEFI 2.1 Hackinthosh has a useful affect on old EFI 1.1 Mac Pro. Maybe it's doing more than memory fixing. Should have a look at the code to see what all it's doing...

Fixed the link.

Definitely a "byproduct" effect. Haven't found the actual source code as yet and really haven't yet had time to experiment to nail down the actual useful item.

Found the source for successor to this, AptioMemoryFix.efi, but not yet tested that one to see if it has the same benefit.

 

[joevt]

Fixed the link.

Definitely a "byproduct" effect. Haven't found the actual source code as yet and really haven't yet had time to experiment to nail down the actual useful item.

Found the source for successor to this, AptioMemoryFix.efi, but not yet tested that one to see if it has the same benefit.

It's called OsxAptioFix3Drv. For source code of an EFI driver, just add .inf extension which gets me to
https://github.com/MarkVillacampa/clover/blob/master/Clover.dsc
Then search for OsxAptioFix3Drv in that repository which leads to
https://github.com/MarkVillacampa/clover/search?q=OsxAptioFix3Drv
I'm not sure that is the actual code for the OsxAptioFix3Drv that you attached to your post. Going to the actual Clover GitHub at https://github.com/CloverHackyColor/CloverBootloader , then searching for OsxAptioFix3Drv, finds this:
https://github.com/CloverHackyColor/CloverBootloader/search?q=OsxAptioFix3Drv
which is similar to results in the first repository I found

I see some graphics/console code in there (function MOHandleProtocol in OsxAptioFix3Drv.c ).

 

[Dayo]

THIS ISSUE HAS SINCE BEEN ADDRESSED IN REFINDPLUS WITH JOEVT'S HELP.
THE DRIVER IS NO LONGER REQUIRED AND USAGE IS ACTIVELY DISCOURAGED.​

I see some graphics/console code in there (function MOHandleProtocol in OsxAptioFix3Drv.c ).

Probably the active item as it provides GOP on the ConOut Handle (not present in cMP 3,1).
This incidentally allows rEFInd to show bootscreen on cMP as it just uses the GPU version but there is no Stage 1 Loading Screen.

cMP 5,1 has ConOut GOP but it is some mutant Apple specific version hence, rEFInd does not show up on cMP 5,1 but presumably allows the Stage 1 Loading Screen after the OpenCore fix (also present in RefindPlus) is applied.

Will have a go at testing this later.

---

Scratch the above. Might have inadvertently fixed the Stage 1 Loading on cMP5,1 in RefindPlus when running THIS CODE (Runs on cMP5,1 when trying to fix the broken cMP5,1 console GOP).

It basically does what that function in the driver is doing.

Will look closer later.

---

Quick test to extend the bit above to cover cMP3,1 gets the Stage 1 Screen without the driver. So it was just fixed on cMP5,1 without knowing.

 

[Petri Krohn]

HOW MACS BOOT?

I have spent three evenings reading Roderick W. Smith’s rEFInd pages and finally start to have some understanding on how Macs boot. There are three ways to specifying what to boot, Boot Picker, NVRAM BootXXXX variables, and Bless, each of which I will discuss separately. Please correct me if I am wrong.

Boot Picker

The boot picker, or “Startup Manager”, as Apple calls it, builds the list of startup disks from scratch every time it is invoked. It lists five types of disks.

• MBR formated drives or optical disks that will be booted in legacy BIOS mode. These are listed as “Windows” and are outside the scope of this post.
• Network volumes. Pressing “N” in Boot Picker will make the Mac search network volumes to boot from.
• macOS installations on HFS+ or APFS file systems. It is also possible to make a Linux installation mimic a macOS installation, so that boot picker will directly launch its kernel.
• macOS recovery partitions.
• FAT formatted EFI partitions with a BOOTX64.efi file in the /EFI/boot/ folder. There is no way of making the boot picker launch any other EFI file. (Some sources suggest that this EFI partition could be formatted as HFS+.)

There is no way for anything in NVRAM to affect how the boot picker works. The only exception is that the current startup disk is preselected and highlighted on the list.

The boot picker can write too NVRAM. Pressing Ctrl+Enter while in boot picker starts the selected drive and blesses it. It creates a BootXXXX NVRAM variable for the drive, clears the BootOrder list and adds the hex number of the BootXXXX variable as the only boot option on the list.



The boot picker is only launched if the Mac has a Mac EFI graphics card. It is not possible to select the boot drive by blindly pressing the right and left arrow keys and enter. Only three keys at startup affect the choice of the boot volume: C boots to an optical drive. R boots a recovery partition and N boots a network volume. (Refind has many more keys available to select the boot volume and even works without a boot screen.)

The macOS Startup Disk utility in System Preferences works much like the boot picker. The main difference is that it does not show recovery partitions or EFI boot partitions. Linux or Windows installations in EFI format will not show up on the list.

NVRAM Boot#### variables

The BootOrder variable and the BootXXXX variables are part of the EFI specification. On the Mac Pro they behave exactly as specified. The boot picker, the Startup Disk utility and the bless command all manipulate these variables. It is also possible to modify the values from an EFI script, an EFI shell or from Linux (and most likely from Windows.)

When started without interference, the Mac boots to the first boot target defined in the BootOrder list. Usually this list only has one entity. If the boot target is missing, for example because a USB drive has been removed, or the odd case where the Mac does not have EFI drivers to read the boot drive, the Mac waits for about one minute and then tries to boot to the drive it last booted from. (BootFFFF?)

Apple's utilities give the selected boot target the number 0x0080 and define the NVRAM variable Boot0080. The boot picker and the Startup Disk utility clear the BootOrder list of any other boot targets.

Bless

The bless command in its variable versions also writes to the BootOrder and Boot#### variables. Unlike the boot picker it does not overwrite the BootOrder list. This is the output from the Linux command efibootmgr -v after blessing rEFInd from El Capitan:

BootOrder: 0080,0001
Boot0000* ubuntu    HD(1,GPT,44c4643d-d1e7-453e-9d50-866c7787ec9a,0x800,0x100000)/File(\EFI\ubuntu\shimx64.efi)
Boot0001* rEFInd Boot Manager    HD(1,GPT,44c4643d-d1e7-453e-9d50-866c7787ec9a,0x800,0x100000)/File(\EFI\refind\refind_x64.efi)
Boot0080* Mac OS X    HD(1,GPT,44c4643d-d1e7-453e-9d50-866c7787ec9a,0x800,0x100000)/File(\EFI\refind\refind_x64.efi)
Boot0081* Mac OS X    PciRoot(0x0)/Pci(0x1f,0x2)/Sata(3,0,0)/HD(2,GPT,a10fadc5-4c84-49d6-8642-a6677afad36d,0x64028,0x25294b40)
BootFFFF*     PciRoot(0x0)/Pci(0x1f,0x2)/Sata(4,0,0)/HD(2,GPT,3b9502a7-9747-4d32-9c34-aed695e0a632,0x64028,0xddfac40)/File(\System\Library\CoreServices\boot.efi)

The previous boot target at 0x0080 was overwritten.

Bless may also write to Apple-specific NVRAM variables efi-boot-device and efi-boot-device-data. I have no idea how these affect the selection of the boot target and what their relationship is to the Boot#### variables.

Writing to NVRAM does not alone make drives bootable. The bless command must also give some pixie dust to macOS volumes to make them bootable. (What?)

Modifying the boot picker boot screen

None of the NVRAM variables directly affect what is shown by the boot picker. There are however a few ways to change what boot targets are shown and how they are displayed. The most obvious one is adding a volume icon. Also a name can be given to volumes. (How?) Loading additional EFI drivers before the boot picker is launched will make more targets appear on the boot screen.

Masquerading as a macOS boot volume

An old Ubuntu help document list what is need in order to make a Linux or Windows partition appear in the Apple boot picker menu and be selectable as a startup disk.

• /mach_kernel (can be empty)
• /.VolumeIcon.icns (optional icon)
• /System/Library/CoreServices/boot.efi - must be bless'ed and may be booted, if so should be a symlink to the actual bootloader.
• /System/Library/CoreServices/SystemVersion.plist file

This boot partition must be formatted in a format that macOS or the boot picker understands.

Older guides suggested that the Linux boot partition be in HFS+ format, but actually it should be possible to use any format as long as the required EFI drivers are loaded before the boot picker.

Ubuntu installation on USB stick with pure EFI boot (Mac compatible)

All I wanted was just:

medium.com

Loading EFI drivers

It is possible to extend EFI firmware by loading additional drivers. (This is what Extensible Firmware Interface stands for.) In rEFInd the drivers to be loaded are placed in the /refind/drivers_x64/ folder.

It is also possible to load the drivers before the Apple boot picker is shown. This will effect what the boot picker shows. The drivers to be loaded are specified using DriverXXXX variables. Like BootOrder there is also a DriverOrder variable. The drivers to be loaded are placed on a suitable drive that is understood and accessible to the EFI firmware.

Divers typically needed are APFS drivers, NVMe drivers, and ext4 drivers for Linux. @DearthnVader has been able to boot a Mac from a USB 3 add-in card by adding USB 3 drivers to rEFInd.

Bootable XHCI PCI-E for the 3,1.( Experimental).

I've had success making a USB3.0( FL1100 chipset) bootable on my MP3,1. I've also been able to boot from USB2.0 flash drives connected to the USB3.0 card, but I still have some issues to sort out there, as some 2.0 Flashdrives are only bootable with the USB3 to SATA drive I have connected to...

forums.macrumors.com

Booting from NVMe drives

As I posted earlier, I was able to boot from a NVMe drive on an unflashed Mac Pro 3,1 by placing the NVMe driver in rEFInds drivers folder. Using the macOS Startup Disk utility in High Sierra I was also able to select this drive as the default boot volume. Starting the Mac with these settings led to the Mac freezing with a gray screen. This the output of the efibootmgr -v command:

BootOrder: 0080
Boot0000* ubuntu    HD(1,GPT,44c4643d-d1e7-453e-9d50-866c7787ec9a,0x800,0x100000)/File(\EFI\ubuntu\shimx64.efi)
Boot0001* rEFInd Boot Manager    HD(1,GPT,44c4643d-d1e7-453e-9d50-866c7787ec9a,0x800,0x100000)/File(\EFI\refind\refind_x64.efi)
Boot0080* Mac OS X    PciRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x0)//HD(2,GPT,a851f523-f687-40e0-a927-afa6c9614ab8,0x64028,0x1dc8f260)/VenMedia(be74fcf7-0b7c-49f3-9147-01f4042e6842,56daee376492c0478163125fdc3db17d)/File(\5E3C3B21-2EE4-4AC3-B7BE-505EBD6EC7ED\System\Library\CoreServices\boot.efi)
Boot0081* Mac OS X    PciRoot(0x0)/Pci(0x1f,0x2)/Sata(3,0,0)/HD(2,GPT,a10fadc5-4c84-49d6-8642-a6677afad36d,0x64028,0x25294b40)
BootFFFF*     PciRoot(0x0)/Pci(0x1f,0x2)/Sata(3,0,0)/HD(2,GPT,a10fadc5-4c84-49d6-8642-a6677afad36d,0x64028,0x25294b40)/File(\System\Library\CoreServices\boot.efi)

To actually boot from the NVMe drive I would need to load the NVMe drivers using DriverXXXX variables.

Next: Bless-drivers script

My next aim is to write an EFI script that lists all the .efi drivers in the current directory and creates DriverXXXX NVRAM variables for each of them, using the EFI bcfg command. (See page 96 on the EFI shell specification.)

Issues
To be added...

Linux and rEFInd

• Incomplete device paths
• Fallback EFI

El Capitan NVMe drivers

 

[joevt]

Boot Picker

The boot picker, or “Startup Manager”, as Apple calls it, builds the list of startup disks from scratch every time it is invoked. It lists five types of disks.

• MBR formated drives or optical disks that will be booted in legacy BIOS mode. These are listed as “Windows” and are outside the scope of this post.

Any disk with legacy boot code in the MBR (first block) will be listed once as "Windows" even if the disk has multiple bootable legacy partitions. If selected, the Mac enters CSM or legacy boot mode. In this mode only certain disks will be bootable (the built-in DVD drives, and the internal drive bays). It may be possible that some PCIe devices have PCI option roms with BIOS images that can modify the list of legacy bootable devices (maybe advertised as supporting booting for Boot Camp). Some PCIe devices with BIOS images in the option roms may cause legacy booting problems. GPUs usually have legacy BIOS code. The code in the MBR (first block of disk) usually executes the code in the PBR (first block of partition) of the partition that is marked as active in the MBR.

• macOS installations on HFS+ or APFS file systems.

If an HFS+ or APFS volume has bless information (System folder and System file), then the Startup Manager will list that as an option and will use the .disk_label file as the label for the icon. I don't believe RefindPlus and OpenCore look at bless information.
The system folder is usually /System/Library/CoreServices and the System file is usually /System/Library/CoreServices/boot.efi, but will be different on the Preboot volume.
I made a version of bless that can get or set the bless information of any HFS+ or APFS volume. https://github.com/joevt/bless

• It is also possible to make a Linux installation mimic a macOS installation, so that boot picker will directly launch its kernel.

REFInd has an install option --ownhfs that does this for its boot loader.

• macOS recovery partitions.
• FAT formatted EFI partitions with a BOOTX64.efi file in the /EFI/boot/ folder. There is no way of making the boot picker launch any other EFI file. (Some sources suggest that this EFI partition could be formatted as HFS+.)

Startup Manager will list a "EFI Boot" option for any volume that it has a File System driver for and which has a file at /EFI/boot/bootx64.efi. This includes HFS+. It could include Ext4 and NTFS partitions if you install the EFI drivers for those.

There is no way for anything in NVRAM to affect how the boot picker works. The only exception is that the current startup disk is preselected and highlighted on the list.

I've started looking at OpenCore. It may be that the 4D1EDE05-38C7-4A6A-9CC6-4BCCA8B38C14efaultBackgroundColor NVRAM variable has an effect. At least a couple times the Startup Manager has started up and drawn on a black background which is the background color I had setup in OpenCore. I need to test this with other colors to be sure. Black makes the labels disappear because they are also black (they do not get drawn as white like they do on new Macs).
The NVRAM Driver#### variables could load drivers that affect the Startup Manager (device drivers to enable more types of disk devices, file system drivers, a driver to enable USB 3.x, a driver to allow taking screenshots, a driver to override any functionality including what the Startup Manager draws, etc.)

The boot picker can write too NVRAM. Pressing Ctrl+Enter while in boot picker starts the selected drive and blesses it. It creates a BootXXXX NVRAM variable for the drive, clears the BootOrder list and adds the hex number of the BootXXXX variable as the only boot option on the list.

I don't think Startup Manager changes the bless information of an HFS+ or APFS volume since it used the bless information to find that item. It probably just changes the BootXXXX (usually Boot0080) and BootOrder NVRAM variables.

The boot picker is only launched if the Mac has a Mac EFI graphics card.

Mac EFI means that it has EFI that matches the Mac you use it with. Mac edition cards for the Mac Pro usually use UGA. I suppose it may be possible to use a EFI card that uses GOP by adding a UGA on GOP driver to Driver####. Some EFI cards have GOP drivers that require UEFI 2.1. A driver can be added to Driver#### to fake UEFI 2.1. RefindPlus has code to do that but RefindPlus loads after Startup Manager.

The macOS Startup Disk utility in System Preferences works much like the boot picker. The main difference is that it does not show recovery partitions or EFI boot partitions. Linux or Windows installations in EFI format will not show up on the list.

The Startup Disk utility shows a Windows option for each Windows partition of a disk, but only the one marked as active in the MBR will get loaded.
The Startup Disk utility does not use bless information - it searches for certain files in certain folders (e.g. /System/Library/CoreServices/boot.efi). Startup Disk may change the bless information of an HFS+ or APFS volume. It may change the disk label of certain disks (if you create custom disk labels, you may want to lock them so they don't get changed sudo chflags schg .disk_label).

The BootOrder variable and the BootXXXX variables are part of the EFI specification. On the Mac Pro they behave exactly as specified. The boot picker, the Startup Disk utility and the bless command all manipulate these variables. It is also possible to modify the values from an EFI script, an EFI shell or from Linux (and most likely from Windows.)

My script gfxutil.sh script at https://gist.github.com/joevt/477fe842d16095c2bfd839e2ab4794ff can change the BootXXXX variables. In Windows, you can use the EasyUEFI.exe https://www.easyuefi.com/index-us.html

The bless comment in its variable versions also writes to the BootOrder and Boot#### variables. Unlike the boot picker it does not overwrite the BootOrder list. This is the output from the Linux command efibootmgr -v after blessing rEFInd from El Capitan:

gfxutil.sh has the dumpallbootvars command that can show all the same info in macOS.

Bless may also write to Apple-specific NVRAM variables efi-boot-device and efi-boot-device-data. I have no idea how these affect the selection of the boot target and what their relationship is to the Boot#### variables.

The BootXXXX variables contain EFI device paths and may also include parameters. efi-boot-device-data path is just a EFI device path. eft-boot-device is an xml representation of the device path. These are set by bless but I don't know if they get used outside of bless.

dumpallbootvars
BootCurrent: Boot0080 
BootNext: 

BootOrder: Boot0080 
Boot0080: 1, "Mac OS X", "PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)/HD(1,GPT,97CD1709-FE99-4E1F-BF41-75D379A46CC4,0x28,0x64000)/\EFI\BOOT\bootx64.efi"
Searching: Boot0000
Searching: Boot007F
Searching: Boot0081
Boot0081: 1, "Mac OS X", "PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x2,0x0,0x0)/HD(10,GPT,977D4C1D-54CA-4594-B9F6-FCABF87AC36C,0x4B405D88,0x174876E8)/VenMedia(BE74FCF7-0B7C-49F3-9147-01F4042E6842,7E98A08505BD8940A1199C16AA7CD67D)/\10D5CE8C-43FE-43F2-8089-DAEDF6485644\System\Library\CoreServices\boot.efi"
Searching: BootFFFF
BootFFFF: 1, "", "PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x2,0x0,0x0)/HD(9,GPT,717720E1-2443-443E-B26D-D85098BC319E,0x2E34C888,0x1D0B9500)/VenMedia(BE74FCF7-0B7C-49F3-9147-01F4042E6842,9494DC23508D8540A389493878A2404B)/\System\Library\CoreServices\boot.efi"

DriverOrder: Driver0080 
Driver0080: 1, "apfs.efi", "PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)/HD(1,GPT,97CD1709-FE99-4E1F-BF41-75D379A46CC4,0x28,0x64000)/\drivers\apfs.efi"
Searching: Driver0000
Searching: Driver007F
Searching: Driver0081
Driver0081: 1, "FakeUEFI2.efi", "PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)/HD(1,GPT,97CD1709-FE99-4E1F-BF41-75D379A46CC4,0x28,0x64000)/\drivers\FakeUEFI2.efi"
Driver0082: 1, "nv_gop_GF10x", "PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)/HD(1,GPT,97CD1709-FE99-4E1F-BF41-75D379A46CC4,0x28,0x64000)/\efis\nv_gop_GF10x.efi"
Driver0083: 1, "ReloadPCIRom.efi", "PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)/HD(1,GPT,97CD1709-FE99-4E1F-BF41-75D379A46CC4,0x28,0x64000)/\drivers\ReloadPCIRom.efi"
Driver0084: 1, "UgaOnGop.efi", "PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)/HD(1,GPT,97CD1709-FE99-4E1F-BF41-75D379A46CC4,0x28,0x64000)/\drivers\UgaOnGop.efi"
Driver0085: 1, "CrScreenshotDxe.efi", "PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)/HD(1,GPT,97CD1709-FE99-4E1F-BF41-75D379A46CC4,0x28,0x64000)/\drivers\CrScreenshotDxe.efi"
Searching: DriverFFFF


dumpallioregefipaths
/Root//chosen/boot-file-path = \23DC9494-8D50-4085-A389-493878A2404B\System\Library\CoreServices\boot.efi
/Root//chosen/boot-device-path = PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x2,0x0,0x0)/HD(9,GPT,717720E1-2443-443E-B26D-D85098BC319E,0x2E34C888,0x1D0B9500)/VenMedia(BE74FCF7-0B7C-49F3-9147-01F4042E6842,E9484694FD518743B58EAD0F67C43F42)
/Root//options/BootCampHD = PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)
/Root//options/efi-boot-device-data = PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)/HD(1,GPT,97CD1709-FE99-4E1F-BF41-75D379A46CC4,0x28,0x64000)/\EFI\BOOT\bootx64.efi
/Root/MacPro3,1/AppleACPIPlatformExpert/AppleEFIRuntime/AppleEFINVRAM/BootCampHD = PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)
/Root/MacPro3,1/AppleACPIPlatformExpert/AppleEFIRuntime/AppleEFINVRAM/efi-boot-device-data = PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x1,0x0,0x0)/HD(1,GPT,97CD1709-FE99-4E1F-BF41-75D379A46CC4,0x28,0x64000)/\EFI\BOOT\bootx64.efi

nvramp efi-boot-device | plutil -convert json -r -o - -
[
  {
    "BLLastBSDName" : "disk5s1",
    "IOMatch" : {
      "IOPropertyMatch" : {
        "UUID" : "97CD1709-FE99-4E1F-BF41-75D379A46CC4"
      },
      "IOProviderClass" : "IOMedia"
    }
  },
  {
    "IOEFIDevicePathType" : "MediaFilePath",
    "Path" : "\\EFI\\BOOT\\bootx64.efi"
  }
]

Modifying the boot picker boot screen

None of the NVRAM variables directly affect what is shown by the boot picker. There are however a few ways to change what boot targets are shown and how they are displayed. The most obvious one is adding a volume icon. Also a name can be given to volumes. (How?) Loading additional EFI drivers before the boot picker is launched will make more targets appear on the boot screen.

Except maybe the stuff I mentioned above about background color?

The bless command can create a .disk_label file. On old Macs, the label gets displayed only for blessed HFS+ and APFS items. Otherwise the default "EFI Boot" label is displayed (or the default "Windows" label for whole disks having boot code in the MBR). I suppose one could create a driver that overrides that functionality somehow.

.VolumeIcon.icns always works (except for Windows legacy boot option) but you need to make sure it includes a it32 icon type (I have a script to check and fix all my volume icons and also to set the custom icon flag of the volume). .VolumeIcon.icns for Catalina volume needs to be replaced with an actual icns file instead of a link.

OpenCore also has a disklabel command to create disk label (not to be confused with the disklabel command built into macOS which does something totally unrelated).

My diskutil.sh script at https://gist.github.com/joevt/6d7a0ede45106345a39bdfa0ac10ffd6 has a command to create a multi line disk label:
makemultilinedisklabel "/Volumes/Install Catalina 10.15.7 Patched/System/Library/CoreServices" "Install"$'\n'"Catalina"$'\n'"Patched"$'\n'"10.15.7"

When making custom disk labels, especially for macOS installations, you need to lock the disk label so macOS doesn't automatically change it (I don't know when or how it does that). My makemultilinedisklabel command does that. My fixapfsbooter fixes the .VolumeIcon.icns and disk label files for my APFS macOS installations. But you may want to do something different with yours.

fixapfsbooter /Volumes/Mojave4 "Mojave4"
fixapfsbooter /Volumes/Mojave5 "Mojave5"
fixapfsbooter /Volumes/Mojave6 "Mojave6"
mount | grep ' on / ' | grep -q 'read-only' && sudo mount -uw /
fixapfsbooter / "Catalina"
fixapfsbooter "/Volumes/BigSurB6 - Data" "BigSur"

Masquerading as a macOS boot volume

An old Ubuntu help document list what is need in order to make a Linux or Windows partition appear in the Apple boot picker menu and be selectable as a startup disk.

• /mach_kernel (can be empty)
• /.VolumeIcon.icns (optional icon)
• /System/Library/CoreServices/boot.efi - must be bless'ed and may be booted, if so should be a symlink to the actual bootloader.
• /System/Library/CoreServices/SystemVersion.plist file

This boot partition must be formatted in a format that macOS or the boot picker understands.

Older guides suggested that the Linux boot partition be in HFS+ format, but actually it should be possible to use any format as long as the required EFI drivers are loaded before the boot picker.

Ubuntu installation on USB stick with pure EFI boot (Mac compatible)

All I wanted was just:

medium.com

As mentioned above, rEFInd's --ownhfs install method does something similar (but you need to fix the SystemVersion.plist file - which may be formatted incorrectly).
When installed this way, it becomes selectable in the Startup Disk preferences panel. If the SystemVersion.plist is correctly formatted then the version in that file will show in the Startup Disk preferences panel.
With rEFInd, boot.efi is not a symlink - it is the file that will get booted.

Divers typically needed are APFS drivers, NVMe drivers, and ext4 drivers for Linux. @DearthnVader has been able to boot a Mac from a USB 3 add-in card by adding USB 3 drivers to rEFInd.

Bootable XHCI PCI-E for the 3,1.( Experimental).

I've had success making a USB3.0( FL1100 chipset) bootable on my MP3,1. I've also been able to boot from USB2.0 flash drives connected to the USB3.0 card, but I still have some issues to sort out there, as some 2.0 Flashdrives are only bootable with the USB3 to SATA drive I have connected to...

forums.macrumors.com

I tried compiling the xhci driver (with some changes to support EFI 1.1) and it sort of worked when I loaded it but I couldn't get the EFI Shell to show any new file systems when connecting a USB drive. I guess the built-in USB EFI drivers don't work with the XHCI driver even though I think I'm providing the same USB protocols used by the USB 2.0 drivers.

To actually boot from the NVMe drive I would need to load the NVMe drivers using DriverXXXX variables.

Next: Bless-drivers script

My next aim is to write an EFI script that lists all the .efi drivers in the current directory and creates DriverXXXX NVRAM variables for each of them, using the EFI bcfg command. (See page 96 on the EFI shell specification.)

Or you could try the commands in my gfxutil.sh script. I guess one thing I need to add is a more direct method to create a EFI device path to the driver. The current method is to bless the driver as if you were going to boot from it, then see what device path bless created for it, rebless what you really want to boot, use the device path with the setbootvar command. Instead of using bless, use the code that bless uses, or add code to bless that outputs the EFI path without modifying NVRAM.

 

[Dayo]

If the boot target is missing, for example because a USB drive has been removed, or the odd case where the Mac does not have EFI drivers to read the boot drive, the Mac waits for about one minute and then tries to boot to the drive it last booted from. (BootFFFF?)

It acually enters into a boot loader search process following specific rules:

• First, available EFI partitions are searched and the first fallback boot loader, /EFI/BOOT/BOOTx64.efi, found will be loaded.
  • Search order appears to match results of diskutil list ... effectively random.
• If a fallback boot loader is not found, the firmware will start to search through various ports in a predefined order looking for the Mac boot loader, /System/Library/CoreServices/boot.efi, and the first one found will be loaded.
  • The search order is:
    • 2 x PATA (SATA on 5,1) Ports in the SuperDrive Bay (Internal search order seems to be top then bottom port ... not sure)
    • 4 x PCIe Slots (Not sure of internal search order)
      • This might actually be before the PATA Ports ... can't recollect.
    • 4 x SATA Bays (Internal search order seems to be Bay 1 to Bay 4)
    • External [USB etc] Ports (Not sure of internal search order)

 

[Petri Krohn]

SETTING SLOT 2 SPEED to 5 GT/s

I am testing a dual M.2 to PCIe adapter card, so I finally needed to figure out how to set Slot 2 of my Mac Pro 3,1 to PCIe 2.0 speeds of 5 GT/s. I spent half a day googling and reading old post and this is what I found.

To set the link speed in macOS you have to use the pciutils. The package is written between 1997 and 2020 by Martin Mares at ucw.cz. The current version of pciutils is available here:

The PCI Utilities

mj.ucw.cz

There is a also a repository on Github:

GitHub - pciutils/pciutils: The PCI Utilities

The PCI Utilities. Contribute to pciutils/pciutils development by creating an account on GitHub.

github.com

The readme file says it runs on Linux, FreeBSD, Windows and a number of lesser know operating systems. macOS is not mentioned, but I guess the reference to Darwin means it should work.

When using pciutils you have to run the commands after every reboot.

@freqrider mentions lnkspd.

Here is an alternative to pcieutils, for those who have had trouble configuring it. I have tested it and it works perfectly...

lnkspd uses pciutils. It also uses a kext named directhw. The benefit is having a LaunchDaemon (the plist file) that executes the script on boot. (I have not tested it so I cannot comment further.)

GitHub - matatata/lnkspd: sets PCIe target link speed, uses pciutils lib and requires DirectHW.kext to run

sets PCIe target link speed, uses pciutils lib and requires DirectHW.kext to run - matatata/lnkspd

github.com

Googling for "mac install pciutils" returns a list of steps.

Install pciutils

1. Install pciutils.
2. Download and decompress the zip: https://github.com/pciutils/pciutils. Open terminal.
3. > cd ~/Downloads/pciutils-master. > sudo make install. ...
4. Open a new terminal window (so you get the new command path):
5. > sudo update-pciids. ...
6. Restart the computer.
7. Find port:
8. > sudo lspci —nnvt.

The link goes to a PDF file posted somewhere on Macrumors. The document says it is taken from joevt, from post #212 of the old Amfeltec x16 discussion thread.

Simplified How-to

Putting the information together, here are my simplified instructions for setting link speed.

Download and decompress pciutils
Open terminal.

• cd ~/Downloads/pciutils-master

This command assumes you downloaded the latest version of pciutils from Github. I downloaded the latest release from the pciutils home page and ended up with a folder named /Downloads/pciutils-3.7.0.

• sudo make install

The make command launched the installation of X Code command line developer tools. (I was running a fresh install of High Sierra.) After it was finished, I had to issue the command again. The make command creates three executable files named setpci, lspci, and update-pciids in /usr/local/sbin. The folder /usr/local/sbin is regularly used in Unix installations, but is not on the path in macOS.

• sudo echo ‘/usr/local/sbin’ >> /etc/paths

This command can be skipped. For me it failed with "Permission denied". This may be because I have SIP enabled. Instead I navigated to /usr/local/sbin and gave the last command with a relative path as ./...

If we are trying to set the speed for Slot 2 on a Mac Pro 3,1 we do not need to find the PCI path, as joevt has alreary provided the exact command.

# Set Target Link Speed of MacPro3,1 slot 2 root port to 5GT/s
sudo setpci -s 00:1 CAP_EXP+30.w=2:F

(I gave the command as sudo ./setpci -s 00:1 CAP_EXP+30.w=2:F after navigating to the /sbin folder. You could also give the absolute path as /usr/local/sbin/setpci.)

Setting link speed in EFI

It would be better if the link speed could be set by EFI before any operating system starts. joevt has a script that sets the speed in EFI shell. The downside here is that the startup script startup.nsh must be invoked. Could this instead be done by loading a EFI driver at startup? I have not checked if OpenCore has a solution to this.

Setting PCIe 2.0 speed at boot for PCIe 3.0 cards:
I'm using dosdude1's APFS Patch which consists of booting an EFI Shell and running a startup.nsh script #6,938 . I added some PCIe commands to force PCIe 2.0 speed for slot 2. The start of the script looks like this...

The relevant lines:

echo "Enabling PCIe 2.0 for slot 2..."
mm 0000010009C 42 -w 1 -PCIE -n
mm 0000010007C 20 -w 1 -PCIE -n

The mm EFI shell command "Displays or modifies MEM/MMIO/IO/PCI/PCIE address space." It is defined on page 179 of Revision 2.2 of the UEFI Shell Specification.

The usage is given as:
mm address [value] [-w 1|2|4|8] [-MEM | -PMEM | -MMIO | -IO | -PCI | -PCIE] [-n]

The option -PCIE is specified as
PCI Express Configuration Space. The address will have the format ssssbbddffrrr, where ssss = Segment, bb = Bus, dd = Device, ff = Function and rrr = Register.

joevt's first mm command sets the value 0x42 to the byte at
Segment = 0x00,
Bus = 0x00,
Device = 0x01,
Function = 0x00,
Register = 0x09C

(Or did I misalign something? The address in the command has 11 figures while the -PCIE option expects 13.)

 

[joevt]

I am testing a dual M.2 to PCIe adapter card, so I finally needed to figure out how to set Slot 2 of my Mac Pro 3,1 to PCIe 2.0 speeds of 5 GT/s.

Because PCIe 3.0 and PCIe 4.0 devices initialize as PCIe 1.0 on the MacPro3,1. MacPro5,1 have a fix in their latest firmware. MacPro2,1 doesn't have PCIe 2.0 slots.

To set the link speed in macOS you have to use the pciutils. The package is written between 1997 and 2020 by Martin Mares at ucw.cz. The current version of pciutils is available here:

The PCI Utilities

mj.ucw.cz

There is a also a repository on Github:

GitHub - pciutils/pciutils: The PCI Utilities

The PCI Utilities. Contribute to pciutils/pciutils development by creating an account on GitHub.

github.com

I made my own version at https://github.com/joevt/pciutils with some changes/fixes/improvements. I need to update it with the latest pciutils changes but it's good enough for macOS use. There's a script at https://gist.github.com/joevt/e3cd4ff08aae06279134969c98ca3ab7 which lists all the PCIe devices in a hierarchy. It has instructions for installing pciutils.

lnkspd uses pciutils. It also uses a kext named directhw.

DirectHW.kext is not needed with versions of macOS Sierra (10.12) that support the ACPIPlatformExpert.kext access method (known as the Darwin access method in pciutils). DirectHW.kext is used by pciutils in macOS for the intel-conf1 and intel-conf2 access methods (in my version of pciutils at least). intel-conf2 rarely works on any PC. intel-conf1 can only access PCI registers 00-FF. It cannot access PCI Express registers (100-FFF) without modification. DirectHW.kext is required for macOS El Capitan (10.11) or earlier.

The /Library/Apple/System/Library/Extensions/AppleKextExcludeList.kext/Contents/Info.plist file excludes DirectHW.kext if it's bundle identifier is com.coresystems.driver.DirectHW so you may need to edit the Info.plist to change that.

For M1 Macs, a new DirectHW.kext is needed, unless another alternative to ACPIPlatformExport.kext exists (M1 Macs don't use ACPI so they don't use the ACPIPlatformExpert.kext).

• sudo echo ‘/usr/local/sbin’ >> /etc/paths

This command can be skipped. For me it failed with "Permission denied". This may be because I have SIP enabled. Instead I navigated to /usr/local/sbin and gave the last command with a relative path as ./...

I think it fails because >> is outside sudo echo?

How to run a command that involves redirecting or piping with sudo?

I am trying to follow what I assume is best practices of using sudo instead of root account. I am running a simple concat file operation such as: sudo echo 'clock_hctosys="YES"' >> /etc/c...

unix.stackexchange.com

Maybe try grep -q /usr/local/sbin /etc/paths || sudo sed -e $'1i\\\n/usr/local/sbin\\\n' -i "" /etc/paths
This command adds /usr/local/sbin as the highest priority path if it is not included already.

If we are trying to set the speed for Slot 2 on a Mac Pro 3,1 we do not need to find the PCI path, as joevt has alreary provided the exact command.

# Set Target Link Speed of MacPro3,1 slot 2 root port to 5GT/s
sudo setpci -s 00:1 CAP_EXP+30.w=2:F

Always double check the PCI address just to be sure (use the pcitree.sh script). Slot 2 usually doesn't change in the MacPro3,1 since it's the first device. You show the command to set the target link rate but the command to start the retrain is missing. Scroll down further to #212
sudo setpci -s 00:1 CAP_EXP+10.w=20:20

You can create commands like this:

pcispeedget () {
    local pcidevice=$1
    LinkRegisters=$(sudo setpci -s $pcidevice CAP_EXP+12.w CAP_EXP+c.l 2> /dev/null)
    if [[ -n $LinkRegisters ]]; then
        LinkStatusRegister=${LinkRegisters:0:4}
        LinkWidth=$((0x$LinkStatusRegister >> 4 & 31))
        if [[ $LinkWidth -ge 0 ]]; then
            LinkSpeed=$((0x$LinkStatusRegister & 15))
            LinkCapabilitiesRegister=${LinkRegisters:5:8}
            MaxLinkWidth=$((0x$LinkCapabilitiesRegister >> 4 & 31))
            MaxLinkSpeed=$((0x$LinkCapabilitiesRegister & 15))
            echo "g${MaxLinkSpeed}x${MaxLinkWidth} > g${LinkSpeed}x${LinkWidth}" | perl -pe 's/(g.x..?) > \g1/\1/'
        fi
    fi
}

pcispeedset () {
    local speed=$1
    local rootport=$2
    printf "Before: "; pcispeedget $rootport
    sudo setpci -s $rootport CAP_EXP+30.w=$speed:F # Set Link Speed
    sudo setpci -s $rootport CAP_EXP+10.w=20:20 # Start Retrain
    printf " After: "; pcispeedget $rootport
}

Then execute it like this pcispeedset 2 00:01.0

It would be better if the link speed could be set by EFI before any operating system starts. joevt has a script that sets the speed in EFI shell. The downside here is that the startup script startup.nsh must be invoked. Could this instead be done by loading a EFI driver at startup? I have not checked if OpenCore has a solution to this.

I don't think OpenCore has config option for writing PCIe registers. I think the best thing to do would be to write a EFI driver that looks for devices that are configured with a lower than expected PCIe link rate and then try to retrain them. This way, the fix can be used for macOS, Windows, and Linux.

The relevant lines:

echo "Enabling PCIe 2.0 for slot 2..."
mm 0000010009C 42 -w 1 -PCIE -n
mm 0000010007C 20 -w 1 -PCIE -n

The mm EFI shell command "Displays or modifies MEM/MMIO/IO/PCI/PCIE address space." It is defined on page 179 of Revision 2.2 of the UEFI Shell Specification.

The usage is given as:
mm address [value] [-w 1|2|4|8] [-MEM | -PMEM | -MMIO | -IO | -PCI | -PCIE] [-n]

The option -PCIE is specified as
PCI Express Configuration Space. The address will have the format ssssbbddffrrr, where ssss = Segment, bb = Bus, dd = Device, ff = Function and rrr = Register.

joevt's first mm command sets the value 0x42 to the byte at
Segment = 0x00,
Bus = 0x00,
Device = 0x01,
Function = 0x00,
Register = 0x09C

(Or did I misalign something? The address in the command has 11 figures while the -PCIE option expects 13.)

The first parameter to mm is a hex number, so you can remove the leading zeros and it will mean the same thing.

The first command sets the target link rate (2 = 5 GT/s). The mm command assumes the PCI Express Capabilities is at 6C and that the high four bits should be 4. This works for the MacPro3,1 but might not be correct for other Macs. The setpci command finds the PCI Express Capabilities (CAP_EXP), and uses a mask (:F) so that only the low 4 bits that we want to change will change.

The second mm commands starts the link retraining. The mm command assumes the PCI Express Capabilities is at 6C and that all the bits should be 0 except the retrain bit. The corresponding setpci command uses CAP_EXP to find the PCI Express Capabilities and a mask of 20 so that only bit 6 is altered.

The blog at https://rsend.wordpress.com/x210-coreboot/adding-a-thunderbolt-3-port-to-an-x210-laptop/ has a EFI script that loops through multiple PCI addresses to find a particular type of device by checking the value of a PCI register. I don't think you can do math on the PCI register values though (except with a long list of if ... then commands).

Look up PCIe registers in the PCI Express 3.0 spec. The lspci command can list some of them but doesn't show their addresses (except the start of each capability)
sudo lspci -s 00:01.0 -vv

 

[joevt]

Simplified instructions for setting Driver#### at
https://forums.macrumors.com/thread...upgrade-guide-questions.2194878/post-28956730
Update gfxutil.sh to use the new command syntax.

 

[Thrashaxe]

Hi guys, my first post here, hope I am asking about this in the right thread. Has anyone had any success to get cMP 3,1 (early 2008) with El Cap to recognize a basic NVMe adaptor with Plextor M9PeG NVMe drive in it without any kext patching (e.g. as described on Rehab_Man's github -> https://github.com/RehabMan/patch-nvme). I have my bootrom patched with NVMe.ffs and APFS dxe (via dosdude1's APFS Rom Patcher). I installed High Sierra on another SSD and it sees the adaptor (the El Cap does not see) but still does not recognize the Plextor M9PeG NVMe drive in it. Would appreciate any help as I was slamming my head against the wall for 3 days in a row now with no success .

 

[joevt]

Hi guys, my first post here, hope I am asking about this in the right thread. Has anyone had any success to get cMP 3,1 (early 2008) with El Cap to recognize a basic NVMe adaptor with Plextor M9PeG NVMe drive in it without any kext patching (e.g. as described on Rehab_Man's github -> https://github.com/RehabMan/patch-nvme). I have my bootrom patched with NVMe.ffs and APFS dxe (via dosdude1's APFS Rom Patcher). I installed High Sierra on another SSD and it sees the adaptor (the El Cap does not see) but still does not recognize the Plextor M9PeG NVMe drive in it. Would appreciate any help as I was slamming my head against the wall for 3 days in a row now with no success .

First thing you need is a macOS version that can use the Plextor. High Sierra should be sufficient. Do you not have Mojave or Catalina? If you can't do that, then it's not going to work with El Capitan. El Capitan is probably going to require the kext patch unless there's a firmware patch that can make the Plextor behave like an Apple NVMe (use 4K blocks).
https://www.insanelymac.com/forum/t...s-ready/page/40/?tab=comments#comment-2377304

Actually the rehabman page says there's an extra patch for Plextor (I guess you need this if you can't enable 4K?):
https://www.insanelymac.com/forum/t...erra-is-ready/?do=findComment&comment=2356251
https://www.insanelymac.com/forum/t...s-ready/page/54/?tab=comments#comment-2617639
https://www.insanelymac.com/forum/t...erra-is-ready/?do=findComment&comment=2619212

The patch is included in rehabman's config_patches.plist so I don't think you have to do anything manually?

 

[Thrashaxe]

Hey joevt, thanks a lot for your response. Not sure what I am missing but I cannot figure out how to apply the config_patches.plist.

I have done this already:

1) mkdir ~/Projects && cd Projects
git clone https://github.com/RehabMan/patch-nvme.git patch-nvme.git
cd patch-nvme.git

2) ./patch_nvme.sh
(not sure if I need to run ./patch_nvme.sh --spoof)

3) how do I run the config_patches.plist ?

Thanks a lot again for your help!

 

[joevt]

Hey joevt, thanks a lot for your response. Not sure what I am missing but I cannot figure out how to apply the config_patches.plist.

I have done this already:

1) mkdir ~/Projects && cd Projects
git clone https://github.com/RehabMan/patch-nvme.git patch-nvme.git
cd patch-nvme.git

2) ./patch_nvme.sh
(not sure if I need to run ./patch_nvme.sh --spoof)

3) how do I run the config_patches.plist ?

Thanks a lot again for your help!

You don't want --spoof unless you're mixing NVMe's of different types. --spoof requires an SSDT to modify the PCIe device class of the NVMe.

You could use OpenCore to do the patches - translate manually the Clover style patches that you want from config_patches.plist to the OpenCore config.plist file.

Or do the following:

First, try the default patches:
./patch_nvme.sh
Note the version where it says Determined patch automatically from vanilla IONVMeFamily:
In my case, it says 10_11_6_15G22010

It creates the kext. Install it and try it. It should allow NVMe devices that use 512 byte block size to work.

If it doesn't work then maybe you need to include one of the patches in config_patches.plist:

Create a copy of the patches file (NVMe_patches_10_11_6_15G22010.plist in my case), name it something like NVMe_patches_10_11_6_15G22010_Plextor.plist (no spaces), copy the patches that you want from config_patches.plist (just the Plextor patch?) and paste them into your new patches file. Then run the patch command like this:
./patch_nvme.sh 10_11_6_15G22010_Plextor
If you're running the script from a different version of macOS than the one you want to modify, include the path of the nvme kext that you want to patch:
./patch_nvme.sh --unpatched /Volumes/ElCapitan/System/Library/Extensions/IONVMeFamily.kext 10_11_6_15G22010_Plextor

Install the patched kext and try it.

Actually, it may be that the plextor bytes to be patched don't exist in the IONVMeFamily.kext.
I changed quiet=1 to quiet=0 in the patch_nvme.sh script to see which patches were applied to the patched kext. The plextor patch is not applied in the case of 10_11_6_15G22010. That could mean the plextor patch was made for a different macOS version and needs to be modified by macOS.
So what you need to do is read the threads that were linked to find the correct patch.

 

[Thrashaxe]

Thanks a lot! It is much clearer now. I tried something very similar earlier today to what you suggested but might need to try it on a clean install agian tomorrow. NVMe_patches_10_11_6_15G22010.plist is also my case.

I copied the "Plextor/LiteOn/Hynix" part from config_patches.plist :

<dict>
<key>Comment</key>
<string>IONVMeFamily: Ignore FLBAS bit:4 being set - for Plextor/LiteOn/Hynix (credit Pene)</string>
<key>Name</key>
<string>com.apple.iokit.IONVMeFamily</string>
<key>Find</key>
<data>ikga9sEQ</data>
<key>Replace</key>
<data>ikga9sEA</data>
</dict>

and then ran ./patch_nvme.sh 10_11_6_15G22010

It did not make my plextor show up and neither did the controller. I also have High Sierra installed on a different drive and when I boot into it I can see the controller but not the plextor though (I guess this is due to the bootrom patch I made with nvme.ffs earlier this week). I'd like to stick to El Cap for now so I will keep on trying out more options. I will check if my vanilla IONVMEFamily.kext has the string the patch aims to Find/Replace. If it is not there it does not find it and the replace does not happen.

I also tried an updated Find/Replace part with no success.

<key>Find</key>
<data>SBr2wRAPhQ==</data>
<key>Replace</key>
<data>SBr2wQAPhQ==</data>

I recall I read on some thread that if --spoof switch is not used when patching, IONVMEFamily.kext should be removed from S/L/E after moving the newly generated HackrNVMeFamily-*.kext to S/L/E. When I did it I had trouble logging back into my El Cap system. Eventually it worked after I put the original IONVMEFamily.kex back into S/L/E and run KextUtility to fix permissions and rebuild caches and it get backs to normal after 3-4 reboots.

I am not considering a version higher than High Sierra for my Mac Pro 3,1 since I don't have the Metal-capable graphics card.

How exactly do you check which patches were applied to the patched kext by changing the quiet=1 to quiet=0 in the patch_nvme.sh script?

Is there also a place anywhere in the HackrNVMeFamily-*.kext to provide the hardware id of my NVME pcie controller? I am wondering if this would also be needed since in El Cap in the System Profiler it sasys that driver is not installed but in High Sierra it says Yes next to "driver installed:".

Thanks again so much for your help!

 

[joevt]

How exactly do you check which patches were applied to the patched kext by changing the quiet=1 to quiet=0 in the patch_nvme.sh script?

With quiet=0, it says "patching offset:" for each applied patch. If a patch is not applied, then it doesn't show the offset.

Is there also a place anywhere in the HackrNVMeFamily-*.kext to provide the hardware id of my NVME pcie controller? I am wondering if this would also be needed since in El Cap in the System Profiler it sasys that driver is not installed but in High Sierra it says Yes next to "driver installed:".

The info.plist has the matching info. The hacked kext matches by PCI class ID. It should match the class ID of your NVMe.

 

[Thrashaxe]

Thanks, man! I think I have more chances on High Sierra since it sees my PCI controller card and says the driver is installed. Is my interpretation of the below correct?

10.13 High Sierra​

With 10.13, Apple has fixed their IONVMeFamily.kext and now it supports 512 byte block sizes natively. This means for many NVMe SSDs, you do not need these patches. Still, the special patches for LiteOn/Plextor/Hynix SSDs may be necessary (see above).

Does this mean I can apply the below patch only? If yes, should I make a custom patch plist by just copying one of the available plists (e.g. for 10_11 or 10_12), then removing all patches and copy/paste just this one for Plextor/LiteOn/Hynix and run the patch command with --override option to suppress the MD5 checksum validation?

./patch_nvme.sh --override my_custom_10_13_plextor

<dict>
<key>Comment</key>
<string>IONVMeFamily: Ignore FLBAS bit:4 being set - for Plextor/LiteOn/Hynix (credit Pene)</string>
<key>Name</key>
<string>com.apple.iokit.IONVMeFamily</string>
<key>Find</key>
<data>ikga9sEQ</data>
<key>Replace</key>
<data>ikga9sEA</data>
</dict>

I guess this would create the HackrIONVME-my_custom_10_13_plextor.kext file (based on the IONVMEFamily.kext in my High Sierra S/L/E folder), will save it in the patches.git folder, then I would have to remove IONVMEFamily.kext from S/L/E and use KextUtility to add the HackrIONVME-my_custom_10_13_plextor.kext to it. Does it make sense or am I missing something? Should I also copy the HackrIONVME-my_custom_10_13_plextor.kext to L/E?

If the approach is correct but it ends up not working, then I can try other plextor patches reported by users on forums as working for Plextor M9PeG NVME and try my luck with those. It is a bit too tricky for me to go with Linux and attempting to change block size to 4K but if none of the above works, that would be my next thing to test. Please let me know what you think about this plan of mine

 

[joevt]

Should I also copy the HackrIONVME-my_custom_10_13_plextor.kext to L/E?

Just copy it to one of the folders.

 

[Thrashaxe]

Hey! Got it working on High Sierra with this patch from Pene.

Patch for using NVMe under macOS Sierra is ready.

www.insanelymac.com

<dict>
<key>Comment</key>
<string>IONVMeFamily: Ignore FLBAS bit:4 being set - for Plextor/LiteOn/Hynix</string>
<key>Disabled</key>
<false/>
<key>Name</key>
<string>IONVMeFamily</string>
<key>Find</key>
<data>SBr2wRAPhQ==</data>
<key>Replace</key>
<data>SBr2wQAPhQ==</data>
</dict>

I followed the steps in me previous post to create the custom plist and then generated the Hackr kext by runnung ./patch_nvme.sh --override my_custom_10_13_plextor

Plextor showed up as external. I cloned my working instance of 10.13 to Plextor but it loads much slower then the source. Will play with that a bit later.

Wondering if this works on El Cap.

Thanks for all your help so far!

 

[borp99]

Hi all,

Can anyone upload a working 'apfs(dxe).ffs' file which I can inject into my cMP3,1 ROM?

I've tried the one inside DosDude1's 'APFS ROM Patcher' and it just refuses to work with DXEInject for me. I can inject any NVMe.ffs file and it works fine to rebuild a new ROM - so I know I've got my directory and syntax correct, but not for apfs.

The suggestion at the beginning of this forum that there is an 'apfs.ffs' to extract from the Mojave installer firmware file [i.e. MP51.fd], alludes me. It has the ApfsJumpStartDxe file listed to extract but I can't find the other one anywhere.

Other posters have mentioned that they got apfs.ffs injected into their 3,1 ROM - whether it's actually needed anymore (that's another matter...).

However, I think I need to get apfs.ffs injected at firmware level in order to do Big Sur updates from full installers (like I can on my MP5,1) on my 3,1 - which won't presently work, as the installer keeps insisting that I need a firmware update to write to an APFS volume (so therefore the inclusion of just the ApfsJumpStart code in my current ROM isn't enough for this to work from Recovery). Conversely, you can't overwrite BigSur full updates under a normal boot because the installer insists that the target volumes is HFS+ and there's no point to erasing your already-setup OS (...you 'could' use CCC to back up your Data volume and restore it after a full erase and fresh install of the newer BS version but it would be so much easier to natively run the overwrite/update from Recovery). I've tried to get OC running on my 3,1 for BigSur OTA updates but OC crashes back to rEFInd each time, no matter how I configure it.

 

[freqrider]

Just use the APFS patcher by dosdude. All u need.

 

[borp99]

Just use the APFS patcher by dosdude. All u need.

That doesn't work for me. It stops part way through with 'an unexpected error' message popping up. I tried running the patcher on 10.12, 10.15 and 11.2. That's why I would like the raw apfs.ffs file to inject myself.

 

[freqrider]

Did you put the mac in program mode by holding the power button down from a cold start until you hear the strange tone? Then run the patcher?