OBS - error with the encoder

[rm5]

Hey,

I'm experiencing a strange error with OBS on my 5,1 Mac Pro with the Radeon Pro W5500 graphics card. Every time I try to start an OBS recording/stream with the hardware encoder "Apple VT H.264 Hardware Encoder," it gives the error message in the screenshot below. I'm running Catalina 10.15.7 via the Dosdude patcher, as I was having issues with OpenCore.

Would OpenCore solve this problem, or is it a driver issue that can not be fixed for this particular GPU? The issue happens on both v27.2.4 and v28.0.3. The screenshot below is of 27.2.4.

Any help greatly appreciated!

 

[bogdanw]

WhateverGreen fixes video problems for unsupported hardware https://github.com/acidanthera/WhateverGreen

 

[rm5]

@bogdanw It's not letting me copy the kexts to /System/Library/Extensions, even when logged in as root, which is really weird. Am I doing something wrong there? Also, is that all I need to do or is there further configuration that needs to be done after copying the kexts (if I can figure out how to do that)?

 

[bogdanw]

OpenCore guide and forum https://forums.macrumors.com/threads/opencore-on-the-mac-pro.2207814/
" Copy the files "Lilu.kext", "WhateverGreen.kext" and "FeatureUnlock.kext" to your ESP as indicated below:"