Happy New Year, Radeon VII users!
With grateful thanks for support from the OCLP hardware fund, I now have a Radeon VII and I think I’ve got a solution to the Radeon VII GOP problems.
Additional research into PCI IO cache settings and some feedback from Vit and Martin Lo has convinced me that this is really looking like a bug in the Radeon VII's GOP implementation, not a bug in OpenCore.
Then what? AMD Linux drivers are OpenSource, so it might be possible to find some info there, or it might be possible to reverse-engineer the GOP driver itself, to try to work out what setting AMD haven't set that they should have. (It still looks like the video memory cache settings are wrong, but it no longer looks like this is something fixable at the UEFI level.)
But before that, maybe there is a chance that there is another Vega 20 GOP implementation out there which already works? Apart from the different Radeon VII BIOS versions available, there is one unverified Radeon VII Pro BIOS available (at least, only one that I've seen), and it is a few months newer than the non-Pro BIOSes. So first things first, maybe it's worth trying to combine the UEFI GOP from the Radeon VII Pro BIOS with the legacy BIOS section of the non-Pro BIOS, and … well, it just works! With OpenCore and EnableGop. DirectGopRendering and EnableGopDirect are even no longer needed.
I’m not quite sure what AMD were playing at here - it at least looks very much like they knew something else needed fixing, but only fixed it in the Pro GOP version - but hey, finally, here is what it seems they should have released all along.
You can just flash this to your card with
amdvbflash
, as normal, and then add EnableGop to your machine's firmware as per the instructions on p1. You no longer need DirectGopRendering in OpenCore.
---
@ZNDK - Based on these results I think your best bet for your similar problem is to try upgrading your AMD RX Vega 56 video BIOS - especially since your card already seems to have worked for other users. Failing that, you could possibly try the same trick of taking the GOP from the Pro version of your card and using it to replace the GOP which you currently have.
NB This is the BIOS from my PowerColor Radeon VII, with just the GOP replaced, as described. Received wisdom is that all these cards are the same, so in theory this should work on any Radeon VII, but backup your video BIOS first, and worst case scenario be prepared to reflash your card with a hardware clip. Success or failure results welcome.