I have no recommendation right now. Trying a different NVMe drive MIGHT do the trick but I don't really know. I do know that in my case, there seems to be a relationship between the RX580 / 970 to the problem. Pulling the RX580 and installing my HD7970 completely solves the issue. Booting to a different SSD (conventional SSD in drive bay) also solves the issue.
I've been trying to diagnose and fix this for months. I'm at my Witt's end at the moment too.
As a side note...
There are other's here on MR that have the EXACT same configuration that I have, and no booting issues at all. It seems we are in the minority on this...
I've been through 3 Logic Board, 4 GPU's (2x RX580's), several iterations of BootRom, several installs, striped down hardware to bare minimum and re-introduced one at a time. I'm still working on this but may give up on NVMe or the RX580. I haven't decided as of yet.
I was contemplating purchasing a different NVMe/PCIe carrier to see if that made any difference. Since you are using a simple passthrough adapter like I was considering, it seems it would be a waste of time for me. I may purchase a different brand NVMe with a different controller onboard and see if that helps.