- Mac Pro 1.1 can't run above 10.11.x so Metal shouldn't be a concern
- Seeing as Mojave isn't possible, there's no reason to worry about the NVidia Web Driver situation
- The RX 580s aren't compatible thanks to the lack of the SSE 4.2 instruction set
- The 1000 series NVidia cards are compatible with 10.12 and above (I don't believe they added support to older versions of the OS) so that nixes modern GPUs.
I'd probably score a used NVidia GTX 780 as those are around $100 on ebay these days and plenty performant if you don't mind a non-EFI card. No reason to get anything less. You could also go for the 970 or 980.