The non-metal card is a Sapphire ATI Radeon HD 5770. I'm wondering if the two cards are to be used in conjunction with one another however, that'd take up two PCIe slots
The MSI GEFORCE GTX 1050TI AERO ITX 4G OCV1 GRAPHICS CARD doesn't seem to have an input for a power source from the MacPro? ( 6-pin connector).
The OS is an OC version of Monterey which appears to be stable. I have WAVES plugins installed using Pro Tools and Logic Pro but they appear without any GI (graphic interface).
My apologies: I'm not sure what a 1050Ti is.
Thank you very much for addressing my post
h9826790!
OK, I have better understanding of what's happening now.
My understanding so far.
1) You just bought a cMP 5,1.
2) which comes with a HD5770. Shapphire, not the original Apple card, may be flashed to show boot screen?
3) It also comes with a 1050Ti (MSI GEFORCE GTX
1050TI AERO ITX 4G OCV1 GRAPHICS CARD). The card has no aux power input (which is actually normal for low power cards)
4) Somehow, it shipped with Monterey, a macOS that is not supported by the cMP (and both of the shipped graphic cards). The OS is patched to run with the HD5770.
5) Monterey seems stable, but your WAVES plugins has GI issue.
If the above points are correct, then as Tsialex said, the 1050Ti won't work properly in Monterey due to lack of driver. This is a known issue. If you want to use the 1050Ti, you should format the boot drive, and re-install High Sierra onto it. HOWEVER, please make sure you plan the whole workflow before you start to do that. It is still not confirm your HD5770 can show boot screen. If not, and you formatted the boot drive, it may be quite hard for you to recover from that situation (especially if there is no other Mac for you to create any macOS installation media).
TBH, it's a very non standard (in fact, poor) config to sell cMP. IMO, the seller shouldn't sell a cMP like that (has a graphic card inside which cannot be used in the pre-installed OS). And I bet he hasn't tell you that the pre-installed OS isn't a supported OS, and most likely patched.
From your questions, I believe that your knownledge isn't at the level to handle this kind of config. Of course, you can learn. However, I believe it is not your intention to buy this machine.
The seller is correct about the 1050Ti is a Metal compliant card. However, it's not a supported GPU in any macOS at all, even in High Sierra. We can install web driver from Nvidia to make it work in High Sierra. But "make it work" doesn't means "supported". And TBH, most people looking for Metal supported card because they want to run Mojave or higher macOS. If this is also what you want, then the 1050Ti is completely useless. It can't even display in Monterey. This is completely normal.
Anyway, the possible actions for you
A) return the machine, get the money back, buy another one (with proper software and hardware).
B) buy a proper GPU (e.g. RX580) for your cMP, and remove the unsupported HD5770 and 1050Ti. However, highly likely you still need to go through another OCLP root patching (I believe you have absolutely no idea what I am saying) to make the new GPU work properly.
C) format your existing boot drive, install High Sierra and Nvidia web driver, then remove the HD5770 and use the 1050Ti. Again, I beleive you have no idea what Nvidia web driver is, and the 1050Ti won't display anything at all without it. And we have no idea if the HD5770 can display any boot screen for you to install High Sierra. So, it may not be as easy as you believe.
TBH, if I were you, and I can get the money back, I will go for option A.
But if you already pass the point of no return, and must make it work. Then you better prepare to learn the relationship about different GPU and macOS, and what is OpenCore etc, and buy a proper GPU to go option B.
Option C is the cheapest way (in fact, no extra cost), but you will stuck at a OS that's already outdated. And still need to learn how to make the 1050Ti works.