The iMacs that uses Nvdia are late 2012 to late 2013 models and they have the GPU soldered on the motherboard, so only way to have a Nvidia card it's to get one from a PC laptop, download the Nvidia drivers for mac on their website and use it. But you are going to lose the boot screen with the apple logo, when the operative system boots, it's going to work ok.
The bad side is if you want to select another drive when booting or some problems when you are reinstalling the operative system. On that case you should conserve your original video card, and put it when you want to reinstall the operative system.
I have putted an AMD 6970m on my iMac that comes from a pc, and yesterday I had to put the original ATI card because I had some troubles when i was installing High Sierra, nothing really important, but I decided to put the original one meanwhile. Then put it back the PC AMD 6970m.
You can find right now some Nvidia 780M 4GB on ebay for about 340€
I choosed the AMD because I could buy it for only 55€ but I was lucky on the ebay sale.