This is a crazy long thread, but it might be a good place to start:
iMac 2009 to 2011 MXM3 graphics card upgrade (Lastest update 10/13/2023 - this is the official documentation page - you found it finally!) This is the single and only documentation post. During the last years we constantly updated this post to reflect the lastet development of vBIOS and OCLP...
forums.macrumors.com
Depending on your skill level, you could put a beefy GPU in these Macs.
Some GPUs are plug and play, while others require modifications, flashing, etc.
IIRC, your iMac has the two-pipe GPU heatsink, which only works for some of the compatible GPUs. You could probably get a three-pipe heatsink, but between that and the GPU, it would probably be over your budget.
BTW, last Spring, I attempted to put a K4100 in a mid 2011 iMac, modded it, installed it, booted into Linux and successfully flashed it, put it all back together, only to have something go wrong and now it won't even get a full chime when attempting to start.
This was all when I bought a new house and was pretty busy with the move, basically not having any time to troubleshoot it. I plan on doing that after the holidays.
No matter what you do, don't get rid of this Mac!
This is one of the few Macs that Target Display Mode works via the Mini DisplayPort, meaning, you could use the iMac as a monitor for almost any device that has DisplayPort out, including the new M1 Macs.