I had the same experience with my MacPro with an ATI Radeon 5770. I replaced it by the 5870, where the problem didn't occur.
But perhaps you can try to flash your machine's firmware from 4,1 to 5,1.
The ATI Radeon 5770 was fully compatible with the 5,1 computers (2010).
Unfortunately that probably will not help as I have a 2010 running 5,1 firmware and a GTX 680 and it happens to me as well.
One thing I notice is if you constantly just restart (ie. Into Windows then back to OS X) it does not occur. A cold shutdown and boot and the problem will surface.
It seems a bunch of replacement cards are doing this. It isn't just 1 brand or model:
https://forums.macrumors.com/threads/1566964/