Hi, thought it might be beneficial to share experiences slotting a GTX 960 in a Mac Pro 4,1 late '09.
First, as well-documented here, I had to install Yosemite to get any video out on the 960. Vanilla Yosemite provided video out even prior to installing the Nvidia web drivers, but not at my monitors' optimum resolution. I don't recall what the available resolutions were but they were all higher than the 3x 1080p I wanted; the video was scaled and usable if ugly. With the web drivers installed, 1080p was available. I have not tried installing the web drivers on the extant Mavericks boot volume and do not expect that to succeed but intend to try it for the sake of completeness.
Using Bootcamp to install a Win 7 boot volume has also succeeded, although the process was hairy and took several days of troubleshooting to master. I went through two complete clean installs of Windows before I could get the Nvidia installer to successfully implement the current drivers for the 960. I was able to get the correct drivers (.5330) for the 960 both by using Windows Update and by using the Nvidia installer in the end.
However, there are some major issues that remain unresolved.
First, the stock GT 120 (or GT 9500, the OEM EFI card), *must* be removed prior to booting into Windows or Device Manager will become confused and direct video out ONLY the GT 120. Disabling and/or uninstalling either card in Device Manager does not affect this; Device Manager can report either card as active and the other as disabled and the only video out is the 120. On reboot, one of the two cards will be reported as functioning normally while the other is yellow-triangle halted with, I believe, Code 43. It does not matter which card is shown flagged, there will only be video out from the 120. The only way I have noted to get video out of the 960 is to unseat the 120, which makes booting back and forth between Mac and Windows less than convenient.
Second, the Bootcamp / Win 7 audio issue which is customarily resolved by reinstalling the Realtek drivers under Windows does not successfully restore normal audio operability. I have been able to implement an audio solution by plugging in an external USB audio in/out module. This is actually an acceptable long-term solution for me even though it's inelegant.
Hope this helps others running similar configuration experiments.