Ok so there is definitely and issue with Nvidia cards with 10.9.3+. Everyone who says it works is using an AMD card. I am using a non flashed GTX 570.
Well.. I've been using GTX 660 just fine with the MacPro4,1 from ~10.7 through 10.9.4. First NVAudio and when that broke in 10.9.2 I found the updated HDMIAudio here. It worked just fine with 10.9.3 and 10.9.4.
But not anymore with the release Yosemite, 10.10 (14A389). I reinstalled just in case but HDMIAudio 1.0 is no longer working for me .