Results time!
ATI = ATI Radeon 5770 Mac (stock apple config card)
NVIDIA = EVGA NVIDIA GTX 670
Tossed the 670 into the mac pro, booted as expected (black screen until login phase, if you auto login you just see your desktop) in Mountain Lion (10.8.2). Overall the OS and apps feel ever so slightly smoother. Just what I was hoping for out of more vram and a faster GPU. Nothing big, but it wasn't "bad" before, just a little bit off from lower resolution screens in regards to dragging windows etc.
Ran cinebench with the following results:
ATI: 28.56fps
NVIDIA: 25.42fps
Confusing... drop in frame rate...
Updated the nvidia drivers to the latest from NVIDIA rather than the OS embedded ones. new Cinebench result: 27.53fps
This is very odd. But Cinebench is a purely bench test and has very little to do with the real world. Moving on.
Civilization 5: Don't have an accurate way to get FPS here, so just subjective assessment. nvidia seemed a TINY bit faster in low frame rate situations. maybe 18fps when the ATI was 15fps... nothing appreciable. Disappointed since I've been playing a lot of civ 5 recently. But civ 5 is a sloppy port, of a poorly written game from windows. Further it seems ported from the DX11 version of the graphics code which is noted all over to be WAY slower than when running under DX9... I chalk this up to a poorly written game, with a more poorly written graphics engine being poorly ported to the poor mac. Still very disappointing.
Team Fortress 2: Yay steam works! I heard there might be problems but that seems to not be the case.
ATI Game Play: ~90-120fps (quite variable)
ATI Spectating: ~50fps (again variable +/- 10fps)
NVIDIA Game Play: ~230-270fps (not nearly as variable)
NVIDIA Spectating: ~73fps (hardly variable +/- 5fps)
Here we go, this is a real difference. Granted it wasn't really pushing the ATI that hard, but we can at least get a number here that means there's an improvement with new hardware.
Diablo 3: I didn't really play so much as just log in an poke around camp and town in act 3.
ATI Camp: 37
ATI Town: 33
ATI Town (AA Enabled): 30
NVIDIA Camp: 120
NVIDIA Town: 115
NVIDIA Town: 98 (AA Enabled)
Remembered to try AA on/off directly on this one. NVIDIA always about 3x faster. Again, this wasn't real game play with physics etc. that slow down the graphics but with that sort of head room, in a game that was playable but not great before we have a real reason to upgrade here.
Bootcamp:
Rebooting to bootcamp just brought up a black screen. I heard windows log in so I know it was running back there, but without actual nvidia drivers for the specific card you get nothing. That's a problem. I tried throwing my bootcamp drive into a PC to install some generic drivers, but that just destroyed the drive (it's sketchy, thats what I use it for bootcamp). So I had to re-install windows. Had to put the ATI in to do that. I then swapped it into the PC again to try installing some nvidia drivers. No dice, screen still black. I tried putting the NVIDIA into the PC to install the right drivers but that motherboard didn't like it (wouldn't boot) and the other PC I have around for such things is too risky to put new hardware into (too many power surges in it's life). So I couldn't cheat, I had to install the drivers on the mac. But the mac only has 2 6 pin power plugs and it would take 3 to run the ATI and NVIDIA at the same time... so I borrowed a 6 pin line from one of the PCs. For the record I had the ATI in slot 1 and the NVIDIA in slot 2 when installing drivers. I installed the NVIDIA, then the bootcamp driver package, then restarted and yay windows visible on the NVIDIA. I removed the helper computer, swapped the NVIDIA back into slot 1, removed the ATI, booted back to windows and success. Lesson: don't try to cheat, just find the extra power for setup, configure how you want after.
a little windows testing
Tribes Ascend:
ATI Menus: 27fps
ATI Game: 22-30fps (note, I usually turn things a bit lower when playing tribes since the frame rate is SO critical)
NVIDIA Menu: 60fps
NVIDIA Game: 60fps
NVIDIA Game Maxed everything: 45-60fps
I had vsync off but I suspect the adaptive vsync or whatever it's called of the nvidia that helps prevent tearing was acting as a backup vsync. I've seen the ATI go above 70 with everything low low, so there was clearly something capping the nvidia, But that's fine since 60 is the refresh of the screen anyway. The nice thing is that with totally maxed everything it was still smoother and more consistent than the ATI at my normal play settings. Reigning things in a little gives a rock solid 60fps no matter what which is really as good as it can get in tribes. VERY happy.
Torchlight 2: I forgot to get ATI readings but I've never had any major complaints, I ran a few things less than max and occasionally would see frame rate dips but nothing that was too bad.
NVIDIA with AA: 75fps
NVIDIA with 3 monitor surround (1080p x3): 75fps
I forgot to order a display port cable so to run 3 screens currently I have to use HDMI to DVI which can't run full WQHD resolution. but it was interesting to see that if I drop AA, and drop the resolution a little I can get the same performance in surround as in single display. I need to dig up Dirt 2 or something else that really benefits from surround, I expect much fun.
A few other notes
It runs 3 screens totally fine. As noted I forgot a display port cable (thought I had one, I was wrong) so I can only run 2 at WQHD and one at 1920x1080 for the moment. I don't expect any issues running all 3 at 2560x1440, and I expect that with AA off it will even be playable in surround at native resolution (and with that resolution AA really makes no difference to me at all, the pixels are small enough). I suspect it should run 4 displays just like it says on the box without issue (3 max for gaming at a time, 4th can still display desktop level stuff, web browser, etc.) and will test that out when cabling permits.
So there you are. Hope someone finds something in there useful. The biggest hassle really was getting it working in bootcamp. Just don't try to cheat the system (borrow power for setup) and it's smooth sailing. The only things that didn't improve have excuses (artificial test and poorly written graphics). Another bonus is all those CUDA cores when I get around to crunching with something compatible.