I do have an issue with the NVIDIA GeForce GT 640 2 GB card when rebooting or turning on. I do have to confirm though that 3 displays are working perfectly fine if efi decides to load properly. When I do a regular restart, or a regular turn on, for some reason the graphics card is not loading. I can remote in from another computer and see that the system booted, but no graphics, screens are black. If I do a hard reset (holding the power until turns off and and turn on) the card boots.
I have exactly the same problem. Instead of the hard reset via the power button I use the remote login to reboot the system. Upon the second reboot the GT640 is usually properly recognized.
I read on MacVidCards that there is a flashing service available for that card.