I had an odd issue immediately after upgrading my 2009 MP (with GTX 970) today where I could not initially log in to my user account.
1. I entered the sudo nvram boot-args="nv_disable=1" command in Terminal and ran the installer. Mac Pro restarted.
2. Install took 20-30 minutes. Screen was dark the entire time, as expected.
3. The Mac Pro appeared in my MacBook Air's Finder. I initiated screen sharing and could see the login screen with my user and guest accounts.
4. I attempted to log in to my user account. However, after I entered my password, instead of the desktop or setup screens, I was sent back to the main login screen with the user and guest account options. Repeated several times, same result.
5. I logged into the guest account and went to the desktop. From there I was able to (a) install the NVIDIA web drivers and (b) reset boot-args. (I had to su to my user account, then entered the sudo nvram boot-args="nvda_drv=1" command).
6. I rebooted as part of the web driver install. Upon reboot my main display was active. I was able to successfully log in to my user account and finish the install (logging into iCloud, accepting terms and conditions, etc).
Moral of the story: make sure that the guest account is turned on before upgrading.