I ended up rebooting, but I have a theory.
I keep most of my media on an external Drobo USB drive array, including my photo library. I noticed that while it was hanging, the external drive was very active. My Mac has solid state drives, so I can't tell if the internal drive was active but that external drive was moving quite a bit.
Anyhow, about an hour after I rebooted, I decided to launch the new Photos App to see what it was like and the system started to update my library... and about 5 hours later, it's still updating it... and once again, I can hear the drive do it's thing.
My media libraries are HUGE... tens of thousands of photos and videos. I think it's possible Apple started re-organizing the media in the installer making the assumption that the time spent organizing media is generally short.
If I'm right, I'm guessing many of the people who experienced the same problem also have large media libraries.