Very smooth. No problems BUT:
Be sure to thoroughly go through all of your settings menus. There are a TON of new menu items in iOS5, and many of them involve functions like geolocation, data sharing, and data backups. Battery and cellular data intensive stuff. You kind of have to do a cost/benefit analysis to figure out what is worth keeping.
Personally I think iMessage, Photostream, and the camera control upgrades are the best part. So much better.
EDIT: And notifications! Oh my god...so much better.