My issue is a bit different. After upgrading to iOS9, when upgrading certain apps, it will stuck at some point during "installing". Then the app can't be deleted, the app store will show the app is no longer installed (showing the cloud icon). But when you try to redownload the app, it will stuck at the same point.
The workaround I have now is to reboot the phone, so that the stuck app is gone from HOME screen. Then redownload it (weirdly enough at this point App Store showing this app as an empty progress ring, not the cloud). Then when downloading, no progress or app icon can be found on the home screen. But when it's done, the app is back and back to its position and no data was lost (as if it was just upgraded normally.
This has happened to me only after I upgraded to iOS9 on my iPhone 6. And only for certain apps. It is really annoying though.