I just finished updating to 8.4 from 8.1 and it was definitely not worth the time and headaches. Nothing disastrous, but my battery still sucks, wifi still doesn't work right and now I'm stuck with the new Music app and a few other changes that I don't care for.
My advice if you're on the fence: don't bother!
(iPhone 6+)
Update: after using it for a while, I think updating was a good idea. Although it didn't fix my wifi, battery life now seems better, I'm seeing more free RAM in my RAM Widget, and I think it's a bit "snappier" ;-)