I think there are two issues that get conflated with non nexus devices:
Manufacturer's add-ons and carrier add-ons
I always buy my devices sim-free to avoid any possibility of carrier bloat and don't have a problem with my S4 out of the box. Yeah, there are some features I don't really use (such as the eye scrolling), but I just switch them off and they don't really get in the way. IMO, the big problem comes when carriers get their hands on the devices and start adding stuff: Not only does it introduce another delay in getting updates, they can really mess things up.
The last time I bought a subsidised phone was a Windows CE device from Orange: they removed MSN messenger to try and force people to use SMS (this is in the days before unlimited SMS and it was a pretty good money spinner for them). After that, I decided I'd rather pay full price for my phones and get them as the manufacturer intended!
I've got an S4 and a Nexus 7 and comparing the two, I'd say the Samsung modified android experience is better in the following areas:
1) Email - the stock android email is far better than it used to be, but so is Samsung's email program on the S4 and I really like it now
2) Camera - The S4 adds a lot of functionality to the camera over stock Android
3) Quick toggles - yeah, they can be a bit overwhelming, but I really like the way Samsung have implemented them
4) Contacts - nothing wrong with the stock android People app, but I prefer Samsung's implementation
Things I think Stock android does better
1) Keyboard (a bit of a moot point as it can be easily changed and I have the Google keyboard on my S4) - I much prefer the stock Google keyboard to Samsung's, not least for the auto-correct
2) Folder handling - Touchwiz has too much animation when opening folders which slows things down for folders with a lot of stuff in them. Also, creating folders is more intuitive on stock
3) Lock screen - The fact you can't add widgets to a pin locked screen on the S4 is very frustrating!
I also think Touchwiz adds some useful features to stock android such as blocking mode, smart rotate, S voice, etc.