Spontaneous reboots due to poor battery fit was widely publicized during initial launch months.
http://www.digitaltrends.com/mobile/galaxy-note-3-problems/
http://blog.brokenlcds.com/samsung-galaxy-note-3-random-restart-and-boot-loop-problem/
There's lots more, google it if you really think I'm being "over the top."
The remainder of the lag could be Touchwiz since I stayed stock for much of my ownership tenure, although that's another beef I have with Android OS in general. Its very, very difficult to pinpoint issues like this as the non-curated nature of apps in the Play store means you could have downloaded and update or a new app that hoses everything else up entirely and you are effectively on your own as the SysAdmin to figure out what went wrong.
Same for fit/finish, etc. Just because you've been lucky or perhaps just haven't noticed because your perspective is different/limited use cases and/or you are more tolerant of poor design doesn't mean its not true. My phone flexes alarmingly, so much so that the back pops off frequently. It creaks any time i try to adjust the volume. And the home button sticks and is crooked...my original and all the refurb'd replacements I've received exhibited the crooked home button out of the box. On my most recent device (I've had this refurb 3 months...longest time of any of these Note 3's), the chrome has worn down to dull gray plastic showing. I have used a case and am now caseless since the radio is so poor that a case obstructs what little signal I can actually get.
The real issue here is that as a human being you and I both tend to believe that our experiences equals or is of greater relevance than those of others. That's called "bias" my friend.
I freely admit that after the cheap (bias), poor (bias) and plasticy (bias) experiences I've had owning and using my Note 3, I'm done with Samsung. Will I look at the Note 4? Sure, I may even buy one because I like gadgets and enjoy tinkering. But as my primary, day-to-day business productivity tool? No way. Its just a gadget (bias) with overblown specs (bias) that is a sign of a company grasping to hang onto market share by throwing in a lot of useless (bias) feature bloat.
OK that last sentence is an opinion, but a biased one.
This. They focus so much on adding a feature that they forget to bring the rest of the phone along for the ride. As a result you end up with rapid obsolence and a phone that feels very much frankenstein-ish in delivery.