Updating my opinion, longtime note 3 and note 4 user who would sometimes switch to an iPhone. I've been using an iPhone 6+ then a 6s+ for the last 6 months or so and just bought a note 5 and waiting for it to be delivered. My main thing with the iphones are 1) they feel so nice, nothing feels as premium IMO, I know it's silly but I just enjoy the feel of it and 2) It "just works", I never have issues with making calls or doing what I need to do.
With that said, why am I considering going back to a Note? The Note 5 has a nicer build, which helps a lot with point 1. I still don't think it feels as premium in the hand as the iPhone, but it's pretty darn close, although Samsungs horrid branding on front and back cheapens the device quite a bit. Point 2, the "just works" of the iPhone has seen many cracks in its armor IMO. Siri was great 2 years ago, but has not progressed at all and compared to Google Now is mediocre, the voice recognition seems to have worsened for some reason and I've altogether stopped using Siri because it sucks where it used to be better than GN. Also in order to get the things I want, such as a dark mode, I have to jailbreak and it makes the phone somewhat unstable and keeps me from updating to the latest OS. Even unrooted Android I can get a dark theme for, and do other things like turning off the need to swipe/fingerprint when near my wifi or cars Bluetooth.
My worries are about Samsung quality and customer service. I had so many hardware issues with my Note 3's and 4's I just am not confident in Samsungs ability to put out a trouble free piece of hardware. I'm going to give the Note 5 a chance though and see how it does. But you just cannot beat Apple's customer service, the ability to walk into a store nearby and have an issue resolved, their rock solid hardware, etc.
I'm definitely torn between the 2 and will keep them both and switch back and forth. I terribly miss the stylus and can't wait to get it back.