You know I'm a fan of iOS but even then, knowing how you use your phone and what you expect out of it, i wouldn't recommend you go back to an iPhone. There are a few things you may like but overall, but I think you'll realize much of what frustrated you in the past is still there.
You seem to be quite happy with the Note 3--something specific (other than size) that's pushing you to switch, or is it just the same 'new gadget addiction' most of us here suffer from? If you must try something new, I think you've already hit on a great alternative--the Moto X. If I were to switch from an iPhone, it would either be for the Nexus 6 or the Moto X (or maybe the upcoming M9 is rumored specs are true

).
Yeah I think I may check out the MotoX. The nexus 6 is too big for me, although I liked the Nexus 5 quite a bit. I think I want something that is not carrier restricted and can get android updates immediately instead of waiting 5 months.
And yes, part of it is the new gadget addiction Lol.
----------
I never tried to jailbreak my iphone 6, but the only thing that frustrated me was 3rd party apps. I know apple added the ability for some things to work with 3rd party apps, but a lot still did not work. For example, I store my photos on box (cloud storage) and I have to download the photos to share them with the messages app. It will only let you chose icloud photos or your physical library
Also, what lag are you experiencing? That is actually something I am noticing less of on the n4 compared to the iphone 6/6+
I find i have more lag on my 6+ than my N4..........
These are the areas where lag is the worst on my N4:
Opening email app, takes 10-15 seconds to open, it's nuts. This is even with it preloaded in the background. I don't know if maybe it's checking for mail or something.
Opening hangouts. First time it takes 5+ seconds, once it's preloaded then it only takes a second or so. But it's almost never preloaded because I always reset background apps, see below.
Switching from camera to gallery, this takes like 10-15 seconds, even just opening the camera at times can take 3 or 4 seconds.
Recent app menu takes a long time, this varies with how many apps are open. If no apps are open then it's fairly snappy, but the more apps are open the slower it is.
Overall opening apps and navigating can be laggy, not bad at all, but that little half second delay is noticeable. Playing with my wifes iPhone 6 and it's so fast, I would not be scared to use the word "instant". Especially with things like opening email. Now maybe the ip6 will slow down after time and after installing lots of apps, I don't know.
Things I have tried:
Turn animations off, lower them, increase them, etc etc.
Greenify helped a little bit
clear cache helps quite a bit, but only for a little while
factory reset, helps a lot but after a couple of days it progressively worsens
I'm also not super happy with touch wiz changing a lot of its elements to white, I hate the white themes but with Lollipop coming I suppose this is endemic to all android phones eventually.