Some more thoughts:
-Boy do I miss the dedicated back button. Sure iOS has a button that takes you back but it's not always in the same place. Sometimes it's upper left, sometimes it's lower left (Safari's back button), sometimes it's upper right, and other times it's a "Cancel" button in the lower middle of the screen. You have to search for it, instead of just having muscle memory of tapping back. I post example pictures here: https://forums.macrumors.com/posts/17003123/
-Also miss the dedicated menu button. Goodness gracious, iOS Settings is a mess! It's honestly difficult to find certain things. And I hate how you have to leave an app in order to go to the iOS Settings to get to that app's settings. It's... so counterintuitive. Then, once in a while, you'll have an app like Gmail where there are no settings in the iOS Settings screen. It's actually in the app, so you feel like a fool going to look for it in the iOS Settings screen. Very inconsistent.
-Why does the App Store keep asking me for my password every time I download something? It's so annoying. I already registered the device and signed into iTunes/App Store with my Apple ID. Make it stop.
-Scrolling in Safari is dreadfully slow.
-While the keyboard has been faring okay, it's still no where as easy to use as Swiftkey. Boy, do I miss swiping. The iOS keyboard with its narrow screen BEGS for swiping abilities. Goodness, Apple. Update the keyboard. Everything on this keyboard feels so "manual". That's the best way for me to describe it. You have to really labor to get what you want to type and what symbols you want out. Miss predictions too. Swiftkey used to finish sentences for me. It's such a breeze tapping once to get a full word onto the screen.
-Small thing, but I miss my scrollable wallpaper. iOS looks so static and dead. It's really kind of sad.
-All Mail is nice, but it's kind of annoying you can't immediately tell which mailbox the email you're looking at is from. It's annoying that emails are only partially downloaded too. I have to click "download" toward the bottom to get more of it. What the heck is that? It also has to load any old mail or mail in other Labels. With Gmail, your entire mailbox (literally) was available instantaneously.
-BIG MISS: Notification light. I hate having to keep turning the device on just to see if I've got anything waiting for me. This is such a useful feature. To all those people who keep talking about how Apple doesn't implement something until people are ready for it or until they can do it right, here's a perfect example of that philosophy being utter BS. Notification light is majorly useful and it's not hard to get a simple light "wrong."
-I miss not being able to toggle the lock screen on/off. It really sucks having to put in my lock code every single time I access my phone, even if I don't need it, say like when I'm at home. On Android, you can do this, and so when you're at home, you toggle off your lock screen, thus allowing instant access to your phone.
There's a few other small things that bug me about iOS... but other than that, it's not bad. Not as bad as I thought going back to iOS would be. Maybe because my expectations are so... I wouldn't say low... but more like "trained."
EDIT: Another thing I can't stand. Notifications. The little flippy thing that comes down from the top of the screen is so obnoxious. How can anyone accept that this is all Apple could come up with to notify you? It literally blocks any access to the buttons on top. And since the "back" or "cancel" buttons are usually at the top, you have to either wait for it to go away, or swipe it away first. I mean, it's just so unintuitive. I can't believe this is what Apple software developers, after a year, came up with. This is all they could offer? How do iOS users accept this? Really.
----------
I'm genuinely trying hard to keep an open mind and appreciate the iPhone for what it is.
But... I don't think I will last. To say that iOS is behind Android is just the biggest understatement of the year (presently). Everything is just so much more laborious on iOS. Everything is just so much more "manual."
Can't disagree with most of what you said and honestly, not surprised as you're on record with most of these quibbles already and why I thought you'd miss Android.
I find myself able to cope with most of these because, well, I used them for years and the muscle memory is still there. I like how many things are implemented (or available to be implemented if you so choose) on Android better but don't mind using iOS's methods in most cases, again probably because I did it for so long. For example, the Settings inconsistency can be annoying but in my experience, after initially setting up an app, I find I'm not accessing Settings much at all in daily use. And there is some inconsistency, though to a lesser degree in Android as well. The Nexus 4 doesn't have a dedicated menu button so you have to find the settings button in an app and it's not always in the same place--one reason I like my Note 2 better.
Can't say I noticed Safari scrolling being slow--actually find it a bit smoother than Chrome on Android and really do miss the tap to top feature when using my Note 2.
If you do choose to jailbreak, check out AndroidLock XT--lets you use the pattern lock available on Android and even has a feature to bypass it if connected to specific wifi networks--would be a workaround for turning off the lock screen security while at home and you wouldn't have to toggle it manually (this based on the assumption you have a wifi network at home).
Also, while not a back button per se, LastApp (also only available after a jailbreak) allow you to switch back to your previous app. I have it implemented using a swipe for the left edge (similar to multi-touch gestures available on an iPad). I swipe from the right edge to return home--hate using the home button so much. And you absolutely should check out Auxo (I think you actually linked to a preview video from the developer in a thread a while back)--makes the app switching tray much more functional (mini previews of apps vs. just their icon, iPod controls, toggle switches for things like wifi, BT, screen lock, etc).
Having not used iOS for a few months, it's flaws are much more evident to me but I find myself not minding them quite as much as expected. I think the biggest reason I can tolerate many of them is I still find iOS and it apps more aesthetically pleasing, just a better overall 'fit and finish', can't really find better words to explain it--in general smoother and less finicky. Same goes for the hardware--using the Note 2 confirmed I genuinely do prefer a larger screen but it still does feel/look cheap compared to an iPhone. Guess I am just a bit more form over function than I thought.
Not saying I'm switching back to the iPhone because I also find myself missing much of the flexibility/freedom I discovered by using Android. I guess I'm just in a similar situation to you in that I just don't find ANY top end smartphones to fully meet my expectations. If only I could have the fit and finish of the iPhone/iOS with the flexibility/freedom I enjoy in Android and I'd have MY ideal device.
Alright, sorry to derail the thread. I believe my take is already on file a few pages back but in case I'm mistaken, I wouldn't trade my Note 2 for the S4 as I don't see it as enough of a difference to switch but would definitely give it a test drive if I only had an iPhone.