I just switched from an iPhone 5 a Verizon Galaxy S4. To answer your question: The lag is there, but it's not because of the OS, it's because of the manufacturer skins.
I've come to this conclusion quite simply. I've had my GS4 since last Thursday when it was released on Verizon. It was a very fast phone. Sometimes. It would randomly lag, animations would slow down and then rapidly catch up, or they'd freeze for a second or two. The screen would become unresponsive. The keyboard would take a second to show up and the animation of the keyboard appearing would lag so much that it was if there was no animation. These were little lags, as in, you could tell the phone was fast as it caught up with the lag rather quickly. This led me to create a theory. My theory was that the GS4 is not lagging because of Android, it's lagging because of TouchWiz and the all of the **** that gets loaded onto it. I knew how to test this, but custom ROM's for the GS4 had to be released, so I waited for a few days.
Today, an unofficial build of AOKP (a variant of stock Android or AOSP) was released. This is only build 1, but I figured it'd be good enough to test my theory with. I then backed up all of my apps, messages, contacts, etc. from TouchWiz and proceeded in wiping my GS4 and flashing build 1 of AOKP for the Verizon GS4. Upon the first boot and setup I noticed that the keyboard lag I spoke of earlier, was gone. It was very smooth, I'd even say as smooth as iOS. I then began to restore all of my backed up information to see if maybe all the crap that I had installed in the few days I've had the phone had been the cause of the lag. I let it all restore and began using the phone until I went to bed.
Guess what I found out? The PHONE DIDN'T LAG AT ALL. Not. One. Bit. And on top of that, the battery life was significantly better. Honestly, when I first got my GS4, I began to miss the smoothness of iOS, but after flashing a stock ROM, I have no complaints. I do not understand why manufacturers WILLINGLY apply these skins to their devices only to cause them to lag so horribly that their flagship devices seemed "low-end". This truly does puzzle me. So, in the end Android doesn't lag. TouchWiz does.
TL;DR: Android itself doesn't lag, manufacturer skins cause it too. My Galaxy S4 is lag free after flashing a stock AOSP ROM on it and getting rid of all the TouchWiz BS.