Presumably it does not stutter when you watch it on your iPhone or Air, right?
Running with that "presumably," the internet speed won't matter if the source device is getting a stutter-free version. The issue is likely the bandwidth INSIDE of your home.
One quick & easy test is to try the alternative to how you connect to your
TV now. That is, if you are currently connecting your
TV to the internet via wifi, test a wired connection just to rule out (or in) the wifi itself (not necessarily as a permanent solution- just a test). If you hook up the other way and the stuttering stops, you can focus in on the problem much more specifically.
Since you are leaning on airplay, is there anything in between your usual seated position and your
TV that might be wifi unfriendly? Any obstacles? Any RF-disturbing tech? Temporarily but significantly relocate your
TV to a very different location and try to run some stuttering videos again. Still stutters?
Do you have multiple people in the household taking bites out of your overall wifi bandwidth at the same time? There is only so much capacity available. If lots of people are taking bites, there may not be enough leftover for stutter-free consistency.
Are the issues repeatable at different times of day & night? For example, can you find one problematic video that always stutters and then test it at different times of day to see if it always stutters. If it doesn't take note of the times of day and then explore what may be affecting your wifi at those times? Close neighbors running something that might adversely affect bandwidth?
Can you find something on youtube or similar that always stutters when you airplay it? If so, try the native app on
TV locate the exact same video and see if it also stutters via the app? Again, the idea here is to narrow in on the issue vs. suggesting you only use apps instead of airplay.
Are the problematic videos barely compatible? Are you asking too much of the
TVs horses in playing specific videos that are encoded beyond it's capabilities (which may not be the case with your X or with a computer's graphic card). The "4" and even older "3" have aging hardware not able to deal with much beyond their relatively narrow specs. Maybe you are asking them to do more than they can?
Are stutters random or always repeating at the same point in videos?
Are stutters maybe stuttering because you are multitasking with the source device? If you are asking the source device to do a few things other than just airplay a video, maybe the load is overtaxing the source device. Even a loaded iMac will stutter if the multitasking load is too heavy.
If you can possess a few of these stuttering videos, test them another way: dump them into iTunes and directly stream them to your
TV (not via Airplay). While this is just another test of airplay, if they don't stutter as an iTunes stream, you'll know it's not the files themselves, nor a file that is too much for your
TV to process.
And of course, you have tried the old "swap out the HDMI cable" too, right?
If pretty much everything stutters, even- say- Apple's movie trailers- you might try "reset to factory settings" just to clear out any crud that might have accumulated.
And, for the "4", do you have a fair amount of free space or is it near full of apps? Some people report stuttering when they've "filled" an
TV with apps (even though it is supposed to dynamically dump apps when it needs more space).