Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Do you have match frame rate on? if not, try that. This lets the TV handle the different frame rate processing (usually better than the Apple TV). How to link below.

Match Frame Rate
 
I'll take a wild guess and say it is a wifi issue. I occasionally watch content using airplay to my AppleTV4K. It almost always works fine, but occasionally it does not. The Wifi bands are really crowded and a lot of things interfere with the signal, I say this is likely in my case because if I have audio set to my HomePods which use wifi, it is more likely to occur.

Tis is my trouble shooting routine for issues
1) restart appletv, set audio to wired output, power off and then on
2) reset wifi on my Mac. by turning it off, going to network settings, deleting wifi, then adding wifi (I don't remember where I read this, but it is supposed to rebuild a lot of tables used and get rid of glitches)
3) restart my network.
4) make sure all devices have good wifi performance
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.