For me, in watching about 45 minutes of YouTube videos without hardware acceleration, the problem does not occur. With it on, it started about 13 minutes in. Good find, Jig3n.
In looking at Activity Monitor, in watching a particular 360p video I see that it takes about 46%-50% CPU without acceleration, 27%-45% CPU with acceleration.
Besides the decoding of videos, the hardware acceleration is used in 2D/3D animations. A good example of what this looks like cat be seen at:
http://webdesign.tutsplus.com/articles/15-inspiring-examples-of-css-animation-on-codepen--cms-23937
On viewing this page on my Mini, I don't see any on-screen performance degradation between acceleration or not, but the CPU usage is about 100% with acceleration, up to 200% without acceleration. Because lack of acceleration for this page (which has much more animation than the pages I usually visit) seems to just eat more CPU rather than affect on-screen performance, I'll turn it off. However, I'll continue to use Safari for YouTube because it doesn't have this problem and it's more efficient. For the same 360p video, Safari had 3 separate processes - Safari Networking took 1%-10% CPU,
http://ww.youtube.com took about 7% CPU and the Safari process itself was 0%-1% CPU. It looks like WindowServer takes about 2% more CPU when viewing the video in Safari vs. Firefox. So a total of 10%-20% CPU. Also, for whatever reason, when the video finishes in YouTube (still on the YouTube page) and everything settles down, Firefox still takes 16%-27% CPU (this drops to near-0 if you close the tab/window). In Safari it stays at about 3% for a little while then drops to near-0.
I also noticed that one can set hardware acceleration off for Flash (I have tested what effect this has).
https://forums.adobe.com/thread/891337