This probably isn’t the best place to update Handbrake, VTB vs software, quality tests but since it started here, and it’s related to the Studio, this is where I’m placing it.
Per eddie_ducking’s suggestion, I finally made it back to the Apple Store to do some more tests. I really wanted to evaluate the quality of VideoToolBox on the Studio M1 Max (32GB, 24 GPU). I’m pretty much just itching for an excuse to buy one.
preemptive TLDR:
VideoToolbox on the M1 Max is crazy fast, but the quality just isn’t nearly good as software H.265. And, if you compensate by bumping up the quality, you might as well just stick with software H.264.
The M1 Max Pro flew through the renders, we’re talking 350+ fps. And though the Apple Store was loud, the machine itself seemed quiet. I checked activity monitor and during the encodes, the CPUs were around 650% while the GPU was only around 15%, which I found surprising. I thought it would be pushing the GPUs much more.
So the Studio Max was fast and quiet, but the resulting files are not very good - at least when compared to software. Caveat, nobody watches videos or movies this way. That is, I took a single frame and pixel peeped at various encode settings. When you’re actually watching this stuff as a video, the quality is much more fluid.
That said, the differences at similar megabits per second were very noticeable to me. I tried to choose a challenging frame, lots of shadows, high and low contrast, a lot of texture and color gradation. All tests were done using Handbrake 1.5.1 and their recommended benchmark file, “Tears of Steel.” This is frame 3:35:15 at 100%.
As a normal baseline, for HEVC content, I try and hit 2.5 - 3.0 Mbps for 8-bit, 1080p. “Tears of Steel,” is 1920x800 so 2.5 Mbps should result in a better-quality-than-I-normally-do encode.
As you can see, the 2.6 Mbps software encode (2, RF 24.5 in Handbrake) looks good. There’s definitely degradation. But I’m fairly confident that there’s little chance I’d notice while watching at 24 fps. The resulting file was 239 MB for 12 minutes. The original H.264 file (1) is 584 MB.
The 2.6 Mbps VTB encode (3, CF 46) looks really bad in comparison. There’s very noticeable texture loss and blurring. The bricks have lost significant detail and the atmospheric light in the leaves is blurry mess. So I bumped up the quality.
At 3.6 Mbps, the VTB encode (4, CF 56) is still not as good. Though the differences aren’t as drastic. Still the software encode is sharper with less texture blurring.
At 4.0 Mbps, the VTB encode (5, CF 58) starts being actually comparable. It’s till not as sharp as the 2.6 Mbps software encode, but I think it handles the gradations better.
But now we’re at a file that 54% larger.
Out of curiosity, I decided to try and produce a 370 MB H.264 re-encode (6) and compare it to the 370 MB CF 58 VTB encode. My 9700K did this at 115 FPS, which is still quite a bit slower than the M1 Max’s 360+ fps. But is still 2x as fast as my normal ≈60 fps H.265 encodes.
The quality of the H.264 re-encode at 4.1 Mbps is nearly indistinguishable from the original file and it’s just a little bit bigger than the best (5) VTB H.265 encode.
Now, some (a lot?) of you looking at the comparisons, may not see much difference at all. It’s harder to see the differences when viewing side by side. But, if you take the time to clip the images and place them on top of the original, the differences are much more drastic. But even if you don’t take the time to clip and overlay the images, it’s very obvious (to me, at least) that 3 and 4 are unacceptable. Five isn’t quite as good as 2 and it’s 50% larger in size - though it did encode 6x faster. Six is nearly perfect, and encoded very quickly while being just a little bit bigger than 5.