Handbrake slow ? it can't be any or much slower than MKVtools.
The only thing i can think of is that with MKVtools you are not re encoding the video just passing it through.
Passing the video through will work for some but not all videos.
Mkvtools has it's place it's useful for soe things, encoding video Handbrake is far superior. Far more control over the output.
But I think the goal is to not have to re-encode. Unless I burn subtitles, I've never had to re-encode the video track for playback on my PS3. MKV to MP4 video conversions takes minutes instead of hours with no loss in image quality. Why not try for the same with the ATV?
Also, just so you know, I used to religiously tweak settings, minimizing size, optimizing picture quality and then compare the resulting videos with magnifying glass. If the output had some macroblocking around a tree in the upper right background, the settings would be changed to see if it could be minimized. FfmpegX, D-Vision,and the old version of handbrake was awesome tools. The newer version of handbrake is even better. I would have died to have had all that tweakability. I was pretty extreme.
But eventually I realized some things. The first was, that the original video had that macroblocking around the tree. I was starting with an imperfect video and a few more imperfections didn't really matter. The second was that, when I was actually watching the movie, I wasn't looking at that tree in the upper right. I didn't even notice it. The third was, it was hard to find the optimal settings. That's because, all that tweaking, though there were noticeable results, resulted in only slight improvements.
So, I'm going to have to disagree with your "far superior" comment

. Handbrake is great video tool , and does offer far more control over the output. Like I said, I would have loved to have it available in my younger days. But a lot of people just want a simple solution for casual watching, which is the goal of MKVtools. If you use the same bitrate, most people aren't going to be able to tell the difference between outputs from the two. Even if you use passthru with MKVtools, the optimal solution, I doubt that most people could tell the difference.