I was able to get this working with the PS3 using Ubuntu 8.1 (i think 9 works also but havent tried it yet).
Its not a quick process by any means, but works and the results are pretty darn good. Im having some audio issues though, I cant figure out how to get LPCM on my MP4 file for playback on the PS3. But if you are using your Mini, you shouldnt have this issue.
If you need a quick walk through let me know.
If you can give me the walk through I would very much appreciate it. As an alternative to a PS3 if I could find a stand alone Blu-Ray Player that would also connect to Make MKV on a Mac that would work as well.