Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Yes, the mini jack will fit your Mini and the other end should fit your receiver. Check to make sure your receiver takes a standard Toslink connector.

Next, you need to set DVD Player app preference to set digital out. You must do this before you open a DVD or VIDEO_TS folder for the passthrough to take effect.
 
am i right in thinking that i can only make use of the 5.1 via dvd player or will vlc work aswell?
 
You don't need VLC (which is buggy as hell). Here's what you do:

1. Use Handbrake to encode your DVD as an AVI (MPEG-4 with AC-3 passthrough) file. You then open the file with QuickTime Pro and then save as a self-contained movie. This movie can then be dropped into iTunes, making it available to the Mac through Leopard's Front Row. I did this with the first Spiderman movie and it was compressed at 2500 bit rate with AC-3 5.1 DD to a final size of 2.5 gb.

2. Follow the directions here to install Perian, configure Audio Midi Setup to digital output, then a modification of the com.cod3r.a52codec.plist file using Property List Editor (part of Apple's XCode development package and a free download).

Playback in Quicktime or Front Row will give you Dolby Digital 5.1 (provided the original DVD was encoded in DD 5.1).
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.