If you have a reasonable system
Toslink is better
HDMI in it's current form has an issue with jitter
jitter over HDMI has been measured as high as 20 X higher than via toslink or spif coax conection (these two are identical)
jitter is caused by clock errors between sourse and recieving device, HDMI currently has no way to sync the two clocks.
High level of jitter has a few effects, a producing a lack of focus and timing to music.
Note the above applies to all devices with HDMI currently, so applies also to DVD and Blu Ray players used for CD replay.
Several companies are working on a addition to the HDMI interface to fix this issue, Pio in particular.