HDMI - annoyingly - is designed to always see both video and sound.
I once spoke to an Apple technician and they explained that it is not possible to disable either audio or video separately from each other on a HDMI connection.
That's not technically correct in both directions. HDMI cannot be sound only, that is true. Some devices that want to only output sound, like SACD players, will put in a blank 640x480 video signal. I do not believe any computer will do this.
However, HDMI can and is routinely sent without a sound signal. This includes legacy DVI devices, HDMI graphics cards that are too old to support sound, and even a common AV receiver which strips the sound out as to avoid having the TV play simultaneously.
Under Windows, you'd just disable the graphic card's audio device, but Apple has no official way of doing this.