If you were to look at your cpu usage during the time the matroska file was loading, you'd probably notice that one of the cores was maxed out for the whole while. If so, then vlc is having problems loading and reading (obviously) your matroska file, and is causing for a temporary cpu loop, or hang.
I've had that problem for a couple of days with hd matroska files on anime, and it's something to do with vlc not being able to properly decode the subs of the file. This can either be fixed with getting a newer vlc or doing something very specific with its preferences. Anyhow it's a known problem and you can possibly google it and find the solution on what to change in its preferences.