My own thoughts are that the built-in audio will be fine, but each person has to do what pleases them.
I spent about 25 years recording and broadcasting live music, with a special emphasis on Classical. I certainly understand and appreciate quality audio, but have little interest in Audiophile. All too often it seems like trying to get from 99% to 100%.
While it's possible to train the human mind to greater levels of discernment, most people don't even know how to identify old fashion clipping distortion in an analog amp, no less aliasing error in an AD/DA conversion (which is far, far harder to identify). Sure, clipping sounds bad even to the untrained ear, but typically people don't know "clipping" is what's causing their substandard experience. If you do recognize clipping for what it is, you're able to address possible causes (like over-driving the input to the amp).
Higher sampling rates? To me, it's mostly a matter of "it's the thought that counts." The greatest benefit to higher sampling rates is to obtain more samples of high frequency waveforms - the 10 KHz - 20 KHz range (at a 44 KHz or 48 KHz sample rates, a 20 KHz overtone is sampled just a bit more than twice per cycle). Greater accuracy hardly matters if you, like most people (including me, due to my age and professional wear-and-tear), have too much hearing loss to detect much of anything in that top octave of the human hearing range.
Arguably, the presence/absence of data reduction (lossy vs lossless) will be far more noticeable than the benefits of extremely high sampling rates or the fine points of high-end DA converters (to those ears/minds trained to detect them), and your choice of DA converter will have no influence on that, as data reduction is baked into the source material.
And all varieties of connoisseurship tend to suffer from "confirmation bias" - knowing in advance that you're experiencing something "better" or "best" enhances your positive impression of the experience (and vice versa).
So, if spending extra (and complicating your setup with outboard equipment) contributes to greater enjoyment of the listening experience, go right ahead and go for it. But I'm willing to wager that in 99 out of 100 people, a true double-blind listening test will show that "built-in" is good enough.
