I don't have an iMac Pro, but I'm interested in this topic.
My preference strongly leans towards silence; so much so that my old Mac Pro resided in a separate room at the end of a long cable. Any computer sitting 18" from my ears has to be darned-near silent. However, computing costs power (by definition at the quantum level, even), and in real life power has a cost in noise. An ideal computer would be silent while doing simple things like browsing, but would spin up to the degree that is needed when needed.
Anyway, to the Gods of Apple, do the best you can, but in the end give me the flexibility to choose. Thank you.