MBP16" 5500M 4 GB clamshell with Ultrafine 5K (TB3) very much has this issue. iStat Menus reports 20+ Watts of constant power use from Radeon High Side.
Fans run at a minimum of 2800 RPM when the computer is hands-off idle, and 4000-5500 RPM, if watching a video, using any apps or having a telco with Teams/Webex. The same happens with the latest Big Sur beta. Luckily I'm not a podcaster or a voiceover artist, because even a potato microphone would pick up the noise.
The 20W baseline consumption means that any CPU load over 10% makes the fans run at 4000 RPM or more. I can't concentrate to work without closed or in-ear headphones with this Mac. It's not the end of the world, but I would have stuck with the 2014 maxed out iMac if I had know about this "feature" last November. Finance-wise I could exchange this setup to the new iMac without loss, but those 2013 bezels are just too much to look at for 4 more years
When the external monitor is disconnected, fans slow down to 1800 RPM very quickly and stay below 3000 RPM most of the day.
Do you know if the 5300m does the same?