I have been tracking my m4 pro mini's temperature on both the die and the ssd and an external ssd, to get an impression of normal. I have to tell you 50 degrees C is not that hot. In usage spikes I see 66 degrees with just a few apps open. What I've observed is the machine's priority is to conserve energy, which includes how fast it spins the fan, which it prefers to let spin at idle 1000 rpm until it is really, really needed. Also be aware that under full load it will hover around 110 and under continued heavy use will reach 118 and peg itself there.
I've been using LLMs that fully load the GPU with work for long periods at a time, what's when I see 118, and the fans come up to full speed. I also have played several 3D games at 4K and 5K, when I see it then hover near 110 and engage the fans continuously at around 2500rpm.