Maybe it matters to you, maybe it doesn't.I have similar use to yours, and my 16Gb M1 iMac certainly does have to juggle it's resources more than my intel iMac with 24GB RAM. (According the Activity Monitor)
But at no point is the M1 showing any visible signs of struggling - not stuttering, no fans, no beachballs, slowdowns, etc.
So, what does it matter that the memory pressure is higher..? As long as the job gets done...
So far at all times the M1 always feels faster to use, despite having less RAM.
(Okay I do admit that it'd be nice to know I could put more RAM in if I ever needed it - but I think those days are now gone with Apple.)
You're not noticing slowdowns because the standard disk access is very fast - likely 4-40x faster than what you were using previously, depending on what you upgraded from. So if your computer needs to move data from the RAM to the SSD and back again, it's not as noticeable. That used to be a beachball-inducing event in the past.
But shuffling data between your SSD and RAM is still not as fast as keeping everything in the RAM. And it's more wear and tear on your SSD.
If you're not noticing the impacts and you're not counting the seconds on your workflows then you're right, maybe it doesn't matter for you. Most people probably won't have their computers long enough to ever encounter a worn out SSD even with the increased activity between RAM and SSD, either. It's an added strain on the system that wouldn't be occurring if there were more RAM available, and the question is really whether that added strain will impact you or not.