Check my video in previous post, turn off Auto-Brightness.
Can confirm this has worked for me. Usually my memory usage would be through the roof after using my M1 Max all afternoon with even small apps like Mail pushing 6GB, but since following the advice from @TinyMito Mail is currently at a much more manageable 367mb and all of my work apps (Photoshop, Cinema4D, After Effects) are playing nice and not hogging memory when they're idle.
Massive thank you @TinyMito for pinpointing the issue and sharing the solution, truly appreciated.