I have an 8gb M1 Pro. I currently have 6gb swap memory (just dropped to 5gb). No noticeable performance issues.
This is often the way it is for my usage - Is a 6gb swap suggesting 8gb ram not enough?
If you're happy with the performance, you don't need to worry about it that much.
If you really want to look into it more, open terminal and type top. It's a bit like the activity monitor.
On the top two lines you will see something like swapin 232345858(2342) swapout 234243 (112).
Ignore the part that's not in parantheses. Watch these when you're busy and the computer under stress.
Quick explanation: in activity monitor the swap number is total file size. In swapin/out before the parentheses, these are cumulative numbers. In both cases they tend to grow over time since last boot (I'm simplifying a bit).
But what actually matters in terms of performance and problems is how active they are, and the numbers in brackets are basically recent. If both of those are changing quickly, especially at the same time, it is clearly a case of more memory would help. (In my view. And a bit simplified).
If those numbers in brackets only bump once in a while when you do something unusual, well, memory would reduce it, but it's most likely not a problem. If they're almost always on zero, then you're fine.
My guess is with six gb swap file you would benefit from memory, but how much - well, personal question, how much will you pay if you don't really notice the performance hit. But then, maybe you will ntoice it when you try to do something mroe challenging.
