First off, the larger the SSD, the faster it is going to be. So if you want the fastest possible performance from your M1 iMac, get the larger SSD.
Secondly, a larger SSD will incur less wear and tear over time than a smaller SSD because it has more places to swap to.
External drives are always going to be slower than an internal one. The degree to which they are slower has to do with the speed of the drive and the means of interfacing with the computer.
Key thing to remember here, don't treat your system drive like a library. Only store the stuff that you have to on it and dump the rest to an external drive. This frees up space on the system drive to do what it was designed to do... basically act as a swap space.
Remember, your last computer was perfectly fine for you when you bought it... it wasn't until later that it wasn't. This is why buying for the lowest common denominator can backfire on you. If you underestimate your typical usage over time, you end up replacing the computer sooner.
Now if you are merely buying this because of all the hoopla surrounding ARM Macs right now, it doesn't matter what you get because you will replace it when they release the next iteration of ARM Macs.