That is to say, those who did so [mostly] came to regret it sooner rather than later.
I wonder if you can give us your sources for this because it hasn't been my experience at all. In fact, I've been using a 13-inch M1 MBA with 8GB RAM for over 3 years, and liked it so much I bought a 15-inch M3 MBA with 8GB RAM.
And that's allowing the fact that I also have an M1 iMac with 16GB RAM, which means by this kind of reckoning, I ought to be seeing a notable disparity in performance between the 8 and 16GB systems. In fact I'm not.
True, I don't tend to bother much with Activity Monitor because I don't have a problem I need it to help resolve, so on a routine basis I'm not looking at the memory pressure or cache or swap or any of those things some people seem a bit sensitive to and even hung up on. What I actually do is use my computers for the wide range of tasks I bought them to deal with. All three manage quite well.
That use includes video editing in FCP, preparing and editing large volume docx documents, photo editing in Affinity, GraphicsConverter and Pixelstyle, some web development, management for a book library, slide deck authoring for user training courses, server and network management, SIEM platforms, video playback via AirPlay, and the usual web and email type stuff - though I don't use Chrome at all, and since I was around prior to browser tabs even existing, I tend to use bookmarks rather than have lots of tabs open - typically about half a dozen in Safari, sometimes Firefox at the same time too.
I also manage a fairly diverse network (systems and users) which include other 8GB Mac owners, and none have an issue, despite routinely relying on my help and support. These are professionals with jobs to do, deadlines to meet, and close to zero tolerance of inadequate resources.
I wonder, what is it that we're all missing when 'those who did so [buy 8GB systems] came to regret it sooner rather than later'?
Not for a minute would I fail to recommend users buy the most RAM they can afford, and I think Apple's $200 premium for 16GB is unjust, but it looks to me that in the context of the thread and the question it raises, the reason why so few deals involve higher RAM configurations is pretty much as
@theorist9 suggests above.