Last year I upgraded it to 12-core and 128 GB RAM, I wasn’t aware of this. I regret it now.
I have an upgraded Mac Pro 5.1 as well so i understand your situation, however, I don’t regret it at all because i think of it as a piece of gear that is incredibly flexible and can do many many things really well - I can think of very few, if any, other hardware that offer this level of upgradability / expandability and that is still very relevant so many years later... Heck, I’m even running Thunderbolt 3 in it, which was inconceivable only few months ago!
Now, should you absolutely need AVX-only plugins, you could always supplement it with another Mac/PC and keep using the 5.1 as your main recording recording rig. It’s not uncommon to use several computers and split the load between them using something like VE Pro.