Well, I just bought me a new 7,1, base model 3.5Ghz 8-Core/96GB/256GB/580X (I immediately installed a pair of 32GB sticks out-of-the-box) and after installing Big Sur and going through the migration route to SATA Samsung 4TB EVO 970 PRO, I, too, feel it's slower sometimes than my 2012 Dual 3.46 12-core/64GB/Sapphire. Booting sometimes feels slower. Mundane Finder operations, meh.
I used the same EVO in both machines (performed a CCC to a 4TB spinner in order to move the EVO to the 7,1, fresh Big Sur install, then migrate). So it is literally the exact same boot drive, exact same (but fresh) OS, exact same software and user setup.
At 3.5GHz, this is the fastest clock available. And so far, I'm not impressed in regards to normal Finder stuff. I haven't yet done any Photoshop/Illustrator/Indesign stuff, nor any FCP/Resolve.
A SNAFU required me to replace the SATA EVO 4TB ASAP, and hunting for bootable 4TB solutions for the 7,1, I decided to bite the bullet and get the official Apple 4TB SSD kit; I'm in the process of migration and will report back if this improves Finder performance any.
I was told more cores definitely helps with the Adobe suite so I was going to step up to a 16 core, but with your Finder experience, I will probably delay and go with a W5700X upgrade first.