Do you mean creating a new Fusion Drive with the old and new SSDs after replacing the HDD with a new SSD?
Well, no surprise, there are pros and cons to doing it and not doing it. Some of these totally depend on the shape of your current OEM SSD.
Pros:
If the current OEM SSD is in pretty good shape and has decent read and right speeds, creating a new Fusion Drive with the OEM and SATA SSDs could boost drive performance over using the SATA SSD alone.
Even with the tiny 24GB SSD, there could be a noticeable difference with drive performance, especially with write speed.
Cons:
It requires an extra step, and adds complexity to the drive set up.
If the current OEM SSD is in bad shape and has horrible read and right speeds, creating a new Fusion Drive with the OEM and SATA SSDs could have no impact on drive performance over using the SATA SSD alone, making it pointless.
If you want more information about the PCIe SSD options, check out this tread:
Here you can find and discuss all the collected knowledge regarding upgrading the HDD, blade SSD and CPU of a 27" iMac 2012-2019, as well as a list of performed upgrades. Here is a list of all 27" iMac models subjected in this thread: Late 2012 - 13.2 - MD095LL/A (2.9Ghz i5-3470S, HDD only /...
forums.macrumors.com
It has a bunch of information on compatible drives, adapters needed, tools, potential problems and fixes, etc.
Also, you can look up iFixit for guides, tools, and adapters, SSDs, etc.
Replacing the OEM SSD with something aftermarket requires the logic board to be totally pulled, but if you are comfortable doing it, you could get a huge speed boots for not much more $$$ than using a SATA SSD.
I personally don't like using the adapters, though. Maybe people have use the adapters with no issue, but there are also people that have had issues with them. And since it isn't exactly quick or easy to make adjustments or replace the adapter, I would most likely avoid doing it.
It is better to just look at the thread I posted above to get an idea of what you would be getting into. It is a lot more complex than just replacing the HDD with a SATA SSD, and there also seems to be more issues with replacing the blade SSD than just using a SATA SSD.
But, the speed of replacing the blade might be worth it to you to try it.