I have connected OWC TB3 Envoy Express NVMe drives via OWC TB3 14-port docks to both the 2011 iMac and the 2011 Mac Mini. It works. I have booted High Sierra off the TB3 drive for both Mac models.
The dock has its own power supply so it can supply power to the drive. It’s not cheap but it does work. You are also limited to using High Sierra for compatibility with the drive.
It does mean that the bus powered drive that is meant to be portable isn't nearly so portable with these older Macs as you need to have a dock connected to power. I presume they don't promote this setup as they don't see a big market for it.
You get a 2011 iMac
Connect a TB(1/2) cable to the iMac
Connect the other end to an Apple TB3 to TB2 adapter (this is the only bi-directional adapter)
Connect the TB3 end of the Apple TB3 to TB2 adapter to a TB3 dock with its own external power supply such as the OWC TB3 14-port Dock (check system requirements to make sure it works with 10.13 before purchasing). The dock is needed to supply power to the drive.
Connect a TB3 drive (check that it works with 10.13 before purchasing).
I'm using the OWC Envoy Express TB3 enclosure with an Aura P12 NVMe drive inside.
This should work with all Macs with a TB1/2 port, I think, though I've only tested on the 2011 Mac Mini Server and 2011 iMac.
I updated to the latest High Sierra 10.13.6 with all security updates and cloned the hdd onto the external SSD using Super Duper. The Macs feel a lot quicker booted off an SSD.
The TB3 dock does have USB3 ports. Unfortunately it specifically states in the manual that the USB3 ports are only bootable on 2017 and newer Macs so I haven't tried booting off USB3. I haven't connected any USB3 drives yet to test file transfer speeds.
Currently I only have the Mac Mini booted off a TB3 SSD (trying to figure out if I can boot Windows off a TB3 SSD with the iMac - I almost exclusively boot Windows on the iMac, the SSD is still connected and is seen in Windows - though I think booting Windows off NVMe requires UEFI not sure if trying that would be safe on the 2011 iMac). I haven't done speed tests with the iMac yet but I presume the results would probably be similar with numbers a little better than an internal SATA III SSD but not by a huge amount.
View attachment 1698387
View attachment 1698388
View attachment 1698389View attachment 1698390