On iOS13, my AirPods would connect really fast to whichever device last used them. I only needed to go to settings when I wanted to switch devices.
On iOS14, I have start playing sound from the device I want, place the AirPods case close to the device, open the case and wait for the power indicators to show up on the device, then I can take the AirPods out and place them in my ears. Usually, if I follow this exact procedure, it will connect. It takes 3-5 seconds for the power indicator to show up on my device where previously it was near instantaneous.
But still sometimes, I've had to put the AirPods back and repeat. And I've had to mess with Bluetooth settings.
It's been a worse experience for me on iOS14 because of this. I haven't needed the auto-switching feature while working from home. I use them mostly for my iPad, but my phone is always near. So it's just been a big inconvenience for me since updating my phone and tablet.