I opted not to do the phone to phone transfer. I just did it old school: backed up my original to iTunes, then restored it to the new one.This was my experience, too, when transferring from my iPhone SE (original) that was already on 13.4.1. I was pretty impressed by the ease of the transfer.
The only pain in the neck was iCloud Photo Library. It took overnight to sort itself out. But it was done when I woke up this morning!