Thanks for all the suggestions. In the end, TM was crazy fast to restore and everything seems perfect so far.
Glad it worked out. It does sound like Spotlight is still indexing. It will speed up when that is done.
Go to System Prefs and in the Startup Disk pane select the new drive as the boot disk. Without that selection the system searches around for alternate boot sources and is slow to boot. That should speed up your boot times.
----------
Again, the size of the data on the 1TB HDD really doesn't matter. Even if the data on the HDD is than a MB - Apple's Restore utility won't work. The Target drive needs to be the same capacity or smaller in size than the Source drive.
Lou
You can work around that by using Disk Util to shrink down the source volume to smaller than the target. In the OP's case there is 133GB of data on a 1TB drive, so you could just shrink the volume down to 200GB or so then use Disk Util to restore that volume to the 512GB SSD.