In my case, I have disabled continuity and universal control and I don’t have problems anymore.
These features have probably a very intensive usage of BT and can cause lags
Since I have disabled this (I don’t really use that), it works really better (iMac 5k 2015)