You are using the latest version of AP? Which version of Monterey? In the past one solution in case of a very slow startup of AP was to duplicate AP, delete the „original“, rename the duplicate (remove „copy“ from the name) and then start it. See here.
Then again the latest AP version shouldn’t have that problem.
Affinity Photo 1.10.4 for M1
macOS Monterey 12.2.1
AP takes ~13 seconds for the app to launch (the icon just bounces up and down for that long).
I'm just using the default AP settings as it came. I haven't changed anything.