Sync issues are almost always down to WiFi issues (Not internet speed, but the WiFi network inside the home - how congested it is, how many devices are attached, and how far the Access Point is from the Minis all factor into it). Minis prefer 5GHz.
People using ISP-provided WiFi routers usually suffer the most because those simply don't have the horsepower to handle AirPlay's talkative protocols. There's a lot of back-and-forth traffic between your stereo HomePods as they compare notes to make sure the audio is in sync.
I had problems early on with stereo sync - sometimes, one of them would even give up and go silent. Adding more Access Points to my network resolved the issue for me. If you have good WiFi and feel you have coverage, I would check the documentation that came with the system to see if there's any filtering taking place that you might be able to temporarily disable, just to see if that's the problem.