OP, the first thing I would do is use a different computer hooked to the same USB cable to see if it too shows 0% charge as you describe. That will help ascertain if this problem is with the UPS itself. Even if it is new, perhaps the battery is bad/defective. If it IS new, there is sometimes a task of first-time connections/startup to activate the battery (so that it doesn't get powered up by mistake in any "bouncing around" shipping scenario). In my experience, this is handled by sometimes having to connect one wire, pull a plastic tab or, most recently, holding the power button for a few seconds instead of the typical quick click to tell the system you are ready to fully use the battery.
If the second Mac has the same issue, the problem is probably with the UPS. Recheck instructions to see if you need to do something unique to use the battery. If no, open it up to be sure the battery is actually connected. If so, swap the battery with new ones and/or first test the battery if you have access to a tester. Yours would not be the first new UPS with a dud battery.
If all with the UPS and battery is fine, rule out Monterey (and maybe BigSur) by hooking it up to an older Mac or PC too. I'm convinced that Big Sur-Monterey has USB bugs and this might be another example of that. If you have an Intel-based computer or can get a friend to bring one over, check the UPS hooked to one of those too. If it is good there (indicator shows battery is towards 100% charge, etc), that shifts issues back towards Silicon/Monterey (and maybe Big Sur too).
You mention unplug/replug temporarily resolves this issue. So I interpret that to mean that the battery is likely fine and this is more like a dropped USB connection issue: battery is ready to kick in but reporting information shows 0%. compare the 0% reading in the pref pane vs. the UPS on front-panel readouts to see if the latter shows batter at or near 100%.
Unfortunately, if it turns out to be a Mac compatibility issue/USB bug, you are basically at the mercy of waiting on Apple to debug the OS (or possibly a firmware update to the UPS). I'm there with you as perfectly-functional USB hardware on my Intel Macs running macOS BEFORE Big Sur won't work reliably with my Studio Ultra running Monterey. For me, this is generally "unexpected ejections" of USB connected hard drives. I wonder if you are basically seeing "unexpected ejections" of the UPS, resulting in pref pane information showing 0% as if there is no UPS there at all.
All that shared, once you can know that the UPS is working correctly- that is, the battery is engaged and towards 100% of charge- the USB connection is mostly only an informational one. The UPS should still do its primary function whether the reporting of charge percent is 0% or higher. Perhaps just disconnect the USB connection as if your UPS doesn't have one at all and let the typical "beep" alerts when in battery mode warn you to wrap up your work and shut down your computer if power is likely to be out for longer than the battery can support.