I found that this issue came up after the latest firmware for the Air tags.So had a call with a senior Apple tech support agent and after going a little back and forth he was suggesting me to try some other battery optimisation methods.
I mentioned to him that this has categorically started happening from the day I paired my AirTag. He then suggested to turn the AirTag pairing off when I think I’m not going to need it. I said that defeats the whole purpose of having it. I might as well just return it.
He has now booked an appointment for me at the store. Let’s see what happens there.