Yeah, I noticed when doing voice calls the battery drops a lot faster. The buds themselves have been great for me battery wise even while making calls, but the case seems to deplete faster than I thought it would just keeping the buds topped up. That said, I have only now just put mine on the charger today after receiving them on Monday. And I still had 20% on the case and about 90% on the buds, so I probably would've have lasted through today as well.
It is a little annoying that I have to get the case with the buds and my iPhone to see the charge though, I'd much rather the case had two more led's to give more charge info just by looking at it. (Three green = full, one green = 33%, one amber = <10%) That or add a similar "pop-up" battery status window on every device and only show on the closest one (Mac, iPad or iPhone.)