I have the same exact problem. My phone is sitting right next to me at work, and the connection drops constantly. Either stuttering or a few seconds of pure silence. The left AirPod actually stopped working for like 5 seconds while the right one was working just fine. It ended up just kicking back in a few seconds later.
I have a second pair of AirPods coming in the mail today. I will test those out to see if they have the same problem.
I'm also having connectivity issues. When I'm at work the phone is sitting on my desk literally right beside me and I'm on LTE connections only. Sound would skip or drop for a few seconds especially when I'm getting an incoming message. I spoke with Apple Care's senior advisor and she said since it's a new product there might be glitches and they are not sure what's causing it, because it sounded more like a firmware issue.