Just because you’re not bothered by it, doesn’t mean that a lot of others aren’t bothered by it - it’s also less about others being ‘bothered’ bynit, and more about losing a significant visual brand identifier.
The Home Button used to be this unique identifier, until some idiot in Cupertino decided it’s not needed, and that the notch would be new visual differentiator - which turned out to be a seriously dumb decision, since every Android product subsequently introduced their own notch. As a result, from on iconic representation point of view, iphone was no longer able to provide any visually unique identifier. Well done, Apple - NOT.
A newer, smaller punch-out will at least introduce a somewhat more unique (and more pleasant) appearance, but ultimately Apple needs to bring back the Home Button for a variety of reasons.
It’s not about the Notch, alone.