No, it's not that difficult (which is why I'm not returning my APM), but it's something I'll have to remember to do once a week even if I'm not using them (sometimes I get into a "speaker mood" where I don't feel like listening through headphones for a while).
Then there's the issue of constant (and unnecessary) drain/recharge cycles that will shorten the lifespan (with the alternative being keeping it plugged in all the time, which also carries a risk, as they say keeping li ion batteries at 100% all the time is not good for them).
It's not unlike having a car that you can't shut off the engine. You park it, and the engine just idles all night, consuming gas for no purpose other than to keep the car ready to go at a moment's notice. Thanks, but I'd rather have the ability to turn the headphones off when I know I'm not going to be using them for a while (and no, I don't want to fiddle with that ridiculous looking case, I don't even want to have to look at it).
If the APM weren't as good as they are, I'd be returning them for this silly omission.