Thats a good point in my eyes too. You surely can use AP at those shops you go to regulary and where you therefore know it works. But it is not reliable enough in case you want to leave your wallet at home and do a shopping trip in the city or farer away. So it is sadly only a nice tool but nothing more.
As a short work around, like flower shops that support these payment methods, I tell them what I‘d like to have, pay it first and let them do the service afterwards. Not that smooth and makes to explain the case sometimes... but as already told, I kept my wallet at Home for over two years... there‘re really only a few cases where there wouldn‘t be any other possibilty.