I like all-in-ones mainly because of appearance. I have an iMac near the middle of my living room, seen from all sides, and I don't want boxes, dongles, cables, etc. cluttering it up (other than a single power cable which is unavoidable, unless one gets a laptop).
So in my opinion is it basically about looks. Sometimes all-in-ones can be a cost-effective complete package.
If you get an all-in-one then proceed to add external drives, dongles, docks, cables etc, it defeats the purpose and is then the worst of both worlds, IMO. Conversely, if you get an all-in-one and don't care about looks or where it is located, then there seems little point.