Seems that you can do it the same way you might delete an active reminder. I guess an option to auto-delete upon completion would make sense but I'm sure there are plenty out there, who like to have a record of completion as well.
At any rate, yes, there is a way to do it through your iPhone, iPad or iCal.