Yes, you are missing a key point about iPhoto....
iPhoto is a Digital Asset Manager (DAM) which means it is different than just nested folders.
With a DAM (Aperture, Lightroom, Capture One, and a couple others are also DAMs) your photo is either parked inside a hidden folder structure (iPhoto) or often moved to a visible date ordered folder structure when you 'import' your photos. What an 'import' means, at the fundamental level, is that the DAM creates an entry in its database. After that all the edits you make, all the keywords you add, all the Albums (or Collections) you move the image into are simply notations in the database record for that image. The image itself never physically moves or changes.
When you are in iPhoto and add an image to an Album, all that happens is that iPhoto makes a note that the photo appears in that Album. There is no 'real' Album that exists.. it is virtual. Which means you can put the photo into as many Albums as you want, and each instance is actually pointing at a singular original image tucked safely way. There is no storage penalty for having multiple copies of an image in several Albums. Edits and other changes to an image in one Album are reflected wherever that photo appears. But it still really exists in just one place. Deleting a photo from an Album only means that you are deleting the reference for the photo in that Album in iPhoto's database. The photo itself is not being deleted.... it is just the database entry being changed to eliminate the notation that the photo can be found in a particular Album
However, when you Delete the photo from 'Photos' you are actually mucking about with the real, actual photo. Delete the photo and you Deleting the original image, that is safely tucked away. And iPhoto then deletes the references to that photo in the database since those notations no longer point at a real photo. Put it back, and iPhoto is smart enough to rebuild the database record.
Hope this helps. Search this Photo forum on my name to see lots more of what I've written, along with what others have written about this topic. There is a wealth of knowledge there.
Luck.