When you create an album, and put a photo into it, the photo is not duplicated. As such, even if you have 5 albums with the same photo, there is still actually only 1 photo (from "All Photos"), it is just "referenced" into each of the albums. If you delete the photo from "All Photos", it will be deleted from all albums (as you have deleted the only copy of that photo). conversely if you delete a photo from an album, it will still be in "All Photos" and any other album to which you added it, as you have only deleted the reference to it in the album.
Think of an album as a collection of shortcuts to the photos stored in "All Photos".