When you do as Peyton suggests, make sure you are in Library in the Source list, not in a photo album. Deleting from the Library really deletes the photo (actually, moves it to the iPhoto trash, which you can empty afterwards) and removes it from all photo albums it is in as well.
Deleting from a particular album (your own collections that you create and name in the Source list) removes it only from that album but leaves it in your Library.
The Library represents your full collection and the real photo images on the disk. The albums are simply aliases/shortcuts/pointers to collections of photos that are subsets of your Library.