Photo Library will have ALL your photos in it and Camera Roll will have ALL your pix taken with your iDevice.
Any created albums essentially display a duplicate of the photo so you can organize the photos. But the original still resides in the Library or Camera Roll. You can delete from any location, but the permanent delete is only from Camera Roll or Library.
It may be that once a photo is in an album, you have to delete it from there first and then delete it from the Camera Roll.