Your iOS device has a sensor that knows how the camera is being oriented when you take a photo.
Internally, the image produced by the camera is not rotated to take account of this. Instead, some hidden information is added to the JPEG file that describes the orientation of the camera at the time the photo was taken.
Subsequently, when you view such an image file on an Apple device, it is able to use that information to figure out how it needs to rotate the image in order to display it 'correctly'. So, no matter which way up you hold your iPhone camera when you take a picture, or which way up you hold your iPhone when you view your pictures, they will always look the right way up.
Note that the hidden rotation information in the image file is not accessible to Windows users (Windows is blind to that information). It's a bit like me sending you a pack of physical photos in an envelope. On the back of each one it will say "this way up" or "upside down" with an arrow. If you lay them out face down, you will be able to orient them correctly and, when you turn them over, they will appear the right way round. But on a Windows device, you can't see the writing, so you are stuffed.
The e-mail client on OS X has a workaround for this: when you attach photos to an e-mail, you have the option of sending the photos in a Windows-friendly format. When you select this option, OS X will physically rotate the pixels in the original image, and remove the hidden rotation information. So, the image that you send is actually a different non-rotated image.
For the sake of simplicity, this option does not appear to be there in iOS.
By the way, for reasons that I won't go into here, Apple's way of defining images is technically superior to that used by Windows. It's a pity that Windows is blind to Apple's rotation data - MS could read and use it if they wanted to, but they don't.
