to the OP, I've actually been doing the same thing and copying all of my personal DVDs to my computer so that I can then re-play them on my Apple TV, iPhone and soon to be iPad. Everyone has their own opinion on how to encode videos using Handbrake, this is mine. I've figured out that by selecting the iPhone preset and then upscaling the video, that will give me the same quality of video as iTunes does. pre-selecting the Apple Tv quality won't make it compatible with my iPhone or iPod nano, only Apple TV.
I did a lot of testing and I finally settled on one that makes me happy.
1. On the right Preset panel, Select iPhone & iPod touch.

2. I then manually changed the Quality to 1800 kbps. Also, make sure the Video Codec is set to H.264 (it should already be selected)

3.Then go into your Picture Setting on the top panel, and change the Anamorphic quality to Strict (this should give you the original quality of the video)

4. I personally use these filter settings to give me a nice clean image.

5. this is just a video preview of what the video will look like. Handbrake lets you preview 10-15-20-30 seconds of the video before you spend 2 hours encoding the video.
Final Result
