When I rip the Blu-Ray and then create an MKV, the file is still lossless (same quality as on the Blu-Ray disc). It isn't until I use Handbrake to convert the mkv that the file gets compressed further. I use the Universal setting in Handbrake at 1280x720... or 720p. However, because Avatar is a long movie, at 16:9 resolution (no cropping to save space) and lots of action and effects, the video was going to be quite large, so instead of using the constant quality setting, I set a target size of 3.75 gigs (or 3750mb).
The file turned out quite well. I only use the target filesize if I know the file will be over 4 gigs.