I did this for a brief period of time, and pronounced it LAME.
Screen mirroring will work but it will jack up your resolutions, so you're stuck with spanning. The primary problem is that Front Row will only work with the primary display. So if you work on your iMac, you'd then have to open up the Display pane in System Preferences, switch the menu bar to the TV, do your Front Row-ing, and then switch it back before you can do any work. The iMac screen will go black, since it's secondary, but it's a "bright" black with the backlight on, so it'll still be a distraction.
Granted, this was the case in mid-2006. Apple might have updated software to get rid of this problem -- ha ha ha... sorry, been awake too long.
So yeah, it's possible. Yeah, it's also a pain in the ass. I'd suggest something -- anything -- else. The Apple TV is fantastic, imho, and sounds like just what you're looking for. It is also nice because you can move your iMac away from the danged TV without screwing up your setup. Never underestimate the value that a wireless connection or CAT5E cable can provide over a bulky and expensive VGA/HDMI cable with a proprietary adapter on one end.