The only thing I actually hear is the hard drive. If it wasn't for that, with the screen off I'd think it's off...
For fun I tried once to install smc fan control, to see how hard the fans could go. The answer is: very hard. Much louder than anything else than I've ever heard. Then , I realized that the iMac was unsupported by smc, and I had to shut it off. Since then, I've never heard the fans again...
Disclaimer: I don't use it the hard way, but it's extremly silent even when encoding a movie (you'd think it would get hot and use the fans, but it's not the case)
Like you, I also have a C2D MacBook Pro, and apart from the logically louder hard drive humming, the iMac (20", 2Ghz C2D) is more silent, since in normal use, the fans stay off (or at least quiet).
My advice? Go for it. It's much nicer to use for certain types of work.