yea If I were you and I was coming from a quad core pc (I am actually doing the exact same thing) there's no way I would've gotten a mini in the first place.
The iMacs only have C2D processors though from what I can tell. Yes, some are higher GHz, but no quads. Is the Mini really much slower than the lowest iMac?