Well, I don't want to turn this into a display debate. What you need all depends on what you do, and this is an advantage of the Mac Pro over the iMac regardless of price. I need a workstation, but I don't need a a million dpi or what ever color accuracy, because maybe 80% of the time I'm in terminal or text editor. And when I do want to visually display what I work with its simple graphs and tables. So for me, a Mac Pro + a cheap display is certainly not a waste.
You may work in photography or film, and by all means get that $1000+ display, or get 6 of them, ultimately that's an advantage of the Mac Pro and also something that makes it difficult to do a simple cost comparison. It might be 50% more expensive once satisfying all of your needs, but the point is it CAN satisfy those needs, were as an iMac can not. Or at the very least you're not accounting for also paying for the display you need, as well as the iMac. Or the solution to use a 6 or 9TB RAID with an iMac vs a Mac Pro, or what ever else.
Exactly, Ivy Bridge will bring some improvements with heat/performance ratios, but not enough to really matter when it comes to an iMac. The problems really comes from GPUs. And what happens when more programs start making better use of GPGPU computing? Then the MacPro v. iMac gap will widen quite substantially.