It depends on what you want the system for. You can't slap a new high-end GPU in an iMac. You can't slap 4 massive HDs in an iMac. The list goes on and on.
For many people, the Mac Pro is still the best choice, even though it's using 3+ year old technology.
I would wait till next release before buying new. The refurb upgraded to 6 core makes good economic sense unless you are prepared to build a Hackintosh instead or an iMac will meet your needs.
The current range of iMacs are due a refresh soon too mind.