BTW, I am not the "obvious" mac pro user. When I get home an iMac will fit most of my needs (well, some not ideally but workable). Looking at TCO convinced me though and things like expandability and a full-sized graphics card are bonuses.
I bought a Mac Pro in part with upgradability in mind, for sure. And in practice, it's been great; I have a machine that is nearly 6 years old and it still meets my needs quite well. My current uses are music production, video editing, photo management, and software development; so they are at least mildly resource intensive.
But just note that Apple has the tendency for dropping support for hardware, and that's been my biggest disappointment with the Mac Pro (even though it hasn't really mattered in practice). I have a good number of unsupported upgrades/hacks for my 2007 Mac Pro 2,1:
- ATI Radeon 5770 graphics card (official Mac version) - unsupported by Apple in the 2007 Mac Pro, but it works
- CPU upgrade - not officially supported, was more difficult to perform than a regular PC CPU upgrade, but it works great. These were CPUs that the Mac Pro never sold with BTW.
- Mountain Lion - Doesn't support the 2006-2007 Mac Pro 1,1 or 2,1 because of 32-bit EFI, but I used a hack involving Chameleon bootloader and it works well enough other than not being reliably put the machine to sleep
Now I'm hoping that this was a unique case because the 1,1 and 2,1 started with 32-bit EFI, and we won't have such a transition in the future. So the issues with what graphics card work (and are officially supported) will be less of an issue. And hopefully Apple won't have an excuse to drop OS X support on a machine that is still powerful enough. Note that they probably could have done a firmware upgrade to make it 64-bit EFI, or done some bootloader magic to emulate 64-bit EFI like Chameleon does for me.
As for CPUs: CPU upgrades are not officially supported by Apple; they expect you to stick with whatever you buy the machine with. In practice we can do what we want our machines, but that doesn't mean that Apple leaves things open enough that you can upgrade to a future CPU without hacks. There have been cases where the firmware was the only difference between Mac Pro revisions, and the firmware was the only thing keeping an older Mac Pro from using newer CPUs. People have come up with hacks to upgrade the firmware, but Apple definitely did not step up to provide such a thing, because they don't officially support CPU upgrades at all. Examples: 2010 Mac Pro is the same as 2009 except for firmware (and you can hack a 2009 to a 2010 to get later CPU support), same thing with the 2006 1,1 and 2007 2,1 I think.
So my point is: Yes, the Mac Pro is the most upgradable Mac you can buy from Apple. But the limitations on how far you can upgrade it with official support from Apple have historically dropped off fairly early in it's lifecycle, especially compared to what you might experience with typical PC hardware. And maybe it was unique to the 2006-2007 1,1 and 2,1, but Apple is not opposed to dropping OS X support for these machines before the hardware becomes irrelevant.
Now with the help of the community, with individuals who were willing to try out things to see what works, and to produce hacks to get things working, I have been able to upgrade my machine. The question is, is this going to continue to be the case? I don't think Apple is going to officially support more upgradability in the future Mac Pro, if history is any indicator. Hopefully the community (and the possibilities of the hardware itself) will be able to continue making up for this. The upgradability of a future Mac Pro has a certain uncertainty to it. That's the main thing that makes me unable to commit to buying a future Mac Pro (I might as well buy a MBP if the Mac Pro is not as upgradable as I'd like). Maybe once it's released we may find out more details that may sway me in a particular direction.