there are three thermal zones inside of the current Mac Pro design.
a. 5.25 bays / Power Supply
b. 3.5" bays / PCI-e cards
c. CPU/RAM tray
b+ c are roughly 2/3 of the height. That's isn't moving to new requirements.
With modern equivalents for GPU PCI-e cards and CPU, the amount of power ( and hence thermal) doesn't go down at all. In fact may go up a bit since many of the top end GPU cards are now just as big, if not larger, power hogs these days. The need for just as large diameter fans that are present now will still be there if want to efficiently move just as much air (if not more) at the same relatively low levels of noise.
Apple could shift to 2.5" drives and move them to the Power Supply thermal zone. That would perhaps trim some height, but not to the point of some mini tower height.
And given that there are rarely more than a handful of Mac video cards you would ever buy, going with an onboard video solution may not be as weird as it sounds.
embedded video isn't even much of a trade-off if leave most of the slots available. If there are 3-4 slots then not really a big issue. Eventually though I suspect that even the Xeon E5 series will pick up models with GPUs in them. It is already true in the Xeon E3 series.
If they try and pawn off a single cpu Mac Pro due to limited thermal envelope in a smaller case, I think they would lose many higher end users.
I know my art and video guys would balk.
The Mac Pro cases is stretched over a pretty diverse set of users. Even collected together as an aggregate they are still relatively small subset of Mac users. The case to cover single and dual CPU package is probably just necessary to make the product worthwhile.
However, focusing on the CPU is misguided going forward. Currently and going forward there will likely be much more computational "horsepower" in two 200W PCI-e cards than there will be in the CPU "chamber" of a workstation. It isn't primarily just about the CPUs anymore and that is one of the problems with the current Mac Pro case design. How they managed he PCI-e zone is just as import, if not more so, now and into the future. 3-4 slots should be enough but 1 is likely too few and two is likely too inflexible given the highly diverse target audience.