If you get inside the machine and look at it, it looks unlike anything else made by Apple. The board and architecture looks like 1978 more than 2013. They can easily reduce and combine portions of the board, compressing it down, etc.
The case is also wonderfully airy for cooling, but sans the opticals and perhaps with a different way to handle RAM, it can be compressed down to maybe half of what it currently is.
I think the one thing they just never can get around are the cards and the power supply. It needs a hulking PSU, and the PCI cards are what they are. The drives are clunky, but in a few years they might be tossed for SSD and make the whole system smaller again.
Personally, a smaller Mac Pro would be awesome. However, I don't need to tote it around, so it isn't a selling point with me.