Unless there's a reason that next-gen PCIe would strongly influence the "design" direction of the next MP, I still don't see how it matters.
I'm going to reorder this a bit to cover the points.
The actually "real life" user base for the Mac Pro (i.e. not MacRumors) does not care whether it's PCIe 3, 4 or 5.
The fact that doesn't start with a 2 ( 2, 3, 4, or 5) means there actually is a time launched component to this that does matter. I'm not saying at all that they should be waiting long periods of time for 4 ( or 5), but launching with is perceived to be contemporary PCI-e version number with the competitors does matter.
By the end of 2019 PCI-e v4 will probably be considered 'contemporary' for many folks making workstation buys that span Q4 '19 - Q4 '20
PCI-e v5 yes. Outside of the folks doing extreme I/O > 200Gb/s range (heavily occupied by Infiniband ... for which there are no macOS solutions). That is probably in the "don't care" range for several years going into the future. The pragmatic pice per port in that range is so high that vast majority of workstation deployments aren't going to engage that. And the storage cards in that bandwidth range also will be in the stratospheric range too for a couple of years.
I.e., would PCIe 4 or 5 change whether it has zero PCIe slots, 1 PCIe slot or multiple PCIe slots? Would it change whether it has serviceable drive bays? Would it change the form-factor of the computer? If it doesn't affect those things, then I'm not seeing how it matters.
Several factors but a few.
First, Around 2012 the bandwidth of Intel's QPI interconnect between CPU packages was approximately 230 Gb/s. PCI-e v4 x16 bandwidth is around is 256 Gb/s. That's actually higher. The latencies are probably better for QPI but if people got usable multiple package performance out 2010-2012 dual package Mac Pros then using the PCI-e v4 bus as a interconnect between CPU and GPU won't be horrible.
PCI-e v5 will be even higher bandwidth and incrementally closer latencies. ( especially if keep mainly to a point-to-point bus traffic. i.e., the other more mundane stuff of on separate PCI-e bus nexus. )
So the pre 2013 dire need for two CPU packages? Not really. There would be a NUMA cost in accessing 'far' memory but in more cases that would not be so high as to always have to make copies of 'everything' to avoid it. That can feed into overall system design.
Second, PCI-e 4 pragmatically shrinks distance to "furthest slot". So if if two x16 slots spaced at 2-3" widths aparts your are far more likely to run out of "room" for the the others to also be v4. So probably going to get slot segregation. You have differing distance limits. That will influence slot placement, which if doing holistic integrated design will have impact. v 4.0 , and to lessor extent v5.0 , are making a "less distance for greater speed" trade off. The move to 4-5 isn't going to cover everything; just some narrower set of workloads.
The runs substantively opposite of Thunderbolt which makes a "more distance for minor latency tradeoff" an objective.
Thunderbolt standards is also quite likely not gong to chase after PCI-e v4.0 for a long time. ( addiction to highly affordable copper wire cabling solutions, desire for stability multiple implementors will want, etc. ) . PCI-e v3.0 x8 is already twice the bandwidth. PCI-e v4.0 x8 will be 4 times. PCI-e v5.0 x8 will be 8 times. Apple's bubble that they could take away all the slots and just pour on Thunderbolt ports to the Mac Pro will be even less tenable in the PCI-e v4-5 era than it was in the PCI-e v3.0. Apple not having at lest one open slot would be in the drowning in drinking kool-aid mode on their part.
A design with a balance between Thunderbolt and open slots should be a major influence on the design. Neither PCI-e v4-5 nor TB v3+ will be meant to cover "everything' . So the need to find a balance should be so obvious at this point that really would be hard pressed to miss it even if had a major commitment to Thunderbolt across the product line.
Three, access to more cost effect higher I/O. ( presuming get a trickle down effect of new 100+ Gb/s I/o leads to cheaper stuff down the product line price cuts trickle down through 40 , 20 , 10 ) makes NAS/SAN/'distant NAS' storage all that more practical and more widespread.
An example at the higher end. ( not that Apple is planning to be firmly entrenched there (the OS foundation for this I don't think is going to show up any time soon), but where the competition is going...)
"...In an era where it is possible to get 100GB/s or (later this year) 400GB/s, the bandwidth explodes and putting storage in one place and making it look local in terms of latency is possible. And therein lies the hook to what Mellanox, Pure Storage, and startups like Excelero are doing. They are pushing thin ways to pool resources and make remote storage look local with the ability to configure it on the fly according to what each server needs. ... "
https://www.nextplatform.com/2019/02/06/why-nvme-will-flash-forward-in-2019/
Apple brining 1-2 10GbE base T ports to that fight is like bringing a sharp stick to a gun fight. Apple has inherent conflicted overall system design goals if capping out at 10GbE and limiting internal storage . When other workstations are in the 25+ range and on fiber ( or moving up to next gen Base T 25-40Gb/s range. ) they'll have an upper hand on nonpedestrian network storage also. An open slot to keep them more competitive would be a prudent move. It is doubtful though Apple will look to see bulk storage not drifting toward more distance though.
Network storage wanting to expand will push the internal storage in the even faster bandwidths ( so will see PCI-e v4-5 Flash storage showing up at high price points but not so high as to loose too much ground to the network folks. )
Relationship to overall system design. Apple's "one and only one" storage drive for the Mac Pro specially should be dropped. However, that isn't necessarily going to be the old legacy commitment to 3.5 drive bays. SATA is even slower than Thunderbolt. The 'gap' on SATA is growing even higher. HDD speed are relatively going nowhere, so that probably isn't where the local storage performance 'puck' is going to ( so Apple probably won't be skating to there. ).
That really is cheap. Might be the result of Nvidia not allowing Titans in data centers anymore. Two of those in a desktop for $5k is a very reasonable price though for some applications.
