All they need to do is put the SoC on a card that can be swapped out to replace it and thus perform processor upgrades. If the SoC allows it, they could have more than one slot for such cards, just like many systems now have multiple CPU sockets.
There is practically no chance they could offer external memory separately from the processor and have it perform at anywhere near the speeds that the memory can reach while inside the chip - plus the type of memory that would allow for use by both the CPU and GPU with the level of efficiency that they are reaching would be quite expensive as an external card to begin with - so if they do offer the option to add memory outside the chips, I would expect it to be additional to memory on the chips themselves and to be used either NUMA style or (perhaps even more likely) as an earlier stage of swap space before going out to the even slower disks.
I see no issue with storage being made modular while still using the SoC, nor with including a few PCIe slots in such a design.
While the CPU, GPU and memory (at least the "fast" memory) will almost certainly be tied together, they could be "upgraded" together as a unit by making the SoC swappable, and everything else could still be "modular".
My guess for the new Mac Pro would probably be something along those lines.
