I haven't seen one thread yet on if the GPUs in the Mac Pro are swappable, and I haven't seen a firm answer either way. Let's discuss! I'm hoping a reviewer will get an answer.
Best answer so far from CNet:
http://reviews.cnet.com/desktops/apple-mac-pro-2013/4505-3118_7-35781456-2.html
"Could be swapped" is a good start to an answer.
Best answer so far from CNet:
http://reviews.cnet.com/desktops/apple-mac-pro-2013/4505-3118_7-35781456-2.html
Those RAM slots are the most user-accessible component inside, although Apple says the SSD and graphics cards could be swapped out as well, but it certainly wouldn't be as easy as it is on a traditional desktop tower.
"Could be swapped" is a good start to an answer.