To get back to the original Thread:
I successfully installed an ATI 5670 MXM card that I took out of a broken imac (It's about the same size as the original GT120). Just swapped the GT120 cooler - didn't care much about ram-cooling.
The XSERVE runs High Sierra now very smooth - As it was made for it
So a graphics upgrade is worth it as long as you can get a MXM-card for a good price.
A Xserve 3,1 with 2x W5590 CPU's, enough of ram, a Samsung MZHPV512HDGL-0000 in a PCIe-slot and a better MXM graphics unit is still a beefy rack-server that can run High Sierra perfectly.
Although an efi-upgrade would be wonderful to increase CPU-support and get higher FSB-speeds.