I would 100% get the 5,1 if possible. These are the main differences between the 3,1 and 5,1.
CPU choices
The 3,1 has much older "Harpertown" Xeons, which consume more power and output more heat than the 5,1's "Bloomfield", "Gulftown" & "Westmere" Xeons. The best CPU config you could have in the 3,1 is 2x X5482 (4 cores, 4 threads - 8 cores total with no hyper threading) running at 3.2GHz (no turbo boost). The best CPU config for the single CPU 5,1 is the X5690/W3690 (6 cores, 12 threads with hyper threading) running at 3.46GHz (turbos up to 3.73GHz). While you have more physical cores on the 3,1, you only have 8 threads compared to 12 on the 5,1.
The 5,1 also has SSE4.2 support, which quite a few apps and drivers need on newer OS's.
Memory
The 3,1 uses DDR2 FB-DIMM memory, which is still way more expensive than the normal DDR3 ECC that the 5,1 uses. 3,1 supports up to 64GB (62GB is the max without instability and bugs) while the 5,1 supports up to 64GB (full amount usable). Also, RAM speed goes from 800MHz to 1333MHz (1066 on Bloomfield).
Expansion
The 5,1 is equipped with all PCIe 2.0 slots, while the 3,1 only has 2.0 on Slot 1 & 2 IIRC, 3 and 4 are the slower PCIe 1.0. Also, the optical drives on the 3,1 are IDE/PATA, while the 5,1 has SATA. This means that you can run SATA drives and SSDs in the optical bay to get 5 total drives (excluding PCIe drives).
GPU upgrades
The 5,1 is a much better option if you want to run more modern GPUs since support for RX series cards doesn't require any hacking to get working. This is because the new AMD drivers require SSE4.2, which the 3,1 doesn't have. Also, thanks to the superior CPU and memory performance, there will be less of a bottleneck running fancy cards like the RX 6000 series (6600, 6600 XT, 6800, 6800 XT, 6900 XT). The RX 580 is the best you can properly get running on the 3,1, which is in a whole different league to the 6600 XT, or even the Vega 56/64.
Running modern OS's
The 5,1 will have a much better time running new macOS versions like Monterey since the single-threaded performance of the 5,1's CPU is much superior to the 3,1 thanks to the higher clock speed and turbo boost supports. So at max, you're comparing 2.8GHz to 3.73GHz.
Running old OS's
The only real advantage to the 3,1 (apart from price) is that you can run Tiger and Leopard. However, on Tiger you are limited to a small number of ancient (and very slow) ATI graphics cards. On the 5,1, the minimum is Snow Leopard, which will be crazy fast with an SSD, W36xx/X56xx CPU, and a Radeon 5770/5870. I thought Snow Leopard was fast on my 17" '06 MBP and 13" 2010 MacBook, however with my 2011 13" with a Sandy Bridge-based i5, general OS performance is so much snappier and instant. You will feel this on the 5,1 as well. This is because these were the last generation of Mac hardware that ran Snow Leopard.
Can you tell I love the 5,1