Most used XEON CPUs (sold on eBay, for example) are pulled from non-Apple workstations. A used X5675 (assuming it's a used one you're looking at) should work in the Mac Pro, since Apple actually used that specific processor.
I purchased a used X5650 from eBay with no knowledge of what sort of system it came out of. Most likely not another Mac Pro. It works just fine in my 2009 MP.