I've just bought a couple of PCIe cards from SEDNA (they have an online shop) which will mount a 2.5" SATA drive in one case and a pair of mSATA SSDs in the other. I think the SATA-mounting model is the most interesting. Yes, it works with OSX Server and (I haven't tested it) will probably boot OSX (Startup PrefPane thinks so). This is a very fast interface compared with SAS and probably much, much cheaper per Gig as well. It support drives up to 2TB.
Google "sedna pcie 6g" - New Egg stock them.
There's a SATA 6 connector on the end of the card but I can't work out if this is an alternate access to the drive (Wahoo! Dual pathing!) or a daisy-chan for another drive. Daily Motion's YouTube vid shows a standard SATA power connector next to the data connector but mine just has the artwork and the solder pads. Austerity model?
I have been using a USB3 card with problematic (patched) drivers to add external fast, bulk storage to the Xserve but with no guarantee of speed. This seems to be even faster at the cost of some flexibility.