as per current testing, people in this forum have gotten up to 160GB of RAM working in the MP5,1 in OS X with no issues
via 5 32GB sticks, only old versions of OS X (Pre mavericks IIRC) have the 96GB Ram limit. (also people with Xserve3,1s have done 192GB of RAM via 12*16GB RDIMMs)
however no one has tested a full compliment of 6 or 8 32GB sticks in a 5,1 yet... it would be pretty epic if we can get 256GB of RAM working in a 5,1, and if LR-DIMMs work then theoretically we could max out at 512GB of RAM
(we also wonder how a single CPU W35xx/W36xx machine would react with 32GB DIMMs)