Current (2015) Mac Pro Questions

[1215]

Where can I go to learn about these things? I've had no luck at the Apple Stores. I'm wondering specifically about virtualization and serviceability.

There are is a bunch of fairly basic information regarding VMware's ESXi 5.5u2 and 6 supporting the MacPro, including ESXi 6.0 supporting the MacPro OOTB, but it is still early so the information is spotty and not always accurate, and additionally, VMware usually releases a bunch of patches and updates so things are ever changing and two-week old information may not be accurate currently.

With regards to serviceability, I'd love to buy a couple (two or three) barebones MacPro's and buy the CPU/RAM/Graphics myself --but I know that is not possible. As an alternative, I would like to purchase a couple base model MacPro's and upgrade them myself. How do I find out about things like the form factors and exact interface specifications of the SSD and GPUs as well as if the CPU is considered serviceable (as it is not marked "User Accessible") on Apple's website? Further, does Apple ever do BIOS updates or OS releases/patches which allow for larger memory modules to be used --or is the max of 4x16GB memory modules a hardware limitation?

I am researching whether the MacPro would be a suitable alternative to other machines I am considering as ESXi hosts. They would have one and only one purpose as ESXi development or lab machines in a non-rack environment, to replace tower (non-rack) servers running dual Westmere CPUs. The current servers are several years old and starting to experience failures such as power supplies, motherboards and RAID controller cards.

Thanks for any pointers or information.

 

[AidenShaw]

Dude - get a Dell

Where can I go to learn about these things? I've had no luck at the Apple Stores. I'm wondering specifically about virtualization and serviceability.

There are is a bunch of fairly basic information regarding VMware's ESXi 5.5u2 and 6 supporting the MacPro, including ESXi 6.0 supporting the MacPro OOTB, but it is still early so the information is spotty and not always accurate, and additionally, VMware usually releases a bunch of patches and updates so things are ever changing and two-week old information may not be accurate currently.

With regards to serviceability, I'd love to buy a couple (two or three) barebones MacPro's and buy the CPU/RAM/Graphics myself --but I know that is not possible. As an alternative, I would like to purchase a couple base model MacPro's and upgrade them myself. How do I find out about things like the form factors and exact interface specifications of the SSD and GPUs as well as if the CPU is considered serviceable (as it is not marked "User Accessible") on Apple's website? Further, does Apple ever do BIOS updates or OS releases/patches which allow for larger memory modules to be used --or is the max of 4x16GB memory modules a hardware limitation?

I am researching whether the MacPro would be a suitable alternative to other machines I am considering as ESXi hosts. They would have one and only one purpose as ESXi development or lab machines in a non-rack environment, to replace tower (non-rack) servers running dual Westmere CPUs. The current servers are several years old and starting to experience failures such as power supplies, motherboards and RAID controller cards.

Thanks for any pointers or information.

Standard x64 servers from HP or Dell are much more practical as ESX hosts, unless you want to 'legally' run Apple OSX as a guest.

HP and Dell both produce custom ESXi installation .ISOs with the HP and Dell drivers and management tools pre-installed. (These are checked by VMware and downloaded from VMware's site.)

No worries about memory (twelve 32 GiB DIMMs per processor socket, with 64 GiB being tested), support or upgrades. Dozens of terabytes of disk internal. Both HP and Dell have SD card slots on the motherboard, so you can install ESXi on an SD card and save all the disks for VMFS.

Using an MP6,1 is pushing a round peg into a square hole.

 

[1215]

Using an MP6,1 is pushing a round peg into a square hole.

Thanks for the reply. I somewhat agree with you however ESXi 6 supposedly runs OOTB on a MP6,1... allegedly.

The reason why I like the MacPro is it's form factor. It's small and much more aesthetically pleasing than the 4x tower machines I have now. I could put the 2x Mac Pro's on a bookshelf and let them sit out in the open vs. the heat and noise and mess of cable and wires that is the current situation.

I have no need (or desire) to virtualize OSX at this point. I'm currently running four Dell T7500 Precision Workstations (though not technically a server, it's close enough in many regards) in a ESXi 5.5u2 cluster. Each has dual Xeon 5680 CPUs and 72GB RAM. I would guess that my current Dell hosts are slower but fairly comparable to the MP6,1 in that each has 12 cores and DDR3... Except that the current situation is massively large, consumes power like nobody's business, kicks out heat enough that I open the windows in the winter and add a window AC (central air) in the summer.

I fell into the Dell machines for a price that was too good to be true. They perform wonderfully however they are very overkill for my needs. The main reason why bought that chassis size was for the 4x externally accessible 5.25" bays and 4x internal 3.5" bays. After three years of running these things I realized that I needed the space MUCH more than I needed the CPU. We were using all of the RAM to create RAMDRIVEs on the VMs so that we didn't hammer the local disk arrays with random I/O requests however while not using the RAMDRIVEs I am on average using less than 50% of each Host's RAM.

When I ran out of disk space recently I purchased/built a 100TB NAS. (Without thunderbolt which will present its next challenge because of Apple's Direct-Attach mentality.) With the NAS in place I was able to shut down two of the ESXi hosts to save power and only kept the second one running mostly for redundancy.

I work from home and all of this stuff is in my house. Not only in my house it's in my office where I work every day. My thought was that if I could find a way to connect to the NAS via thunderbolt, I could run everything on one or maybe two MacPros. Not only would that chop my physical footprint by about 80%, there would be less power used and heat produced, and, if I could find a way to connect to the NAS via Thunderbolt, I could run a 16-port fanless switch and get rid of the noise.

Knowing very little about Macs (I use an old MBP laptop, all the rest is PC-based), my thought was that I could put the two MacPro's on a bookshelf, direct connect each of them to the NAS and solve a lot of problems. My alternative is to spend about $3,500 repairing and upgrading the existing setup, or, to spend about $8,000 buying newer hardware... or buy the Mac Pro's.

For the record... we have a racked production environment in addition to all of this.

 

[AidenShaw]

Thanks for the reply. I somewhat agree with you however ESXi 6 supposedly runs OOTB on a MP6,1... allegedly.

http://www.virtuallyghetto.com/2015/01/thunderbolt-storage-for-esxi.html

Some storage tips.

http://virtualrealization.blogspot.com/2015/02/mac-pro-61-rack-environment-on-vmware.html

Pointer to 3rd party 128GiB RAM kits for MP6,1 .

 

[1215]

http://www.virtuallyghetto.com/2015/01/thunderbolt-storage-for-esxi.html

Some storage tips.

http://virtualrealization.blogspot.com/2015/02/mac-pro-61-rack-environment-on-vmware.html

Pointer to 3rd party 128GiB RAM kits for MP6,1 .

Thanks for the links. I had never seen the second link referencing the 128GiB RAM and I'll keep it in mind.

Sounds like I'm going to have to do some construction work in the basement... so I can partition off a proper telco closet that can house at least a half rack. I don't see any point in spending such a premium to shove a few Mac Pros into 4u chassis when I can just as easily rack the Super Micro & Dell hardware I already own.

What does Apple do at its own datacenters? Please tell me that iTunes and Apple.com isn't run off of Mac Pros in 4u enclosures???

 

[AidenShaw]

Thanks for the links. I had never seen the second link referencing the 128GiB RAM and I'll keep it in mind.

Sounds like I'm going to have to do some construction work in the basement... so I can partition off a proper telco closet that can house at least a half rack. I don't see any point in spending such a premium to shove a few Mac Pros into 4u chassis when I can just as easily rack the Super Micro & Dell hardware I already own.

What does Apple do at its own datacenters? Please tell me that iTunes and Apple.com isn't run off of Mac Pros in 4u enclosures???

Probably a better choice. The 5000 series CPUs in your older servers do use a lot of electricity even when idle.

If you got the MP6,1 - you'd spend a lot of money for two GPUs that would never be used under ESXi.

Note also that Intel has been doing a great job of reducing the power used at idle - but if the CPUs are busy they're still going to need a lot of electricity. The full load power requirements haven't gone down much over the last few CPU generations.

The webserver in the BIOS of my HP ProLiants has a nice power meter. Attached is the display from a mostly idle ESXi host - average under 170 watts for a dual E5-2650v2 (8 physical cores per socket, 16 physical cores total), 256 GiB RAM (16x16GiB 1866).

The 5000 series Xeons are electricity pigs (when idling) by comparison, and are missing some virtualization extensions that are really important.

 

[1215]

The 5000 series Xeons are electricity pigs (when idling) by comparison, and are missing some virtualization extensions that are really important.

Maybe this should have been an email, not a forum discussion...

I have a boatload (probably 30+) of Dell T5500, T7500 and R5500 (T = Tower and R = Rack) machines. Part of my garage is full of stripped out chassis from a couple upgrades or conversions we did a while back.

I know exactly what you are saying... I already have the R5500 chassis so racking them is easy... its just that they use a ton of power --they are usually running jobs 24/7 but yes, even at idle they aren't great... and they are dead at ESXi 5.5u2 (not esxi6 compliant). Additionally they are old and slower than new stuff.

:sigh... there goes that pipe dream. I'll probably end up spending an equally stupid amount of money on a dual 8/10 core server and sit on that for a few years. If I could only capitalize on the depreciation of CPU values...

Is there anything I can use as a desktop or non-rackmount lab/development setup within or less than the budget of a MP6,1??

 

[ScottishCaptain]

Where can I go to learn about these things? I've had no luck at the Apple Stores. I'm wondering specifically about virtualization and serviceability.

There are is a bunch of fairly basic information regarding VMware's ESXi 5.5u2 and 6 supporting the MacPro, including ESXi 6.0 supporting the MacPro OOTB, but it is still early so the information is spotty and not always accurate, and additionally, VMware usually releases a bunch of patches and updates so things are ever changing and two-week old information may not be accurate currently.

With regards to serviceability, I'd love to buy a couple (two or three) barebones MacPro's and buy the CPU/RAM/Graphics myself --but I know that is not possible. As an alternative, I would like to purchase a couple base model MacPro's and upgrade them myself. How do I find out about things like the form factors and exact interface specifications of the SSD and GPUs as well as if the CPU is considered serviceable (as it is not marked "User Accessible") on Apple's website? Further, does Apple ever do BIOS updates or OS releases/patches which allow for larger memory modules to be used --or is the max of 4x16GB memory modules a hardware limitation?

I am researching whether the MacPro would be a suitable alternative to other machines I am considering as ESXi hosts. They would have one and only one purpose as ESXi development or lab machines in a non-rack environment, to replace tower (non-rack) servers running dual Westmere CPUs. The current servers are several years old and starting to experience failures such as power supplies, motherboards and RAID controller cards.

Thanks for any pointers or information.

ESXi 6.0 seems well supported on the MacPro6,1, but it's far from optimal. We've got a pair of them running like that at our office, but I think it's a bit of a pain in the ass. The firmware seems to constantly forget about what and where it's supposed to be booting from, which means there's a 50/50 chance I have to hookup a keyboard/monitor/mouse just to hold the ALT key and tell it to boot from ESXi if the system is shut down for whatever reason. Likewise, the graphics cards are all but useless- I've been fiddling around with VT-d & passthrough, but have been unable to find a stable configuration that actually works (with OS X or Windows guest VMs). It seems like a "fun" experiment, but that's about it, and I certainly wouldn't recommend it for any kind of production use.

Don't bother trying to upgrade the machines yourself. Parts are prohibitively expensive and rare, it's almost impossible to find the "B" model graphics card (there's two of them, A and B, and they're mirrored) anywhere for a reasonable price. The SSDs are proprietary Apple tech, nobody really knows much about them. Technically, the CPU is upgradable but the LGA socket is 100% unprotected (there's no retention mechanism at all), so you need to be REALLY careful doing this.

Apple doesn't have that great of a record updating EFI firmware. Most of their machines see one or two critical updates over their entire lifespan until EOL, and those patches almost never add features or support. If you're going to go down this route, then I would expect that your machine will never support anything other then what it does OOTB. For all intents and purposes, the MacPro6,1 already seems to be "abandoned" as far as Apple is concerned (lack of firmware updates, lack of hardware updates, broken drivers under OS X and Windows, etc).

If all you're after is the form factor, I'd recommend you look at building some of your own ITX based systems instead. I've done this a few times and there are a couple of ITX (or Micro ATX if you want one size up) boards that support IPMI and occasionally iKVM (usually via SuperMicro or Tyan). Most of those boards also support ESXi without any fuss. You'll probably get a better system that lasts longer for less too.

-SC

 

[AidenShaw]

ESXi 6.0 seems well supported on the MacPro6,1, but it's far from optimal. We've got a pair of them running like that at our office, but I think it's a bit of a pain in the ass...

Do you have to disable EPT virtualization?

The Ivy Bridge CPUs (E5-v2) have a bug which can lead to page table corruption, which causes crashes in the VMM. (The guest OS crashes with various errors, depending on which part of the guest kernel trips on bad memory first.)

A firmware update can workaround the bug, but if Apple hasn't applied that update....

 

[ScottishCaptain]

I haven't explicitly disabled it, but at the same time I haven't ran anything that would require it (nested hypervisors and what not). I don't think EPT has anything to do with passthrough, though if it did that would explain why I can never get that to work properly- the guest OS either craps out, or ESXi PSODs and takes down the entire node.

To the best of my knowledge, I haven't seen any firmware updates that address this. It's possible someone else in the lab applied them without my knowledge, but we haven't had OS X running directly on those machines yet so there's no way to run software update or the MAS. I haven't seen anything pop up on Apple's support website either.

Edit: I just moved a guest ESXi VM over to the nMP and booted it up. It seemed fine until I brought up a VM underneath that, at which point the guest pink screened. Subsequent attempts are yielding some really bizarre behaviour. Either the guest craps out, or the VM underneath that crashes at a random location. Actually, it looks like the entire system has hung up now and has stopped responding in vCentre... Yup, it's dead. Time to go reboot.

-SC

 

[AidenShaw]

I haven't explicitly disabled it, but at the same time I haven't ran anything that would require it (nested hypervisors and what not). I don't think EPT has anything to do with passthrough, though if it did that would explain why I can never get that to work properly- the guest OS either craps out, or ESXi PSODs and takes down the entire node.

To the best of my knowledge, I haven't seen any firmware updates that address this. It's possible someone else in the lab applied them without my knowledge, but we haven't had OS X running directly on those machines yet so there's no way to run software update or the MAS. I haven't seen anything pop up on Apple's support website either.

Edit: I just moved a guest ESXi VM over to the nMP and booted it up. It seemed fine until I brought up a VM underneath that, at which point the guest pink screened. Subsequent attempts are yielding some really bizarre behaviour. Either the guest craps out, or the VM underneath that crashes at a random location. Actually, it looks like the entire system has hung up now and has stopped responding in vCentre... Yup, it's dead. Time to go reboot.

-SC

I was getting guest crashes (running Workstation on a Win7 E5-1650v2) running simple guests. EPT is a performance boost for simple guests.

 

[gapatsfan]

Where can I go to learn about these things? I've had no luck at the Apple Stores. I'm wondering specifically about virtualization and serviceability.

There are is a bunch of fairly basic information regarding VMware's ESXi 5.5u2 and 6 supporting the MacPro, including ESXi 6.0 supporting the MacPro OOTB, but it is still early so the information is spotty and not always accurate, and additionally, VMware usually releases a bunch of patches and updates so things are ever changing and two-week old information may not be accurate currently.

With regards to serviceability, I'd love to buy a couple (two or three) barebones MacPro's and buy the CPU/RAM/Graphics myself --but I know that is not possible. As an alternative, I would like to purchase a couple base model MacPro's and upgrade them myself. How do I find out about things like the form factors and exact interface specifications of the SSD and GPUs as well as if the CPU is considered serviceable (as it is not marked "User Accessible") on Apple's website? Further, does Apple ever do BIOS updates or OS releases/patches which allow for larger memory modules to be used --or is the max of 4x16GB memory modules a hardware limitation?

I am researching whether the MacPro would be a suitable alternative to other machines I am considering as ESXi hosts. They would have one and only one purpose as ESXi development or lab machines in a non-rack environment, to replace tower (non-rack) servers running dual Westmere CPUs. The current servers are several years old and starting to experience failures such as power supplies, motherboards and RAID controller cards.

Thanks for any pointers or information.

 

[gapatsfan]

What is the maximum number of (VMs) Guest Mac OS X we can provision on a single MacPro Server ?

 

[AidenShaw]

What is the maximum number of (VMs) Guest Mac OS X we can provision on a single MacPro Server ?

It depends...

On disk storage: Each VM is a full installation, so if the standard installation uses 30 GB, the max is <total VM storage>/30.
On disk IOPs: Often the virtual disks are stored in a single filesystem, so the IO demands of the VMs are all directed at the same disks. VMs with heavy IO loads will saturate the filesystem quite easily.
On RAM: Each VM will use about 1.2 times its configured memory. If you run 4GiB VMs, then the max is <total free host RAM>/5.
On CPU: Quite variable. Obviously you can't have more active threads in VMs than you have logical cores. Often VMs will be idle most of the time, so it works out OK. Note: VMware does gang scheduling. If a quad core VM needs compute time, then 4 logical cores are given to the machine. If the VM is running a single-threaded job, then 3 of the 4 logical cores are idle - but unavailable for other VMs.

All of these issues are independent of the type of guest OS.

 

[gapatsfan]

It depends...

On disk storage: Each VM is a full installation, so if the standard installation uses 30 GB, the max is <total VM storage>/30.
On disk IOPs: Often the virtual disks are stored in a single filesystem, so the IO demands of the VMs are all directed at the same disks. VMs with heavy IO loads will saturate the filesystem quite easily.
On RAM: Each VM will use about 1.2 times its configured memory. If you run 4GiB VMs, then the max is <total free host RAM>/5.
On CPU: Quite variable. Obviously you can't have more active threads in VMs than you have logical cores. Often VMs will be idle most of the time, so it works out OK. Note: VMware does gang scheduling. If a quad core VM needs compute time, then 4 logical cores are given to the machine. If the VM is running a single-threaded job, then 3 of the 4 logical cores are idle - but unavailable for other VMs.

All of these issues are independent of the type of guest OS.

 

[gapatsfan]

Sorry, My question was related to Apple EULA limitation for MacPro..

My dumb Apple Rep referred me to the EULA below and said we are not allowed to install any Hypervisor on a MacPro and said .... I was violating Apple License agreement.

https://www.macrumors.com/2011/07/0...le-copies-on-the-same-machine-virtualization/

(iii) to install, use and run up to two (2) additional copies or instances of the Apple Software within virtual operating system environments on each Mac Computer you own or control that is already running the Apple Software.

I feel it is related to installing Vmware Fusion on a MacPro and installing two copies of Max OS .