MP 1,1-5,1 Do any of you run Proxmox on your Mac Pro 5,1?

[Project Alice]

Hey all, I couldn’t really find a direct answer to this so I figured I’d make a post.

I want to setup my spare 4,1 > 5,1 to run ProxMox, virtualizing what is currently two different physical machines. Pfsense, and TrueNAS. TrueNAS also is hosting Plex, which would be getting it’s own VM under ProxMox.

I have a couple concerns I’m hoping you guys will know about. I’m going to be dropping two X5675s into the Mac Pro. ProMox obviously needs VT-x, and one of the reasons I want to do this is for VT-d. I want to make sure the Mac Pro’s EFI will enable both of those for ProxMox.
In the past I have had problems with Apple’s EFI with other OS’s and usually end up booting with CSM, which isn’t ideal.

So the main two questions are;
1: Will ProxMox work properly booting off a 2009 “5,1” Mac Pro, or is CSM going to be required?
2: under whichever EFI or CSM, will VT-d be working? I will be passing through a GPU to my Plex VM (something I’m unable to do on my current TrueNAS FreeBSD setup) and I might also pass through a physical NIC to pfsense.

 

[startergo]

will VT-d be working?

VT-d normally works on the cMP5,1. I don't see a reason why it wouldn't on 4,1 > 5,1. X5675 supports VT-d.
Please share you progress. It would be interesting to see.

 

[Project Alice]

VT-d normally works on the cMP5,1. I don't see a reason why it wouldn't on 4,1 > 5,1. X5675 supports VT-d.
Please share you progress. It would be interesting to see.

Good to know, thank you. I will update this thread for sure.

It’ll be a little bit though. I had the PSU go out last year on my main 5,1 so I took it out of my other one. I ordered a PSU on eBay a couple days ago, so when it arrives I’ll get it installed and start setting up the Mac. It still has the factory quad cores in it so I’ll need to get a pair of delidded X5675s too but I can at least install ProxMox and get it partially setup when the PSU gets here.

 

[Cerlak]

I recently was given an old cMP and am currently running Proxmox and High Sierra on it (not at the same time obviously). I have multiple LXC containers and VMs running and it seems to work pretty well. Current items I’m running include:

• Ubuntu 20.04 server (VM)
• Ubuntu 20.04 desktop (VM)
• Windows 10 (VM)
• PiHole (LXC)
• NodeJS (LXC)
• Gitlab (LXC)

It’s a 4,1 that I upgraded the firmware to 5,1 and installed 2 x5675 (delidding was a bit of a pain, but not too bad if you’re patient). I also added 48GB of 1333mhz RAM.

I will say I’m a total Proxmox noob and just followed standard tutorials to get things working as an experiment, so I don’t really know about optimal configurations or anything like that, but I can tell you it will work.

If you have any questions I can attempt to answer them.

 

[startergo]

I recently was given an old cMP and am currently running Proxmox and High Sierra on it (not at the same time obviously). I have multiple LXC containers and VMs running and it seems to work pretty well. Current items I’m running include:

• Ubuntu 20.04 server (VM)
• Ubuntu 20.04 desktop (VM)
• Windows 10 (VM)
• PiHole (LXC)
• NodeJS (LXC)
• Gitlab (LXC)

It’s a 4,1 that I upgraded the firmware to 5,1 and installed 2 x5675 (delidding was a bit of a pain, but not too bad if you’re patient). I also added 48GB of 1333mhz RAM.

I will say I’m a total Proxmox noob and just followed standard tutorials to get things working as an experiment, so I don’t really know about optimal configurations or anything like that, but I can tell you it will work.

If you have any questions I can attempt to answer them.

What is preventing you from running macOS in proxmox too?

 

[Cerlak]

What is preventing you from running macOS in proxmox too?

Nothing I suppose. ?‍♂️ I already have 3 other Macs in the house so I don’t really need too.

 

[Project Alice]

I recently was given an old cMP and am currently running Proxmox and High Sierra on it (not at the same time obviously). I have multiple LXC containers and VMs running and it seems to work pretty well. Current items I’m running include:

• Ubuntu 20.04 server (VM)
• Ubuntu 20.04 desktop (VM)
• Windows 10 (VM)
• PiHole (LXC)
• NodeJS (LXC)
• Gitlab (LXC)

It’s a 4,1 that I upgraded the firmware to 5,1 and installed 2 x5675 (delidding was a bit of a pain, but not too bad if you’re patient). I also added 48GB of 1333mhz RAM.

I will say I’m a total Proxmox noob and just followed standard tutorials to get things working as an experiment, so I don’t really know about optimal configurations or anything like that, but I can tell you it will work.

If you have any questions I can attempt to answer them.

This will be my first time with proxmox too.

The main things will be passthrough NICs and the GPU in plex. Pfsense I think will need it’s own PCIe NIC, but the others should be fine on the MPs built in ports. I did install proxmox the other day as a test and it installed fine using EFI so that’s good.

I still haven’t gotten my SSDs in the mail yet and I need to order a pair of Xeons.

 

[Cerlak]

Yeah, my next step is to pass through the gpu to the windows VMs and rebuild the whole thing with an ssd.

 

[elvisimprsntr]

@Project Alice

I bought a used x86_64 QNAP NAS off evilBay and installed Proxmox. Suits my needs for running VMs.

I have a separate dedicated appliance for pfSense. The folks at iXsystems will advise against virtualizing TrueNAS

 

[Project Alice]

I have a separate dedicated appliance for pfSense. The folks at iXsystems will advise against virtualizing TrueNAS

Yeah I’ve read that. I think it’ll be fine with a type1 though.
I just re-read through this.
I should be able to passthrough the Mac Pros onboard SATA controller to give TrueNAS direct access.. however, do you know how many controllers it has? Which ports are on which controllers? I was going to boot Proxmox off an optical bay SSD, and have VMs in another optical bay SSD; my 4 TrueNAS data drives in the HDD bays. Are the 4 bays on a separate controller than the ODD bays?

I could get an external PCIe controller if I have to.. however I wouldn’t know how to plug the drives in as the sleds put them right on the logicboard.

 

[Project Alice]

Okay looking at this; it looks the 4 drive bays are independent of the ODD bays.. if that’s correct I should be able to pass them through to TrueNAS for direct access correct?

 

[Project Alice]

Okay I finally ordered everything I should need.
I’ve got two X5675s coming, and 6x 16GB sticks. The RAM was cheaper than I thought it would be and I have already installed Proxmox and setup a random VM just to see how it runs.
it was using around 15GB of its 24GB and the VM was only set for 4GB. So 96GB triple channel should be enough.

The only thing I’m worried about is the HDD passthrough.. I would prefer to use an HBA card and pass that through but I have no idea how I would connect the internal bays to it. Anyone have any ideas there?

Edit: I found out this exists. So I’ve ordered one, an HBA card and break out SFF8087 cable so I should be good to go with giving TrueNAS full direct control over the drives.

 

[OldMacPro2]

Any updates? I have a 3,1 and a 4,1 and thinking about trying ProxMox on the latter.

I’ve flashed it to a 5,1, but haven’t upgraded the CPUs yet.

 

[Project Alice]

Any updates? I have a 3,1 and a 4,1 and thinking about trying ProxMox on the latter.

I’ve flashed it to a 5,1, but haven’t upgraded the CPUs yet.

Yes, sorry. Kinda forgot I posted this thread.

I ended up getting it all setup, pass through working on the card. However, it did not work out.
It started throwing all sorts of errors on my drives, and telling me the drives were bad. It happened within the first week. I bought a new HBA hoping that would fix it. It did not. I then ordered two new drives (as my drives are already around 4 years old, it wouldn’t be too much of a stretch if they had failed) the new drives started throwing unrecoverable data errors right off the bat.

Kind of a TL;DR. I migrated back to the older server (HP Z400) and found my drives were all working fine. For some reason the HP seemingly failed (stopped recognizing a couple RAM slots properly). I am now using the Mac Pro, but not with ProxMox. It’s just running TrueNAS on the bare metal. I don’t know if TrueNAS really hated the HBA cards or if it is an issue with ProxMox and PCIe pass through.

That said, ProxMox itself worked beautifully on the Mac Pro. I would say yes, use ProxMox as long as you do not intend to run TrueNas Or anything similar as a guest.

The other VMs were pfsense, and Windows Server 2012 R2. I had no problems using those.
I installed ProxMox using EFI, and everything just worked like it was supposed to as far as ProxMox is concerned. TrueNAS is a very hard OS to virtualize. Some people have done it successfully, but it clearly isn’t a good idea.

 

[OldMacPro2]

Good info. I have two other intel-based workstations (non-Apple, one Dell, one DIY), and I think I will use those for ProxMox.

I actually had it running on an old AMD-based desktop, but that didn't have the horsepower These other two boxes are Xeon and FX based, with more cores and RAM.

 

[darknetone]

Just to toss my hat into the Proxmox corner - I'm presently running Proxmox 7.2 on a Mac6,1 - with two LXcs and 1 VM - I am getting ready to add an OSX VM, we'll see how well that goes,... aside from that Proxmox works quite well on a MacPro6,1 - I know it works well on other MacPros as well.

 

[thecrownguy]

I knew I wasn't the only one. I have been running Proxmox on a bare metal loaded put 5,1 for about a year. Believe it or not, these machines are quite capable. I have about 7 VMs running all the time, mostly Ubuntu server hosting things like Pihole, pfsense, portainer, and my biggest one, TrueNAS scale with Plex, jellyfish, and photoprism running inside it. I have two pools running and one replicates to another, then the replicant is backed up to backblaze. Total usable space is 23.5 TB. She's a monster.
Anyway, it has been a blast but the issue I have had is getting an AMD RX580 to transcode plex and Jellyfin. I believe the only right way to do that is with an Nvidia card. Well, with GPU prices tanking, I am picking up a T400 (vs the p400 because it's new and will be supported longer), and that should do the trick.

Is anyone else in the same boat?

 

[Project Alice]

I knew I wasn't the only one. I have been running Proxmox on a bare metal loaded put 5,1 for about a year. Believe it or not, these machines are quite capable. I have about 7 VMs running all the time, mostly Ubuntu server hosting things like Pihole, pfsense, portainer, and my biggest one, TrueNAS scale with Plex, jellyfish, and photoprism running inside it. I have two pools running and one replicates to another, then the replicant is backed up to backblaze. Total usable space is 23.5 TB. She's a monster.
Anyway, it has been a blast but the issue I have had is getting an AMD RX580 to transcode plex and Jellyfin. I believe the only right way to do that is with an Nvidia card. Well, with GPU prices tanking, I am picking up a T400 (vs the p400 because it's new and will be supported longer), and that should do the trick.

Is anyone else in the same boat?

How do you have your TrueNAS drives hooked up? If you see my posts above I was having problems using HBA cards and it kept corrupting data; well it said it was corrupting data. I lucked out and nothing was actually lost.
I am now running TrueNAS (Core) natively by itself on my 5,1.

 

[thecrownguy]

How do you have your TrueNAS drives hooked up? If you see my posts above I was having problems using HBA cards and it kept corrupting data; well it said it was corrupting data. I lucked out and nothing was actually lost.
I am now running TrueNAS (Core) natively by itself on my 5,1.

Well, at the moment, I set up PCI passthrough and then set the disks to pass through just to truenas scale. Google Harddrive passthrough truenas, and there are tons of help videos. It works great with one exception. The smart values do not pass through without an HBA controller. The way to solve this is to pass the motherboard SATA Controller through to truenas and you will get SMART info. I am running a 1TB nvme as my proxmox drive on one of the PCI-E slots, and it shouldn't be a problem. However, I am scared to try it.

 

[saltarob]

Hi, has anyone tried a macOS VM in Proxmox on Mac Pro 5,1? I am wondering how to setup the OpenCore configuration for the macOS VM?

 

[roger45365]

Yes, sorry. Kinda forgot I posted this thread.

I ended up getting it all setup, pass through working on the card. However, it did not work out.
It started throwing all sorts of errors on my drives, and telling me the drives were bad. It happened within the first week. I bought a new HBA hoping that would fix it. It did not. I then ordered two new drives (as my drives are already around 4 years old, it wouldn’t be too much of a stretch if they had failed) the new drives started throwing unrecoverable data errors right off the bat.

Kind of a TL;DR. I migrated back to the older server (HP Z400) and found my drives were all working fine. For some reason the HP seemingly failed (stopped recognizing a couple RAM slots properly). I am now using the Mac Pro, but not with ProxMox. It’s just running TrueNAS on the bare metal. I don’t know if TrueNAS really hated the HBA cards or if it is an issue with ProxMox and PCIe pass through.

That said, ProxMox itself worked beautifully on the Mac Pro. I would say yes, use ProxMox as long as you do not intend to run TrueNas Or anything similar as a guest.

The other VMs were pfsense, and Windows Server 2012 R2. I had no problems using those.
I installed ProxMox using EFI, and everything just worked like it was supposed to as far as ProxMox is concerned. TrueNAS is a very hard OS to virtualize. Some people have done it successfully, but it clearly isn’t a good idea.

I’m trying to set up a 5,1 PCI Passthrough having some issues. Some guidance with to how you successfully was able to set up the nic Passthrough to work with Pfsense would be greatly appreciated! 🙂

 

[saltarob]

I’m trying to set up a 5,1 PCI Passthrough having some issues. Some guidance with to how you successfully was able to set up the nic Passthrough to work with Pfsense would be greatly appreciated! 🙂

[TUTORIAL] - PCI/GPU Passthrough on Proxmox VE 8 : Installation and configuration

In this article, I propose taking a closer look at the configuration process for setting up PCI Passthrough on Proxmox VE 8.0 (I had initially planned this article for Proxmox VE 7, but since the new version has just been released, it's an opportunity to test!). This article will be the...

forum.proxmox.com

NIC PCI-e Passthrough Bottleneck on Guest pfSense

Hello, I have a 25-Gbe embedded Intel E823-L controller which I'm trying to get as close to wirespeed as possible. When running iperf3 directly on the proxmox host I can hit full wirespeed. As soon I I try on a pfSense guest is when I run into problems. The maximum I could hit using a VirtIO...

forum.proxmox.com

 

[roger45365]

[TUTORIAL] - PCI/GPU Passthrough on Proxmox VE 8 : Installation and configuration

In this article, I propose taking a closer look at the configuration process for setting up PCI Passthrough on Proxmox VE 8.0 (I had initially planned this article for Proxmox VE 7, but since the new version has just been released, it's an opportunity to test!). This article will be the...

forum.proxmox.com

NIC PCI-e Passthrough Bottleneck on Guest pfSense

Hello, I have a 25-Gbe embedded Intel E823-L controller which I'm trying to get as close to wirespeed as possible. When running iperf3 directly on the proxmox host I can hit full wirespeed. As soon I I try on a pfSense guest is when I run into problems. The maximum I could hit using a VirtIO...

forum.proxmox.com

I have run most of these commands related to my machine and verified the driver is loaded. Might just be missing how to use it with my Pfsense VM or the driver is loading and there is still some miss configuration. It’s also not detecting that there is a connection on the ether port. It’s and Intel 350 1Gb Ethernet card.

 

[saltarob]

I have run most of these commands related to my machine and verified the driver is loaded. Might just be missing how to use it with my Pfsense VM or the driver is loading and there is still some miss configuration. It’s also not detecting that there is a connection on the ether port. It’s and Intel 350 1Gb Ethernet card.

Are you on a Mac Pro 5,1 and you want to use the original Ethernet card of the Mac Pro? Is your graphics card passthrough working now? There is usually no need to passthrough the Ethernet card:

• the VirtIO paravirtualized NIC should be used if you aim for maximum performance. Like all VirtIO devices, the guest OS should have the proper driver installed.

https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_virtual_machines_settings

 

[roger45365]

Ahh so I need to install the Nic driver on the pfsense VM OS