Guide to getting Win10, Boot Camp, USB3.1, and Geforce 1060 playing (mostly) nicely on Mac4,1/5,1

[tu2thepoo]

I picked up a 2009 Mac Pro from an IT yard sale for pretty cheap ($250), so I moved my existing Windows PC stuff into it. If I'd only known how much "fun" things had become since I last used a Mac with Boot Camp... Anyway, most of this is common knowledge, but I thought it'd be nice to have one thread to reference instead of having to search all through here and other Apple-centric forums.

Adding USB3 to the Mac 4,1/5,1:
This was the easiest part - just buy one of these Sonnet cards:

• 4 port USB3.0 standard card
• USB 3.0 4 port pro (has one controller per port, so better if you're running a multiple high-speed drive arrays
• USB 3.1 type-C 2 port

I bought the Type-C version and it was automatically recognized by MacOS High Sierra (10.13.1) and the Windows 10 Fall Creator's Update (ver 1709). I wrote more details in the main USB thread but there's not much to say - it works. Other options are available in the main USB thread.

Upgrading wifi/BT:
It seems the only plug-and-play solution still readily available is OSXwifi's product. MVC is currently out of stock but I dunno if it's a "email us and we'll make one" situation. There are a handful of cheap adapters you can find on eBay, but the BCM94360CD is actually pretty hard to find for less than $80 these days. If you're gonna pay ~$30 for an adapter and ~$80 for the BCM94360CD, you may as well get the premade solution.

My Mac Pro is affected by my Anker USB3.0 hub putting out 2.4ghz interference. Adding foil tape over the Airport/BT cards and wrapping the USB hub in foil tape definitely helped. Be careful with the kind you buy - if you get double-sided conductive tape you'll want to cover the Airport/BT cards with electrical tape first to avoid any shorts.

Anyway, my headphones work fine with BT2.1 and I don't use Continuity, so I skipped the upgrade in the end.

Moving hard disks/SSDs from my Windows PC to Mac
Actually moving the drives and copying data back-and forth wasn't a real problem. If you're looking for any suggestions for backups/drive cloning, I highly recommend freefilesync for Windows and carbon copy cloner for MacOS. I like that I can stop and restart the clone/sync over several days, instead of having to leave the Mac running for hours without interruption. Also, I can read the backup drives without any special setup, unlike Time Machine or an image-based backup.

Hardware-wise, OWC still sells their excellent 2.5" sleds and this 2.5" + 3.5" to 5.25" adapter is cheap and fits nicely in the optical bay (I already had it from my PC). I don't have any PCIe or NVMe SSDs to worry about, so no comment on that.

Getting Win10 working:
This takes a good bit of work if you're starting from scratch in High Sierra. After a lot of trial and error, I came up with this list of stuff to watch for:

High Sierra autoconverts internal SATA SSDs to APFS, and Win10 Boot Camp does not currently support APFS:
That means you can't click on the "Restart to Mac OSX" button in the Windows Boot Camp Control Panel - which makes it really hard to switch startup disks without an EFI video card. APFS can be avoided if you install 10.12 Sierra and run the 10.13 installer with the --converttoapfs NO flag. However, if you have an EFI-compatible card you can install 10.13 then just select the default startup disk at the boot screen. I tried APFS and it seemed to be buggy on my Mushkin Reactor SSD, so I installed 10.12 then upgraded to 10.13.

10.13 Boot Camp Assistant REALLY doesn't want you booting from USB or installing Win10
There are a few tutorials for editing Boot Camp Assistant's plists to enable creating a USB boot disk. You have to make one edit to mark the Mac Pro 5,1 as a USB Boot Supported Device, and another to remove it from the list of Win7-only models. However, if you're partitioning your main drive for Boot Camp, the Assistant creates a Hybrid MBR; as of the Fall Creator's Update (or maybe earlier, I dunno), Win10 won't let you install to a Hybrid MBR drive (at least I couldn't find a way to do it).

In my opinion, you're better off just getting a second drive and installing Win10 directly on it, without involving Boot Camp Assistant. If you have a Windows PC (or a friend with one), just grab a flash drive and the Windows Media Creation Tool. Hold "option" on bootup, select "EFI Boot", then install directly to the second drive.

Alternatively, I guess you could edit Boot Camp Assistant to create a USB boot drive, or make a windows VM to run the media creation tool inside that. If you visit this Microsoft link from a non-Windows PC it'll let you generate a link to download the ISO directly. I had the best luck using the Media Creation Tool, but your mileage may vary.

Apple makes it pretty hard to download Win10 Boot Camp files for the Mac Pro 4,1/5,1
The default Boot Camp files for the Mac Pro 4,1/5,1 are Win7-only and the installer will block you from running it on Win10. With Google's help, I figured out a workaround using Brigadier.

1. Download the regular Mac Pro4,1/5,1 boot camp files (either via Boot Camp Assistant or using Brigadier)
2. Download the Mac Pro 6,1 Win10 boot camp files with Brigadier (use the "--model MacPro6,1" flag).
3. Create a new folder (I made a separate Drivers/Apple folder but I don't think it matters), and copy the following:
   • From the MacPro 5,1 Boot Camp folder: AppleBluetoothBroadcomInstaller64.exe
   • From the MacPro 6,1 Boot Camp folder: BootCamp.msi, BCDriverMerge.exe, AppleWirelessTrackpad64.exe, and AppleKeyboardInstaller64.exe.
4. The folder should look like this when you're done:
   
   
   
   Note the modified date of 3/31/16 for the Win10 boot camp files, and 6/29/11 for the bluetooth driver.
5. Run "cmd" as an administrator, go to the folder you created, and run bootcamp.msi.
6. This will install the boot camp control panel, Apple HFS drivers, bluetooth drivers, and Apple keyboard/trackpad drivers (if applicable). You now have a more-or-less complete Boot Camp install.
   • You can omit the applekeyboardinstaller and applewirelesstrackpad .exes if you don't use the Apple Keyboard/Magic Trackpad. There may be separate a separate .exe file for the Magic Trackpad 2 (can't remember), and you may need to do a bit more digging/copying if you're using the 802.11ac/BT4 Broadcom adapter.

Boot Camp JHFS drivers are currently kinda broken in Win10.
I read that this has been broken for a long time now - anyway, it's definitely a problem if you have several journaled HFS drives. I have two 4TB drives set up as a 8TB JHFS RAID0 set, and Win10 will blue screen as soon as it tries to read the drives (which happens every boot). The only workaround I've found is to open Device Manager as soon as you log in, and disable hard drives until you stop getting BSODs.

I'm able to have my MacOS boot drive enabled without a problem (which is needed to use "Restart to Mac OSX" from the control panel), but not my other MacOS drives. You may have to play around to see what causes a crash - I'm not sure if it's FileVault, or certain drives with JHFS, or if it's MacOS's soft RAID, or what.

Getting a non-EFI Geforce 1060 (or any Pascal card) to play nicely with High Sierra, Boot Camp, and a Geforce GT120
This took almost as much work as just getting Win10 loaded. After much trial and error with the Apple Geforce GT120 and a Zotac Geforce 1060 6gb, I was able to get things sorted by doing the following:

• For Win10, I couldn't get my Mac Pro to boot without having an EFI video card installed.
  • Booting with only the Geforce 1060 would load MacOS fine, but setting the Startup Disk to the windows 10 drive resulted in a "no bootable device found" hung state when I restarted.
• Always boot with the monitor plugged into the Geforce GT120.
  • This may be more important if you're using FileVault because I'm not sure the NVIDIA web driver loads before the Disk Password or User login screen.

In MacOS:

• Install the NVIDIA web driver with SIP enabled - unless you want to manually disable Gatekeeper and have to keep track of that.
• After each successful reboot, you can just unplug the monitor from the Geforce GT120 and plug it straight into the Geforce 1060.
  • This worked for all Displayport and HDMI connections, on my Zotac card at least.
  • I'm not sure if you can keep both cards plugged into into a multi-input monitor/TV and switch between them. Haven't tried, really.
  • My Geforce 1060 doesn't seem to come out of sleep mode gracefully, so when in doubt plug the monitor into the GT120, wait for the screen to come back, then switch back to the Geforce 1060. This seems to reset the graphics stack.

In Windows 10 / boot camp

• If the Mac boots to an EFI video card, you have to leave it plugged in while Win10 is booting. Switching to the non-EFI Geforce 1060 midway seems to screw up the boot process.
• Installing the Geforce drivers with the GT120 and a Pascal card installed at the same time seems to cause an instant BSOD.
  • I read that the GT120 is only supported by the built-in Win10 drivers, and the web drivers can't support running both at the same time.
• A weird workaround I stumbled on is to disable the Geforce GT120 in Device Manager:

1. Boot using only the built-in Win10 drivers, and only the GT120 installed.
2. Disable the GT120 in Device Manager (it'll show either as a Geforce GT9500 or Basic Display Adapter).
3. Reboot to confirm that the GT120 still shows as "disabled" in device manager - you'll still get a display signal on reboot but I think all GPU acceleration is disabled.
4. Shut down and install the Geforce 1060.
5. Reboot into Win10 with the monitor plugged in to the GT120.
6. Install the NVIDIA drivers with the monitor still plugged into the GT120.
7. Reboot after the drivers are installed. Device manager should look something like this:
   
8. Once you've logged in, unplug the monitor from the GT120 and plug into the 1060.
9. Right click on the desktop, click "display settings", then click "Identify". The screen should show the Geforce 1060 is displaying on "Monitor 2" (even though nothing's plugged into the GT120):
   
10. Under "multiple displays", select "Show Only on Monitor 2":
    
11. You now have full GPU acceleration from the Geforce 1060 without any extended desktop weirdness.
12. Each time you restart, just have the monitor plugged into the GT120 during bootup. Once you're logged in, switch the HDMI/DP plug to the Geforce 1060 and repeat steps 9 and 10.

Now, you can totally avoid this by using an ATI video card as your EFI boot device. The reason I don't is that none of the EFI ATI cards I know of (x1900XTX, flashed 6850/6870, Radeon 4870, Radeon 5770) are 1) single slot, 2) run without additional 6pin power, or 3) available for cheap off eBay/craigslist. That means you have a hot-running video card running right next to either your Geforce or your hard disks, and then you have to figure out how to split the SATA power connectors to PCIe 6/8-pin. This may be less important if you're not running with your Mac Pro crammed full of drives like I do.

Anyway, if you stuck with this to the end hopefully you have enough information to get your Mac Pro set up with a Pascal Geforce and Win10. It was a ton of hassle and kernel panics, but now I have a computer that'll serve me great until Apple makes it incompatible with the next MacOS release

 

[h9826790]

Thanks for creating a nice detailed post that consolidate all the info from other posts. Just some extra info.

If anyone want to further lower the cost. A Inateck KT-4004 USB 3.0 card also work OOTB in cMP. And there is no need to use any adaptor to install 2.5" SSD. Just plug it into the optical bay, or into any HDD bay (prefer to remove the SSD case. e.g. as the following pic).

And Apple software RAID doesn't work well with Windows. To avoid crash, simply remove AppleHFS.sys and AppleMNT.sys in C:\Windows\System32\drivers\ can avoid the associated BSOD permanently. This procedure can be done in other OS (e.g. DOS, MacOS, Linux), much easier than rush to device manager and hopefully the computer didn't crash before you disable the associated HDD. The HFS driver won't work in the latest Windows update anyway, no point to keep it.

To install Windows 10 on cMP. It's so much easier to just burn a disc. In fact, it's very easy to install Windows with just the GTX1060 in there. Simply insert the disc, hold C during boot, then follow the on screen instruction. There is no need to have boot screen for Windows installation, the GPU will display.

To switch between the latest MacOS (APFS) and Windows 10 without a Mac EFI GPU. The easiest way should be using BootChamp with SIP Disabled.