2-NVMe blades, 2-PCIe to NVMe, Raid 0 & Mojave install. What can go wrong?

[AnimeFunTv]

So yesterday I began looking into upgrading my MacPro 5,1 storage and looking a the new 140.0.0.0.0 bootrom firmware that supports native NVMe boot, i took the plunge and bought two 500GB Samsung 970 Evo NVMe M.2 blades and two Vantech M.2 NVMe PCIe x4 Adapters.

Why 2 of each? Well, I wanted to get a single 1TB 970 Evo but it was sold out so I decided to make an experiment in RAID 0 and combine both the 500GB into a RAID'ed 1TB. This is where the adventure begins....

Hardware install is straight forward, install the SSD blade into each of the PCIe cards and place each card onto the MacPro, nothing special.

But, installing MacOS became a adventure, or nightmare.

First off, will MacOS be able to create a RAID using two PCIe to NVMe cards?
I created a Mojave USB installer and booting onto it, using Disk Utility and RAID Assistant MacOS WAS able create a Raid 0 using both PCIe cards with the NVMe blades. So success on that part. I formatted it using Mac OS Extended (Journaled).

Second, will I be able to fresh install MacOS Mojave in Raid 0? In short, No.
Using Mac OS Extended (Journaled) I could not get the installer to work in my RAID 0, the disk was grayed out and the prompt "You may not instal to this volume because the computer is missing a firmware partition"

Curious, I deleted the Raid and created a new Raid, this time using APFS. This time the installer recognized the drive and began installing Mojave. Success? Nope. About 3/4 done doing the pre-install I got a prompt saying "Could not create reboot volume for APFS Install".

Thinking this was a bug, I installed a HDD into one of the trays and using the same USB installer i attempted to install Mojave on to the hard disk. That was successful.

So now that i have a hard drive with Mojave, under the OS I deleted the Raid and re-created a fresh Raid 0 using HFS+ and attempted to install Mojave under the OS; but again disk was grayed but had a different prompt saying "You may not install to this volume because it is part of a AppleRAID".
Again I reformatted the Raid under APFS, the installer saw the drive and began installing but alas, i was prompted with the same prompt as before "Could not create reboot volume for APFS Install".

At this point I was frustrated and began searching online for others who may come across this issue. My setup is somewhat unique. There have been some people who've tried doing a fresh install but failed as well but there is a workaround apparently and it involves drive cloning.

Using the hard disk with the fresh install of Mojave, I downloaded Carbon Copy Cloner and cloned the OS that's on the hard disk onto the Raid 0 volume.

After that I went into system preferences and under Startup Disk, it saw the Raid as a boot drive. So crossing my fingers, I rebooted on to that volume.

And.... Success! My MacPro started booting into the Raid volume and boy it was quicker than I thought!

I wanted to see what kind of Write/Read speeds it was doing so I downloaded Disk Speed Test by Blackmagic.

Not bad, 2506MB/s Write & 2763MB/s Read.

So after all this, I've been reading more into the forum and general consensus is that MacOS 10.12 was possibly the last OS that would let you do a fresh install onto a RAID 0, while 10.13+ will not let you install the OS onto a RAID volume. The only workaround would be installing the OS onto another non-raid'ed, then cloning that drive onto a raid'ed volume and then boot off it, which will work; However it may have its drawbacks, some have reported that some MacOS updates will crash or just simply won't update due it being on a Raid'ed volume and subsequently would have to clone the Raid'ed volume back-to a non-Raid'ed drive, update, then clone the updated OS back to the Raid'ed volume.

If it weren't for the install and update limitations I would say this setup would be something to keep, but for now I'll be returning both my SSD blades and at least one of the PCIe to NVMe cards and just wait for a 1Tb blade to become available.

 

[reukiodo]

Thanks for taking the time to experiment with the RAID install and share your results!

 

[Juicy Box]

Second, will I be able to fresh install MacOS Mojave in Raid 0? In short, No.
Using Mac OS Extended (Journaled) I could not get the installer to work in my RAID 0, the disk was grayed out and the prompt "You may not instal to this volume because the computer is missing a firmware partition"

Curious, I deleted the Raid and created a new Raid, this time using APFS. This time the installer recognized the drive and began installing Mojave. Success? Nope. About 3/4 done doing the pre-install I got a prompt saying "Could not create reboot volume for APFS Install".

Thinking this was a bug, I installed a HDD into one of the trays and using the same USB installer i attempted to install Mojave on to the hard disk. That was successful.

So now that i have a hard drive with Mojave, under the OS I deleted the Raid and re-created a fresh Raid 0 using HFS+ and attempted to install Mojave under the OS; but again disk was grayed but had a different prompt saying "You may not install to this volume because it is part of a AppleRAID".
Again I reformatted the Raid under APFS, the installer saw the drive and began installing but alas, i was prompted with the same prompt as before "Could not create reboot volume for APFS Install".

At this point I was frustrated and began searching online for others who may come across this issue. My setup is somewhat unique. There have been some people who've tried doing a fresh install but failed as well but there is a workaround apparently and it involves drive cloning.

Using the hard disk with the fresh install of Mojave, I downloaded Carbon Copy Cloner and cloned the OS that's on the hard disk onto the Raid 0 volume.

I think this is typical behavior for trying to install MacOS onto a software RAID0 drive. If you install on a single drive first, you can then clone the single drive to the RAID0, then it would work.

I have not tried this on APFS though.

 

[AnimeFunTv]

I think this is typical behavior for trying to install MacOS onto a software RAID0 drive. If you install on a single drive first, you can then clone the single drive to the RAID0, then it would work.

I have not tried this on APFS though.

That is true, you can clean install MacOS on a non-raid volume, then clone that volume to a Raid 0 volume.

I've been reading up a lot info since my post.

My main goal was to have both my 500GB NVMe to PCIe cards Raid'ed 0, install MacOs and have it bootable.

While they both cards can be Raid'ed 0 and boot into MacOS (if carbon copied from another volume), Clean installing MacOS Mojave has severe limitations under Raid 0.

From what I've been reading, Pre-High Sierra can clean install onto SSD's that are Raid 0 only on SATA, not NVMe. It wasn't till High Sierra that can see NVMe drives and clean install, but not under Raid. (I can confirm this as I tried to install El Capitan & Sierra and the installer could not see the any NVMe Raid'ed or not, using HSF+; until I used High Serra and then it found the NVMe drives)

I wanna hope that some day Apple will be able to make MacOS install onto NVMe with raid 0 natively, so I'm going to keep my current setup (carbon copied Mojave on-to the Raid 0). Besides looking at the price I spent on the two 500GB NVMe (just around $250) the two PCIe to NVMe cards ($33) 2506MB/s Write & 2763MB/s Read vs. just one 1TB NVMe ($299-$399ish) with 1400MB/s Write & 1450MB/s Read. I think it's a better deal with close to double the speed.

 

[DearthnVader]

I was thinking, back in the day, that the macOS needed some sort of "helper" partition to boot from Raid0, maybe CCC is creating it?

 

[AnimeFunTv]

I was thinking, back in the day, that the macOS needed some sort of "helper" partition to boot from Raid0, maybe CCC is creating it?

Actually, No.

CCC does not create or copy the recovery partition, i know this as i tried to use my icloud account and ‘find my mac’ was disabled by default and message saying that it could not be enabled because there is no recovery partition.

In the end I just installed a fresh copy on one of the NVMe Pcie cards and the other i’ll use as a quick capture storage until ‘hopefully’ Apple makes changes to the RAID setup that allows installing MacOS onto a Raid 0 with NVMe on PCIe cards upon a clean install.

 

[DearthnVader]

Actually, No.

CCC does not create or copy the recovery partition, i know this as i tried to use my icloud account and ‘find my mac’ was disabled by default and message saying that it could not be enabled because there is no recovery partition.

In the end I just installed a fresh copy on one of the NVMe Pcie cards and the other i’ll use as a quick capture storage until ‘hopefully’ Apple makes changes to the RAID setup that allows installing MacOS onto a Raid 0 with NVMe on PCIe cards upon a clean install.

I wasn't talking about the recovery partition, Software Raid0 was at one time not visible for booting on a Macintosh, because the data was striped across multiple drives. As a work around a "helper" partition was created to make SW RIAD0 bootable, the system would boot the kernel from the helper partition and then use a Redirect( RD ) to use the System Software installed on the RAID0 volume.

Likely, Mac Pro's have the ability to read RAID0 volumes in the EFI Firmware.

 

[pepetato]

I will try to find a way to follow this guide on this thread according to toru173, but my main concern is how to deal with my rx580 since I can not see the boot screen on my mac.

 

[Cecco]

Using the hard disk with the fresh install of Mojave, I downloaded Carbon Copy Cloner and cloned the OS that's on the hard disk onto the Raid 0 volume

You cloned it to a Raid 0 formated as HFS+ or APFS?

 

[AnimeFunTv]

You cloned it to a Raid 0 formated as HFS+ or APFS?

strangely it did it as APFS.

I thought I had formatted the hard disk as HFS+ before installing Mojave but the installer converted the hard disk to APFS, so when cloning, it cloned the APFS.

 

[Cecco]

strangely it did it as APFS.

I thought I had formatted the hard disk as HFS+ before installing Mojave but the installer converted the hard disk to APFS, so when cloning, it cloned the APFS.

That’s normal. Mojave is converting spinning hard drives to APFS as well during installation while High Sierra did this only with SSDs.

I‘m wondering that you can boot Mojave from that APFS RAID 0. I‘ve read blessing the boot partition is not working on APFS RAID without special tricks like booting to single user mode.

Have you done any minor system update since? That seems to be the second problem after the initial installation.

Even with HFS+ RAID people seem to be faced with a non-bootable system after the update and the only save procedure seems to be cloning the system to a non-RAID drive, doing the update there and then cloning it back to the RAID. A tedious job for every update. :-(

I‘m seriously thinking about splitting my current 4x RAID 0 bootdrive on High Sierra into a non-RAID bootdrive + 3x RAID 0 for just the home folder for Mojave. Slightly less performing and more wasted space but less trouble with updates + a working recovery partion.

 

[naerct]

I've been testing with NVMe drives since they were recognized, then allowed to boot. I had huge problems with the 970 EVO Plus-much worse than SCSI Hell. I had more kernel panics than in the previous 9 years of use. It didn't end until I took it back and replaced it with the old EVO. At first I wanted to know if they could get faster than the 1500MB/s reads on my original 960. On separate cards, I used SoftRaid to stripe them, and it added 1000MB/s to the speeds with reads around 2400MB/s!!! But I realized that both my fast volumes would be on the same set of controllers, so that Adobe's scratch disk was competing with the macOS for control of the controllers. Everything worked better not in a RAID. But after another MacRumors forum helped me realize that I needed a PCI card with a heat sync. Good ones are $50 each, and it still uses 2 precious slots. I opted for the iocrest switch which was $200, but it gave me 2X speed without the RAID0, and used only one slot for both drives. I know get reads of 2700MB/s from the old 970EVO (not PLUS) and slightly slower for my 960EVO. That unit is an enclosure which has a heat sink and a fan. The fan is a bit noisy until I access my spindle drive RAID5, and then is drowned out.
More germane to the topic, I have some feedback on file system. After creating the RAID0 with both drives using HFS+, I then converted that RAID to APFS, and used a clone to copy Mojave. It didn't work. I then broke it down, but this time left it in HFS+, cloned it with my boot drive, and it booted and worked fine. It may just be SoftRaid which doesn't like APFS, but I made both SR and Apple RAIDS, as I tried it in my other MP which doesn't have SoftRaid. BTW SoftRaid's tools for this computer work has been a life saver. I have taken PCI cards from my 4-core, put it in the 8-core with SR and fixed the drive where Disk Utility couldn't. Writing the full 1 TB 970EVO with zeros only takes 15 min.

 

[freqrider]

Get a 16x NVMe raid controller if you want to kick it into high gear. IO Crest, Highpoint are good. $$$ though!

 

[naerct]

As I mentioned in my post, I did get the IO Crest switch (for my 2009 8-core), since the High Point ones are so expensive. I'm retired so the extra speed isn't really essential. I get 2.7GB/s reads and 2 drives from one PCI slot, as well as heat protection. I have the feeling that no matter how fast the drive is, the MP infrastructure isn't going to make the best use of it, and the fan is noisy.;-) But I highly recommend the IO Crest switch since it adds thermal protection and 2 SSDs in one slot. I could also put them into a RAID for greater results from BlackMagic Speed Tester. While I am not too concerned, boot times with the NVMe are still slower than using the RAID0 with 2 SATA SSDs on the Sonnet SATA III full length card. I am hoping that changes in the future with OS changes, as most of the Macs now use NVMe drives. Incidentally, my pic to the left has changed. I use the 960 in the IO Crest and I got an M.2 to SATA 2.5" converter, and can use that externally as well as inside the box. I'm planning on replacing the old 240GB SSDs on the Sonnet card with that 860 and another 500GB SSD, and see if it can beat the current 925MB/s read speed on my 4-core.

 

[joevt]

I did get the IO Crest switch

Can you post a link to the IO Crest Switch?

 

[naerct]

http://www.iocrest.com/en/product_details602.html
I got it from NewEgg.

 

[joevt]

http://www.iocrest.com/en/product_details602.html
I got it from NewEgg.

Is it the same as https://www.amazon.com/CREST-SI-PEX40129-Ports-Bifurcation-Controller/dp/B07HYZY7P2 ?

It uses an Asmedia ASM2824 bridge chip instead of Broadcom PLX bridge chip. The bridge chip is only x8, so you can get full performance from only one fast NVMe drive. i.e. You can't get the 6000+ MB/s that a x16 card can give in a PCIe 2.0 x16 slot of a Mac Pro. What is the PCI vendor and device ID of the bridge chip on your card?

 

[MIKX]

http://www.iocrest.com/en/product_details602.html
I got it from NewEgg.

naerct

A little off topic, apologies to all.

If possible, could you ( or anyone owning an IOcrest card ) post the IOcrest fan diameter & height please ?

I will be buying an IOcrest this or next month, I've seen posts saying that the IOcrest fan is noisy & if so think that a Sunon 12v ( don't buy 5v ) 3 wire 50mm x 50 x 15mm 17CFM fan might be able to be fitted ( hacked ) to the card.

The Sunon fans use " MagLev" magnetic bearings, I'm already using a 12v 50mmx50mmx10mm 11CFM model to cool my Northbridge chip - it's virtually silent.

Here's the DIgiKey list https://www.digikey.jp/products/en?FV=ffecfe96

I found that some of the out-of-stock models are available on Amazon. DigiKey's shipping is horrendously expensive.

 

[paulcons]

Just found this thread, way interesting. Always been a big fan of RAID 0, I have been using those suckers for well over a decade. Routinely got browbeaten for it, but I always had a good backup strategy AND in all that time, only had ONE of them actually fail (spinning platters, it was the TM backup, RAID on 2 1T drives, got a slot back now that it's on a single 2T platter-drive), and as it WAS a backup itself, no big deal. My current RAID array is 2 Sammy 500G SATA SSDs on a Seritek PCI card (5,1 cMP of course). This setup is now 5 years old and has served me quite well. BUT I am starting to at least be aware of possibilities should it fail.

BUT more immediately, I have a GTX 980, so 10.14 is simply a non-go. Unless we get the shock of the ages, I doubt they are going to fix their issues with nVidia with 10.15 (and HTF can they come out with a new Pro machine that can ONLY use AMD cards? Boggles the mind... AND I'd almost put money we will NOT know anything after WWDC). More directly, based on what I think I understand herein, trying to upgrade my 10.12.6 array to 10.13.6 may very well totally fail? Do I have that right?

Not to mention I could swear I read them clearly state when HS was launched that it WOULD fully support RAID even using APFS (which is a whole nother ball of wax). THAT event never happened, seems we are stuck just where we were before HS shipped.

 

[naerct]

The latest version of Mojave is bootable from a striped RAID. You need to use HFS+ to make the RAID. The upgrade requirements are far stricter than the working restrictions. So, I don't know if your RAID will work, but I doubt it since the installer is going to try to convert it to APFS. I use a similar Sonnet card with 2 SATA SSDs (RAID0) on the card. I used an external SSD with the old system OS, already in APFS, took my old video card out (required: no non-Metal vid cards) and do the upgrade to HS (in your case). Once the external SSD has the new OS, boot from it and then clone the new OS to your SATA RAID. Then just boot from the RAID, and even my old GT 120 runs HS or Mojave. I got an Nvidia 10-series card, but had to replace it to install Mojave. The RX 580 w/8GB VRAM makes a huge speed difference and was under $200. Its not a GTX 980, but not that far behind, and it works... My work is mostly 2D photography, so I don't need a monster card, but all the VRAM sure helps with Adobe PS and LightroomClassic. FYI, SoftRaid has promised its new version in the near future (I'm guessing that means this year) that works with APFS. I am guessing that the Apple RAID will work with APFS as well at that point. But I am going away from RAIDs for my boot drives. SInce flashing bR 140 BEFORE upgrading to Mojave, I added my 960 and got a 970 with separate PCI M.2 NVMe cards. I got the 1500MB/s speeds, which is normal for the 5,1. While one worked pretty well, I decided to try a RAID of the NVM2 drives, and BINGO, 2600MB/s. But with more reading and use, I discovered the cheap cards without a heat sink are not recommended. The better cards were $50 each, and took two PCI slots. Then I read up on the IO Crest switch card which has a sink and a fan and holds two NVMe blades ($200). I made the drives APFS and cloned my OS boot drive to one of the blades. So that switch card in a X16 slot, gives the drives full speed 2700MB/s for the 970 and slightly less or the 960 reads. I can also nearly double those speeds by RAIDing them as before, but the idea is that you want to keep your OS scratch disk separate from the scratch disk used for software like photoshop. Otherwise, they are always fighting for the same controllers. I use the 960 for the OS and the 970 for the "cache/Adobe swap file" drive. Now I feel on top of the latest Adobe software with a 10 year old computer. I must also admit I upgraded my 8-core with stock CPUs to a 12-core 3.33GHz CPUs and my RAM from 24GB of 1066 to 48GB of 1333GHz DIMMs (all for $280). That helped too... It was a little skittish in Mojave until the bR flash to 144 at the point upgrade to 14.5, and that made things a lot smoother. I'm looking for another ten years of use or at least to 10.15...

 

[paulcons]

Sorry I have zero confidence when the OS vendor makes one go through the shenanigans of creating a boot volume somewhere else then cloning it! There ARE public quotes from 2 years ago how they are supporting RAID... they have yet even vaguely demonstrate that. Not going to get started on the abomination that apfs is... then again, I was in the camp that thought the best move was to make a deal for ZFS, a FAR better file system. Nor a company that makes computers but oh, only being able to use some of one company's GPUs. I do understand sometimes having to go through some hoops, but saying I have to spend ANOTHER 200 bucks to replace a much better GPU is nuts, if they supported it AS THE SHOULD, I'd have none of these issues. I AM hearing the Radeon 7 is coming real close to nVidia performance, but big hoops need to be done AND you get no boot screen, not to mention about 800 for a GPU isn't in the cards.

My needs are far simpler these days, my poor little cMP may become a 80% win10 machine sooner than I'd want.

 

[naerct]

Sorry I have zero confidence when the OS vendor makes one go through the shenanigans of creating a boot volume somewhere else then cloning it! There ARE public quotes from 2 years ago how they are supporting RAID... they have yet even vaguely demonstrate that. Not going to get started on the abomination that apfs is... then again, I was in the camp that thought the best move was to make a deal for ZFS, a FAR better file system. Nor a company that makes computers but oh, only being able to use some of one company's GPUs. I do understand sometimes having to go through some hoops, but saying I have to spend ANOTHER 200 bucks to replace a much better GPU is nuts, if they supported it AS THE SHOULD, I'd have none of these issues. I AM hearing the Radeon 7 is coming real close to nVidia performance, but big hoops need to be done AND you get no boot screen, not to mention about 800 for a GPU isn't in the cards.

My needs are far simpler these days, my poor little cMP may become a 80% win10 machine sooner than I'd want.

I hear you about all this voodoo Apple is trying to perform, especially with the vid cards. But how many 10 YO servers do you know that are getting new bootROMs that allow us to continue to upgrade our cMPs? Besides, I'm not sure how you get by WITHOUT at least one system clone, and that can be connected to an interior drive bay, or USB or eSATA external connections. Then if you need a RAID, make it in HFS+, and clone the upgraded clone to the RAID... On my 12-core, I have 2 SSDs in RAID0, and 5 HDDs in RAID5, and a eSATA backup 4-bay external enclosure in RAID5. I use APFS for the NVMe boot and cache drives, have a WD USB3 8TB external which uses one container for six volumes with the slow spindle drive. All those 46.5 TB of storage are working well with regular clone updates which are less than 5 minutes with SuperDuper Smart Update. Weekly updates of all the data takes less than ten minutes for each backup set (2). Also, the bootROM 144.0.0... has done a lot to improve the smoothness of operation.

 

[paulcons]

Oh, I DO run a SSD array boot volume from a PCI card, HFS. It IS cloned to a single spinning platter. I DO have a Mojave issue as I use a nVidia 980. I AM nervous about doing the bootROM firmware update that comes with HS... I know it's a bit silly, but if I somehow bricked this machine I'd be in serious doo-doo, I have to depend on it so much (and I do NOT want to be pigeonholed into even considering a built-in screen laptop with thermal issues AKA iMac).

I AM kinda, almost searching for a compelling reason to even go to HighSierra and APFS that only got documented 6 months ago kinda scares me..

BTW, I used to do SuperDuper for the clone. Just switched to CCC, a VASTLY superior program btw. I
did miss a lot of scheduled SD backups if my machine was NOT running when it was supposed to go off and all it could do was wait for the next line. My CCC boots the machine in the middle of the night, runs, then shuts it down. AND it actually does a kind of versioning... it sticks files it replaces in a special folder. When the clonedrive gets to something like 50G free left, it starts pruning those files in order. I have about 300G to accommodate those "previous version" files.

 

[naerct]

Hi Paul, thanks for the response. i don't remember a bR flash for HS. However, the major update is at the start of the upgrade to Mojave. I remember I was at 139.0.0.0, and the bR flash to 140 takes place before the installation of Mojave starts, and you can just quit, unlless you want Mojave. At that point you can go back as far as Sierra and still have booting from NVMe. I doubt you could brick your cMP as long as you are not trying to boot to unrecommended beta versions. One of the betas was a problem and did brick single CPU cMPs. There is no danger of that if you download from Apple. Make certain that you delete any copy of the older Mojave install from your apps folder and download a 6GB file. You start like it was any update, and when it's through figuring things out, it will tell you about the flash with having to restart by holding the power switch down for about 12 seconds, until the light starts bling and then you get the weird beep, and then you hear it reboot and let your finger loose. If you have a Mac EFI vid card, the screen will fo white as usual, but I don't think there's a progress bar like a normal boot. Then your SuperDrive opens and the bootROM flash takes place. Then the DVD will close and it will boot up to the start of the actual Mojave, but you need not go further, unless you want to install Mojave. How can you live without The Dark Side feature and the Stacks on the desktop?
Many of my photo friends are still stuck on High Sierra because of Nvidia video cards or Capture-1. There's not a lot of difference between Sierra and High Sierra in everyday use, but both worked better for me than Mojave. But I am now on 14.5 with a bootROM of 144.0.0.0, and that last flash was mostly bug fixes, so things are smoother. Part of that difference could have been a small upgrade from a 8-core 2.2GHz with 24GB 1066 RAM to a 12-core 3.33GHz monster with 48GB of 1333RAM (for under $300 installed on my CPU tray). There's sure a lot of heat. But my 4-core was smoother too. So, I do recommend Mojave, and that looks like our last hurrah with cMP's.
Here's the bottom line on file systems. Both the old HFS+ and the new APFS have worked well for me, mostly without problems. If you boot from a SATA SSD or a NVMe drive, APFS is what you want, or it will convert it to APFS. Evidently it's supposed to convert all the drives, so, to be sure there's no problem, I just pull all the SATA bays. Also, you cannot use a RAID or if you have any non-Metal video cards in your system. I use softRaid on my spindle drives, and it doesn't really recognize APFS drives. I pulled all 5 of my RAID5 drives, and my Sonnet card (SATA3) from the PCIe slot. I just used a HS clone in one of the drive bays to boot from, with the latest version and all updates finished. Once I had the bootROM and 14.1, I cloned the NVMe drive to boot, and did the other point updates with the Samsung blade. So, if you need a RAID make sure they are erased as HFS+. Then you can use SOftRaid to make the RAID0 or the Disk Utility RAID, so that you can clone your OS to it without problems. Both the 12-core and the 4-core worked well, even for booting. I use the RAID on the Sonnet card to boot the 4-core without problems, and Mojave ran well from the RAID in HFS+ format. I am now committed to Mojave on all 3 of my Macs, for consistency if nothing else. I think your only problem is going to be that video card, which need to be addressed before moving to Mojave. I just got a call from a friend with a 12-core using the famed Nvidia powerhouse GTX 680 which will work in Mojave with the macOS drivers. I got a new Gigabyte RX 580 for $180 with 8GB of VRAM. When I ran the OpenCL diagnostic, it was way ahead of the GTX 680 on the score, and it's not that big like the 680. It still uses both the 6pin power connectors on the MP to one 8pin connector to the card. It's been cranking for a couple months now without problems, and the $20 rebate finally came lowering the price below $200. The call was to say he upgraded directly from 13.6 to 14.5 in one operation and he now has bootROM 144.0.0.0. I think I know a bit more about this stuff, but he's the type who rarely makes mistakes, unlike me. He was cloning (SD) his new Mojave drive back to his main system drives. Now that he can boot from NVMe, I'll take a cheap blade with a card for him to try. Once he sees how fast it is, he'll probably upgrade his SATA SSDs to NVMe SSDs. He has the same Sonnet card on which he has two 2TB OWC top line SATA SSDs, so he can keep all his files and databases on fast drives (over 1GB/s) in RAID0, just like mine, but 10% faster with his premium drives. If you'd like, when you decide to do the upgrade to Mojave let me know via this forum, and we may be able to connect by phone in case you have problems or questions. At least the OS background shot for Mojave is well made compared to the High Sierra one, and it's dynamic... I'm going to be looking at that sand dune for a long time, I hope.