NVMe for Mac Pro 5,1

[Lauwie]

If the OP can show me how he reassembles the scap and fd files with the correct checksums so that they pass all the self-checking as the EFI loads, I can show him how to add and remove different EFI volumes from other machines' firmware. He just needs to explain what tools he's using and what steps he's taking.

Edit: I'm assuming he used IDA Pro with EFI plugins to disassemble the firmware and identify where the NVMe code was. Adding native NTFS and support for other disk formats, and upgrading to the latest USB spec and microcode update should be trivial compared to what he's trying to do - reassembling a working EFI was the stumbling block, which he seems to have overcome.

Thats indeed what I'm using.

 

[Lauwie]

Thus far I've been trying to get NVMe working by modding the firmware of NVMe drives to use Apple's class codes, but I'm having to much issues with it.
Guess I'll also have to rewrite the kext :'(

 

[MacVidCards]

Maybe start with just getting the kext working. It could work as a storage drive if you did whatever mods the next needs to work as a HD. I can boot into Windows 8.1 and all of a sudden the NVME drive appears as a normal drive. It plays peek-a-boo, only appearing in Windows.

So if you could get kext support first, it might get more interest in getting EFI working.

 

[Draeconis]

It would be interesting if the NVMe kext did a similar check that AHCIBlockStorage does on SSDs for TRIM support. Looks for 'APPLE' in the Model field.

Perhaps you could use perl to re-write the kext if it's looking for an 'APPLE' and replace it with a wildcard value?

Obviously this would then me an issue with kext-signing.

 

[jimj740]

It is nowhere near this simple.

I have a generic NVMe kext running as a block storage device already. I am currently working on support for management ioctls and direct application access to bypass the file system stack.

I do not know if I can release this publicly or not. This is why I privately contacted the OP. And then there is the matter of whether Apple will allow such a driver to be signed. They turned down my last request...

But as I indicated earlier, it works fine in thunderbolt adapters such as the Akitio as well as classic Mac Pro machines as a very fast data drive.

There is no magic here, and I do not have the time to educate the novices nor correct all the wild inaccuracies of this thread. [Not directed at Draeconis, who asks a question, rather at those that state opinion as fact and try to promote their misinformed beliefs as fact]

-JimJ

It would be interesting if the NVMe kext did a similar check that AHCIBlockStorage does on SSDs for TRIM support. Looks for 'APPLE' in the Model field.

Perhaps you could use perl to re-write the kext if it's looking for an 'APPLE' and replace it with a wildcard value?

Obviously this would then me an issue with kext-signing.

t t

 

[Draeconis]

So you've got a generic NVMe kext that can see NVMe devices and read/write to them as a basic storage device?

I assume the way you've written your comment means that it's not a bootable volume because the kext isn't signed? Or can't be loaded at boot?

 

[jimj740]

So you've got a generic NVMe kext that can see NVMe devices and read/write to them as a basic storage device?

I assume the way you've written your comment means that it's not a bootable volume because the kext isn't signed? Or can't be loaded at boot?

It is not bootable because there is no EFI support for generic (standard compliant) NVMe devices.
Kext signing must be disabled for the non-signed driver to load. Once loaded, each namespace of the NVMe device shows up as a separate physical disk that may be partitioned, formatted, mounted, read from, and written to.

So much like a USB 3.0 external disk on the classic Mac Pro, you cannot boot from it, but once booted the system can use it for any other purpose.

-JimJ

 

[handheldgames]

It is not bootable because there is no EFI support for generic (standard compliant) NVMe devices.
Kext signing must be disabled for the non-signed driver to load. Once loaded, each namespace of the NVMe device shows up as a separate physical disk that may be partitioned, formatted, mounted, read from, and written to.

So much like a USB 3.0 external disk on the classic Mac Pro, you cannot boot from it, but once booted the system can use it for any other purpose.

-JimJ

Amazing work enabling wicked fast storage I'd be interested to see how the SM951 NVE's perform in the empty Squid Quad M.2 PCIe adapter I'm looking at. With the right driver, I would have a good excuse to pick up some new silicon for the test bench.

 

[h9826790]

So, it mean this card can work in the cMP? (Not for boot at this moment)

 

[Draeconis]

If that card is a PCIe switch, then technically, yes (I assume).

It obviously won't run at 9GBps though, only around 6, but still.

 

[DPUser]

This is fun… watching with interest.

 

[Loa]

Hello,

I understand that figuring this from outside Apple HQ is a lot of work. Just out of curiosity, how hard/easy would it be for Apple to provide this? (I know they won't do it, but I'm curious.)

Loa

 

[jimj740]

Hello,

I understand that figuring this from outside Apple HQ is a lot of work. Just out of curiosity, how hard/easy would it be for Apple to provide this? (I know they won't do it, but I'm curious.)

Loa

It would be absolutely trivial for Apple to have adhered to the standard. This is part of the reason I delayed so long in creating my own driver - in the hopes that Apple would "do the right thing"(TM). At any moment Apple could make my efforts a waste of time.

The counter argument is that not all of the devices out there are 100% spec compliant either since the specification is relatively young and many are first generation products. So it would be entirely possible for Apple to release a compliant driver that would not work with all possible devices... However this would have the effect of getting people to complain to hardware vendors to improve their products to the benefit of all.

It really is a shame that Apple chose the path it did - though they probably did so to control their configurations tightly as well as the user experience. If that is the case, they should not label their devices as NVMe or NVMexpress, rather "Apple Proprietary SSDs".

-JimJ

 

[Loa]

Ok. Thanks for the info.

Hopefully, someone will find a way to add yet another string to our cMPs!

 

[wooddrift]

How's this going? Ive noticed Windows users can do this to get boot?

"One of the key things is that you need to build the USB to install windows with "Rufus", as usual (or original DVD). You need to select "GPT/UEFI" AND ALSO ENSURE ITS USING FAT32.
Anything that is not FAT32 will not work.

As soon you have the USB, then you need to play with the BIOS few times:

BIOS - BOOT - CSM Mode - DISABLED
BIOS - BOOT - Secure Boot - Other OS
Boot from USB /DVD
Install Windows from a bootable USB with GPT partition scheme or from the original Windows DVD
When the system reboots for 1st time:
Go back to BIOS and Enable CSM
Change Secure Boot - Windows UEFI Mode
Select M.2 as boot device
Save and exit.

The system will keep installing and will boot from the M.2"

Quote from http://www.tomshardware.co.uk/answe...lling-windows-sm951-z170-pro-gaming-mobo.html

 

[jimj740]

How's this going? Ive noticed Windows users can do this to get boot?

"One of the key things is that you need to build the USB to install windows with "Rufus", as usual (or original DVD). You need to select "GPT/UEFI" AND ALSO ENSURE ITS USING FAT32.
Anything that is not FAT32 will not work.

As soon you have the USB, then you need to play with the BIOS few times:

BIOS - BOOT - CSM Mode - DISABLED
BIOS - BOOT - Secure Boot - Other OS
Boot from USB /DVD
Install Windows from a bootable USB with GPT partition scheme or from the original Windows DVD
When the system reboots for 1st time:
Go back to BIOS and Enable CSM
Change Secure Boot - Windows UEFI Mode
Select M.2 as boot device
Save and exit.

The system will keep installing and will boot from the M.2"

Quote from http://www.tomshardware.co.uk/answe...lling-windows-sm951-z170-pro-gaming-mobo.html

It is going well... and being tested by a handful of people on both internal cMP and external ThunderBolt configurations.

Please note that this will NOT be a bootable solution for OS X on Apple machines due to the lack of EFI support.

-JimJ

 

[handheldgames]

It is going well... and being tested by a handful of people on both internal cMP and external ThunderBolt configurations.

Please note that this will NOT be a bootable solution for OS X on Apple machines due to the lack of EFI support.

-JimJ

thanks for the update. irrespective of the lack of efi support, this represents a great step forwsrd in storage technology for os/x. congrats!

thinking outside of the box, could the boot techniques used to enable the 1,1 and 2,1 mac pro's to boot el Capitan be leveraged for the 4,1 and 5,1?

 

[jimj740]

thanks for the update. irrespective of the lack of efi support, this represents a great step forwsrd in storage technology for os/x. congrats!

thinking outside of the box, could the boot techniques used to enable the 1,1 and 2,1 mac pro's to boot el Capitan be leveraged for the 4,1 and 5,1?

Perhaps... but in all honesty this is not an area I have any current plans to pursue. My work leverages NVMe devices, but there is no need for boot support. Not enough time in the day to do everything I would like!

-JimJ

 

[wooddrift]

JimJ Any chance I could beta test it?

 

[jimj740]

JimJ Any chance I could beta test it?

Release is very near... Final edits/revisions to the download page.

-JimJ

[Edit] To be clear, the reason for waiting for the download page to be complete is that you will need to read/accept the legal disclaimer and terms of use. Lawyers got involved...

 

[Lauwie]

If all goes well we'll publish the source around the end of february.
Didn't have to much luck with writing/modifying the kext.
I hope releasing the source of the modded EFI binary will help you folks and maybe someone can figure out how to get it working

 

[lowendlinux]

If all goes well we'll publish the source around the end of february.
Didn't have to much luck with writing/modifying the kext.
I hope releasing the source of the modded EFI binary will help you folks and maybe someone can figure out how to get it working

I think that's probably the only way, lots of fresh eyes

 

[DPUser]

Great stuff guys! Thanks, and please keep us posted.

 

[wooddrift]

@jimj740 What an effort. It works brilliantly. Thanks so much for your hard work. Big smile across my face right now

 

[rawweb]

Great work guys. Thanks for your hard work!

I can see a lot of uses for this as a boot volume, but, understand the technical limitations currently.