Mac Pro (Late 2013) - Amfeltec AngelShark Carrier Board - Storage Upgrade

[RyanXM]

Test Mac Pro Specifications
MacPro6,1
BootROM: MP61.0120.B00
macOS High Sierra 10.13.1 (17B1003)
E5-1660v2 3.7GHz (15MB L3) Hexacore
16GB of Apple/SKHynix PC3-14900 ECC
Apple/Samsung SM0256G PCIe SSD (SSUBX based)
2x AMD FirePro D300 with 2GB VRAM

Testing Tools for SSD Benchmarks
AJA System Test Lite - Primary Test - 4GB File Size
BlackMagic Disk Speed Test (Outdated, but still a decent tool to check against)

http://amfeltec.com/products/mac-pro-late-2013-carrier-board-for-m-2-pcie-ssd-modules/

I'm working on getting one in to test. Really interesting how they power the board from the Bus Bars that go to the GPU. They are using the two black screws in the upper right hand corner of Graphics Board B. They include two standoffs. Pretty cool!

UPDATE 1/16/18 12:00pm CST

Added pictures to post.

10.13.2 didn't change the compatibility of any drives at the bottom of this post.

The Samsung 960 EVO and 960 PRO are the best available to use with this adapter board. Because of the internal link speed, you will not see any increase in speed with the 960 EVO or 960 PRO, but you will be able to get up to 6TB of internal storage, which is nice.

The other interesting thing is the board has boot LEDs so that you are aware that it is working probably. These do not blink or stay on once the system has done its NVRAM and POST checks.

https://photos.google.com/u/2/photo/AF1QipMkoDZcx4TGyFJqZa3GGSbnvoJjNMGFwqezgHo2

UPDATE 12/1/17 12:30pm CST
I have received the card and I'm currently taking pictures and documenting benchmarks on difference NVMe drives.

UPDATE 12/2/17 10:30am CST
Pictures of AngelShark Board installed. It's a rather well engineered board. It is rather thin (looks to be a three or four layer board) and kinda flexible.

UPDATE 12/3/17 10:30am CST
I've been testing the board for multiple days and I'm still experiencing slow write speeds on any and all compatible m.2 SSDs. 300MB/s is the fastest I've been able to achieve, on BlackMagic Disk Speed Test and AJA System Test Lite.

Internal Apple SSD shown as external drive within Disk utility.

UPDATE 12/3/17 11:15am CST
300MB/s writes are still present even after a fresh install of High Sierra on the PM951 as the boot drive. Apple SM0256G speeds are the same within the boot environment on the PM951, 1400R/1200W.

UPDATE 12/3/17 5:30pm CST
Ran out and grabbed a couple of Samsung 960 EVO 256GB. They show full bandwidth on read and write. They are only performing at PCIe 2.0 x4 speed (up to 2GB/s). I RAIDed them in a RAID0 and same result as a single drive. ~1500MB/s READ & ~1350MB/s WRITE.

Single 960 EVO

RAID0 of 2x 256GB 960 EVO

COMPATIBLE
Samsung 960 EVO - Full Bandwidth on Read & Write
Samsung 960 Pro - Full Bandwidth on Read & Write
Samsung PM951 NVMe - Full Bandwidth Read, 1/4 Bandwidth Write
Toshiba XG4 - Full Bandwidth Read, 1/4 Bandwidth Write
Toshiba XG3 - Full Bandwidth Read, 1/4 Bandwidth Write

NOT COMPATIBLE
LiteOn CX2 (commonly used in Dell Latitude & Precision Notebooks)

(List up to date as of December 3rd, 2017 at 5:30pm CST)

Keywords
Mac Pro (Late 2013) Carrier Board for M.2 PCIe SSD Modules
Amfeltec AngelShark Carrier Board
SKU-088-01 AngelShark Carrier Board for M.2 SSD modules (M.2 key M)

 

[jclmavg]

And it costs?

 

[RyanXM]

And it costs?

Awaiting quote from Amfeltec. See below.

---------------------------------------------------------------------------------------------------------------------------------------------------

SKU Item Qty List Price Discount Unit Price

----------------------------------------------------------------------------------------------------------------------------------------------------

SKU-088-01 AngelShark Carrier Board™ for M.2 SSD modules 1 $ 518.62 USD 25% $ 388.97 USD

(Mac Pro (Later 2013) cylinder) (M.2 key M)


Freight Shipping and Handling $ 44.07 USD

FedEx

-----------------------------------------------------------------------------------------------------------------------------------------------------

 

[theitsage]

Very cool adapter.

 

[jclmavg]

While cool that's still quite pricey IMO, even with the discount.

 

[AidenShaw]

Very cool adapter. It splits the x16 PCIe connection four way.

Actually, it splits the x4 PCIe into three x4 PCIe slots.

​

 

[RyanXM]

One is on the way to me! Hopefully be here on Monday.

 

[mikeboss]

nifty little board. the board might be PCIe 3.0 compliant, but the MacPro6,1 only supports PCIe 2.0

 

[AidenShaw]

nifty little board. the board might be PCIe 3.0 compliant, but the MacPro6,1 only supports PCIe 2.0

The Xeon CPUs have PCIe 3.0. Maybe you're thinking of the MP6,1 T-Bolt, which is PCIe 2.0 speed.

 

[theitsage]

@AidenShaw thank you for clarifying that. I overlooked and thought the PCIe flash storage was sharing bandwidth with the GPU. Reading further, it's off the PCH which is PCIe 2.0 as @mikeboss said. Throughput then is capped at 2GB/s no matter how fast the PCIe drives are.

 

[RyanXM]

https://www.anandtech.com/show/7603/mac-pro-review-late-2013/8

The PCIe for the storage on Graphics Board B is only PCIe 2.0 x4.

EDITED. Thanks AidenShaw!

 

[AidenShaw]

https://www.anandtech.com/show/7603/mac-pro-review-late-2013/8

The PCIe for the storage on Graphics Board B is only PCIe 2.0 x8.

2.0 x4.

Note that the PCH itself is on what is effectively a PCIe 2.0 x4 - so the SSD shares bandwidth with the Ethernet ports and wireless and a few other small items.

 

[RyanXM]

Fixed.

I'm not really all that interested in pure speed, but reliable additional internal storage. I have an OEM Apple/Samsung SSPOLARIS PCIe NVMe SSD coming to see if it truly does work in the MacPro6,1 as one eBay listing seems to show.

 

[jclmavg]

Fixed.

I'm not really all that interested in pure speed, but reliable additional internal storage. I have an OEM Apple/Samsung SSPOLARIS PCIe NVMe SSD coming to see if it truly does work in the MacPro6,1 as one eBay listing seems to show.

Why not a Samsung 960 Evo/pro?

 

[RyanXM]

Why not a Samsung 960 Evo/pro?

I enjoy out of the box Apple compatibility for my clients. So I'm simply getting a 512GB SSPOLARIS from the Late 2017 iMac to see if they will work under High Sierra. High Sierra is still too buggy for my end users and client so the using it, as seen by the debacle this week with "root".

I will personally be using some 960 EVOs for testing of the adapter.

And Amfeltec isn't messing around! They are sending me the card via FedEx International Priority so I will be getting the card tomorrow! Happy Dance!

 

[jclmavg]

I enjoy out of the box Apple compatibility for my clients. So I'm simply getting a 512GB SSPOLARIS from the Late 2017 iMac to see if they will work under High Sierra. High Sierra is still too buggy for my end users and client so the using it, as seen by the debacle this week with "root".

I will personally be using some 960 EVOs for testing of the adapter.

And Amfeltec isn't messing around! They are sending me the card via FedEx International Priority so I will be getting the card tomorrow! Happy Dance!

I'll be following your results closely. Do keep us informed.

 

[saulinpa]

Great concept. So you can now go to 6TB of internal storage.

Two concerns:
1) is any speed penalty when only accessing one of the 3 SSDs?
2) any heat or power supply issues?

 

[bax2003]

This is great idea, and reminds me when 3rd party graphics cards started to pop up 2-3 years after Mac Pro 2008/2009 came out (Quadro 4000, Quadro K5000, Sapphire Radeon 7950, GeForce GTX 680...).

RFQ sent.....

 

[jclmavg]

I mailed them for a quote as well, but no discount price mentioned in their answer. At $518 it's just ridiculously expensive.

 

[RyanXM]

I mailed them for a quote as well, but no discount price mentioned in their answer. At $518 it's just ridiculously expensive.

Let me see if I can do a bulk order and get better pricing.

Mine arrived today. I will be updating my original post with pictures, etc. I will say this: I'm not sure its worth more than $350 after holding it in my hands. Uses a PLX PEX8718-AB80B1 Bridge.

 

[AidenShaw]

Great concept. So you can now go to 6TB of internal storage.

Two concerns:
1) is any speed penalty when only accessing one of the 3 SSDs?
2) any heat or power supply issues?

1) PCIe switches are pretty fast. They add latency of a few nanoseconds per transfer - but disk transfers are typically quite large so the practical effect is insignificant.

Of course, the 4 lanes -> 12 lanes fan out means that when using all three disks you'll be limited to slightly less than 2 GB/sec. That's still pretty fast, though.

2) A Samsung 960 Pro 2TB uses about 3 watts, so the power load is also insignificant for the total system. Since they include a heat spreader for the additional two drives, and a heat sink for the original drive - it looks like they considered the thermals.

 

[RyanXM]

I'm working on the full write up today.

There is some cause for concern at the moment. I'll be working with Amfeltec this week to see if what I'm seeing is normal or if I got a bad card (card appears to be tweaked a little and is missing a screw hole for the PLX heatsink)

Writes on a PM951 NVMe are currently sitting at 300MB/s. Reads are where they should be: 1499MB/s.

 

[CodeJingle]

Seems like an awful waste of hard drive throughput that is being paid for but not being used. Obviously if what the client is looking for is an 'out of the box' solution then that rules out most other possibilities. But I still cringe at the thought of switching 4 lanes out to 12 then flooding those 12 lanes so the throughput of the things connected is cut by 1/3.

I am still working on pulling out the PCIe 3.0 lanes for custom use. It is taking a long time starting with 2,011 pins on the CPU. The number of lanes going to the GPU could be cut in half and I would still expect for most purposes the GPU bandwidth would not saturate. Then that gives you an extra 16 lanes of PCIe 3.0 to work with. I expect any hardware that works correctly off the PCIe 2.0 lanes will work correctly off the 3.0 lanes. Ideally the solution could be made into a custom version of the connector coming off of the logic board. Something not too intrusive so as to minimize liability for damage during the installation process. Anyone buying such a solution would still have to have their computer taken apart to the point of being able to replace 2 out of the 3 original cables coming off the logic board with custom ones.

 

[RyanXM]

@CodeJingle I'm honestly surprised that they didn't think about this when doing the MacPro6,1, but then again, this is Apple that we are talking about. They really have forgotten the true Pro market. The iMac Pro is a bandaid while they figure out the Mac Pro. The iMac Pro will fit in nicely, but the iMac 5K was already in that position and most developers that use the iMac 5K for development don't need any extra cores. They just needed something with more horsepower than the MacPro6,1 could give them.

 

[CodeJingle]

@CodeJingleThe iMac Pro is a bandaid while they figure out the Mac Pro. The iMac Pro will fit in nicely, but the iMac 5K was already in that position and most developers that use the iMac 5K for development don't need any extra cores. They just needed something with more horsepower than the MacPro6,1 could give them.

I agree. For something like the iMac Pro which is forced to only have one processor on the motherboard I don't see much difference between a high-end consumer CPU and a high-end Xeon CPU. What makes a Xeon rise above a consumer CPU is the scalability of multiple processors on a single board. They could have taken the existing iMac 5K and released a version of it with a Vega board and a PRO SSD and called it good. I feel like the iMac Pro only serves Apple and not their customers. I really hope the next Mac Pro has an option not only for regular PCIe slots but also for multiple CPU.