What are the options for 10GbE on Mac Pros?

[pastrychef]

I recently put together a Synology clone NAS and I'm considering getting 10GbE to maximize transfer speeds. Ideally, I'd like to use SPF+ connections (for cost reasons).

1. Are there any off the shelf 10GbE cards that works with Mac Pros?
2. Which commercial "made for Mac" work well on Mac Pros? I'm always very worried about potential issues with drivers. Historically, I've always tried to avoid installing 3rd party drivers whenever possible.

Thanks in advance.

 

[Machines]

I recently put together a Synology clone NAS and I'm considering getting 10GbE to maximize transfer speeds. Ideally, I'd like to use SPF+ connections (for cost reasons).

1. Are there any off the shelf 10GbE cards that works with Mac Pros?
2. Which commercial "made for Mac" work well on Mac Pros? I'm always very worried about potential issues with drivers. Historically, I've always tried to avoid installing 3rd party drivers whenever possible.

Thanks in advance.

I have not tried their cards yet, but Small Tree specializes in this sort of networking product for the Mac Pros .

https://www.small-tree.com/categories/10gb-ethernet-cards/

Here's a link to an older used product by them on eBay , this will be much cheaper than new :

http://www.ebay.com/itm/10GbE-Dual-...D&orig_cvip=true&rt=nc&_trksid=p2047675.l2557

Sonnet also has a card for around 400 bucks, but I think you need to buy the transceivers separately .

http://www.sonnettech.com/product/presto10gbe.html

10GbE is not cheap , compared to gigabit ethernet .

 

[Squuiid]

Wouldn't any Intel X540 card just work in a Mac Pro?

 

[pastrychef]

Since no one seemed to have any definitive answers, I investigated on my own. I truly did not want to pay the ransoms that Small Tree and Sonnet were asking, so I looked for cards whose manufacturers offered Mac drivers available for download. I finally stumbled upon SolarFlare which offers Mac drivers which can be easily downloaded from their website!!

Then, I searched form some deals on on eBay. Here's what I ended up buying:

$55.00 SolarFlare SFN5122F dual SFP 10GbE PCI-e card
$18.50 Mellanox ConnectX-2 10GbE PCI-e card
$14.75 Cisco 5M Twinax 10GbE cable
--------
$88.25

The Mellanox card went in to my DIY Synology clone. I chose this card because it has built-in drivers for this card. Once I had both cards installed and cable connecting both of them together, I set MTU to 9000 on both ends. Here are my results from my eight drive SHR2 (dual disk redundancy) array which consists of eight 8TB drives.

This is a significant improvement over the ~110MB/s that I was getting over normal gigabit ethernet. Performance is probably hampered by the use of the the Seagate 8TB archive drives but plenty fast for my modest needs. Not a bad upgrade for less than $90.

 

[Machines]

Since no one seemed to have any definitive answers, I investigated on my own. I truly did not want to pay the ransoms that Small Tree and Sonnet were asking, so I looked for cards whose manufacturers offered Mac drivers available for download. I finally stumbled upon SolarFlare which offers Mac drivers which can be easily downloaded from their website!!

Then, I searched form some deals on on eBay. Here's what I ended up buying:

$55.00 SolarFlare SFN5122F dual SFP 10GbE PCI-e card
$18.50 Mellanox ConnectX-2 10GbE PCI-e card
$14.75 Cisco 5M Twinax 10GbE cable
--------
$88.25

The Mellanox card went in to my DIY Synology clone. I chose this card because it has built-in drivers for this card. Once I had both cards installed and cable connecting both of them together, I set MTU to 9000 on both ends. Here are my results from my eight drive SHR2 (dual disk redundancy) array which consists of eight 8TB drives.

View attachment 621709

This is a significant improvement over the ~110MB/s that I was getting over normal gigabit ethernet. Performance is probably hampered by the use of the the Seagate 8TB archive drives but plenty fast for my modest needs. Not a bad upgrade for less than $90.

Very interesting and thanks for taking a leap into this pool .

I wonder why there is such a huge discrepancy between the reads and writes ?

10GbE in theory should push up to 1250 MB/s , minus any overhead .

Is SHR2 a proprietary Synology standard ?

How about a mSAS RAID chassis , like a $500 Sans Digital TowerRAID TR8X6G ?

http://www.sansdigital.com/towerraid-6g/tr8x6g.html

It might be compatible with a $230 NewerTech mSAS RAID card installable in a cMP .

http://eshop.macsales.com/item/NewerTech/MXPRMS6G1E1I/

or

http://eshop.macsales.com/item/NewerTech/MXPRMS6G2E/

In theory , 4800 MB/s throughput .

 

[pastrychef]

I can only speculate on why the writes are much slower. That being said, I suspect that it's mostly due to the Seagate 8TB archive drives usage of "Shingled Magnetic Recording" which is notorious for having poor write performance.

From what I understand, SHR and SHR2 (aka Synology Hybrid RAID) is Synology's implementation of mdadm. I chose it for its ability to dynamically expand and for it's ability mix and match different drive type and sizes (similar to what Drobos can do). SHR has single disk redundancy and SHR2 has dual.

I'm sure faster performance can be achieved using mSAS but I like the convenience of a NAS and I run Plex server from mine. I stream my movies to my Apple TV 4 form it. Secondly, my NAS is far more power efficient than running my Mac Pro 24/7. My NAS build uses an Intel Avoton C2550 CPU which has a TDP of 17W.

 

[AidenShaw]

I can only speculate on why the writes are much slower. That being said, I suspect that it's mostly due to the Seagate 8TB archive drives usage of "Shingled Magnetic Recording" which is notorious for having poor write performance.

Which is exactly why Seagate markets those drives as "Archival Drives", and not "put your database transaction logs here" drives.

I have more than a handful of them in RAID-0 sets. They're used for archiving (which means "write once, probably never read") of large .ISO files.

Love them, and don't care about the write speeds (although when you're writing a 45GB file to a RAID-0 array of shingled drives the write speeds are very acceptable). Small random writes, though, are sad. (But what sane person would buy a drive labeled as "archive" for a small random write load?)

 

[pastrychef]

Yup. I never complained about the performance. I went in knowing what to expect from those drives. The fact that I can get the performance that I do from just adding 10GbE, I'm very, very happy.

I also love these drives. Lots of storage at extremely reasonable prices. With my setup, I have 48TB of storage with dual disk redundancy and I only spent about $2500 and that includes the cost of building the NAS with hot swap bays.

 

[AidenShaw]

Yup. I never complained about the performance. I went in knowing what to expect from those drives. The fact that I can get the performance that I do from just adding 10GbE, I'm very, very happy.

Yep - we're pretty spoiled when 100 MB/sec is considered "poor write performance". I spent over $250K for my first disk array that had 100 MB/sec write performance.

Right now I'm setting up a cluster of servers that have 8TB of NVMe disk per system. I'm trying to figure out what's wrong when I only get 4 GB/sec write speeds. The 7 GB/sec read speed seems OK, but why are writes so "slow"?

 

[pastrychef]

Yep - we're pretty spoiled when 100 MB/sec is considered "poor write performance". I spent over $250K for my first disk array that had 100 MB/sec write performance.

Right now I'm setting up a cluster of servers that have 8TB of NVMe disk per system. I'm trying to figure out what's wrong when I only get 4 GB/sec write speeds. The 7 GB/sec read speed seems OK, but why are writes so "slow"?

Lol. Hysterical!!

I still remember setting up my first 5.25" hard drive. lol

 

[Silencio]

Yes, the SolarFlares seem to be the good low-end choice. I'll probably buy a couple to test out when I'm ready to upgrade my next round of workstations to 10Gb.

We went with ATTO cards at our shop. NS12 PCIe cards for our cMP's, and ThunderLink NS 2102's for our nMP's. They're definitely in the Small-Tree / Sonnet price range, though the drivers are very tweakable and provide a lot of information for troubleshooting and performance tuning. We get around 400MB/sec writes and 500MB/sec reads to our Isilon cluster, and we're not running jumbo frames or anything.

It will be nice when 10Gb-E is incorporated in Thunderbolt 3, then all you will need is the appropriate inexpensive media adapter (SFP+ or 10G-Base-T). The switches are still pretty expensive, but have come down quite a bit in cost over the past year.

 

[pastrychef]

I looked at the ATTO cards but was very turned off by the need to log in to access the downloads page.

I was wondering how much longer it will be before 10GbE will be standardized on Macs. None of my other Macs will every be able to have 10GbE, so I still don't have any need for a switch yet...

 

[nigelbb]

Lol. Hysterical!!

I still remember setting up my first 5.25" hard drive. lol

So do I & upgrading the original DEC RD50 to an RD51 with the enormous capacity of 10MB. That's MEGA Bytes not GB!!!
[doublepost=1458288313][/doublepost]I had no idea that prices on 10GB equipment had dropped so much. I guess that redundant gear from server rooms is hitting the market. Even the switches are affordable & can be found for less than I paid for my first 4-port 10MB hub.

 

[pastrychef]

Yes, I believe my first hard drive was also 10MB. lol I think it was a Conner MFM drive and I went running to Radio Shack to buy a ribbon cable and connectors to put together the SCSI cable. Great memories.

Yes, I would not have attempted going 10GbE if the costs hadn't dropped. I certainly had no intention of paying Sonnet or Small Tree prices.

 

[Squuiid]

Then, I searched form some deals on on eBay. Here's what I ended up buying:
$55.00 SolarFlare SFN5122F dual SFP 10GbE PCI-e card

You're kidding me! $55?! These retail for $600+

 

[Machines]

You're kidding me! $55?! These retail for $600+

$60 used on eBay .

SolarFlare SFN5122F 10GbE Dual SFP PCI-e Server Adapter SF329-9021 High Profile

http://www.ebay.com/itm/SolarFlare-...750174?hash=item1eacd6635e:g:9QEAAOSw~OVWxNNn

 

[pastrychef]

You're kidding me! $55?! These retail for $600+

lol.

Yes. I found a "like new" one on eBay for $55.

Here's another one for similar price:
http://www.ebay.com/itm/SolarFlare-SFN5122F-10GbE-Dual-SFP-PCI-e-Server-Adapter-SF329-9021-High-Profile-/371574926170
[doublepost=1458305622][/doublepost]I want to add that after running the card and driver for two days, everything seems to be running fine. Sleep and wake have also been working fine.

 

[Machines]

Oh , I think I've found the fly in the ointment . The drivers for the SolarFlare card seem limited to OS X 10.6.x Snow to 10.9.x Mavericks . They were last updated in November 2013 . No support for Yosemite or EL Cap ?

https://support.solarflare.com/index.php?view=categories&id=1795&option=com_cognidox&Itemid=11

 

[Squuiid]

Oh , I think I've found the fly in the ointment . The drivers for the SolarFlare card seem limited to OS X 10.6.x Snow to 10.9.x Mavericks . They were last updated in November 2013 . No support for Yosemite or EL Cap ?

https://support.solarflare.com/index.php?view=categories&id=1795&option=com_cognidox&Itemid=11

Yep, I literally just saw this too. They seem pretty old.
pastrychef, any issues in El Cap, despite the old driver?

 

[pastrychef]

Nope. I'm running the signed drivers in El Capitan perfectly. Didn't have to mess with SIP or anything.

Just because they haven't been updated in a while doesn't mean they don't work. The built-in drivers for Nvidia video cards haven't been updated in years too.

 

[fuchsdh]

Thanks for jumping in and testing. Might be something I'll be visiting myself in a few months.

And according to Mactracker, my first HDD was 40MB on a IIci

 

[Silencio]

My first hard drive was a 10MB 5.25" unit for my Apple //e. I never really did come close to filling that thing up.

Before I bought my first Mac, I acquired a 44MB SyQuest external drive and lugged that thing around everywhere, along with a handful of cartridges and a fat SCSI cable. Good times.

As for 10Gb ethernet: I expect we'll finally see a tipping point for more widespread adoption over the next couple of years. It's been a game-changer for our workflow, for sure.

 

[Squuiid]

pastrychef, you're making me spend more money on this Mac
Just bought an SFN5122F for £40 off eBay and an Intel X520-DA2 for £85 for my VMware server. I have a spare 3m SFP+ cable lying around so good to go.
Will report back once installed.

 

[pastrychef]

Omg! I remember those SyQuest drives. I wanted one in the worst way but was never able to save up enough allowance money. lol

@Squuiid :

Hahaha. I'm sure you will find that it's money well spent. I was made aware that some members of the hackintosh community have managed to hack Small Tree drivers to work with the Intel X520-DA2 cards but I feel the SolarFlare is a cleaner solution.

 

[ActionableMango]

Wow, $88.25 total! What a great find and a great upgrade for the price.