10 GbE network suggestions?

[applCore]

I've tried for the longest time to keep storage on a workstation to reduce costs, but in the long run it has really cost me a lot more time and headaches than I'd like to deal with.

Presently I only have a 1 Gb CAT6 network. I'd like to build this out to a 10 Gb over ethernet network that at least supports at minimum 2 workstations and 1 server with 10 Gb over CAT6 and the rest with gigabit. The more 10Gb ports the merrier, but I can't break the bank.

The machines that would be 10 Gb would be an older 2010 Mac Pro, a newer Hackintosh system, and a FreeBSD server that I'll be offloading the storage to.

I'm looking to do this as affordably as possible and was hoping that I could get some input. Any suggestions as far as the switch and network cards go that would work on older Mac Pros?

So far I'm hitting a lot of mixed information and I'd like to keep this as simple and clean as possible with the ability to expand.

 

[belvdr]

The issue I found with the older and the newer less expensive 10GE switches was they were oversubscribed, sometimes more than 4:1. In other words, let's say you have an 8 port switch where each support 10Gb and is oversubscribed 4:1. That means out of the total 80Gb of bandwidth (8 x 10Gb), only 20Gb is available (80 / 4).

However, that only comes into play if all machines are saturating their links, but it's worth the exercise to think through what is required.

The only 10GE switches I have experience with are the Cisco Nexus 7K (really pricey) and the Force10 S4810. Both were solid performers and had no issues with them.

For an 8-port 10GE/1Gb switch, I'd expect to pay around $1k. For example, NetGear has the XS708E-200NES model for $683 (https://www.amazon.com/dp/B01GTWPTJY/ref=twister_B01LEIHZQS?_encoding=UTF8&psc=1). I've never used it, but wouldn't hesitate to give it a try. I've had really good experiences with Netgear switches.

All that said, are you sure any of these machines can transfer data at greater than 1Gb speeds (i.e. > 80-90MB/sec)?

 

[applCore]

All of the machines have pretty hefty SSD setups to run on. My goal is to isolate out the storage so that I'm not mucking up the macOS waters with ZFS as I've been doing for some time.

As far as I can tell, the switch is going to be the most important decision here, but yeah, that 8 port (XS708E-200NES) probably is about the right price point given my constraints.

As far as cards go, I guess I could try some Intel X540-T2 10G cards for $99 on eBay with the Small Tree "upgrade". I don't know if that's really worth it or if I should bite the bullet and get a few Startech ST10GSPEXNB or Akitio NBASETNC-A01 NICs...

 

[belvdr]

I had really solid performance and stability with the Intel cards. In fact, those are my preferred cards over any other, including Broadcom.

None of my systems were macOS so YMMV.

 

[Mikael H]

All of the machines have pretty hefty SSD setups to run on. My goal is to isolate out the storage so that I'm not mucking up the macOS waters with ZFS as I've been doing for some time.

As far as I can tell, the switch is going to be the most important decision here, but yeah, that 8 port (XS708E-200NES) probably is about the right price point given my constraints.

As far as cards go, I guess I could try some Intel X540-T2 10G cards for $99 on eBay with the Small Tree "upgrade". I don't know if that's really worth it or if I should bite the bullet and get a few Startech ST10GSPEXNB or Akitio NBASETNC-A01 NICs...

If you’re running FreeBSD on the storage server, the general recommendation from them has traditionally been to run with Intel NICs to avoid strange driver related performance issues.

@belvdr: My old AMD N54L-based home server with 4xWD RED drives on ZFS (RAID10) routinely saturates its GbE NIC, only unlike the OP I don’t have any economic incentive to throw more money at my home kit at the moment.

 

[belvdr]

My old AMD N54L-based home server with 4xWD RED drives on ZFS (RAID10) routinely saturates its GbE NIC, only unlike the OP I don’t have any economic incentive to throw more money at my home kit at the moment.

I was only ensuring the OP's gear could saturate a gig link, not whether it was possible.

 

[Mikael H]

I was only ensuring the OP's gear could saturate a gig link, not whether it was possible.

No offense meant; my point was just that GbE may be a bottleneck even for extremely modest hardware by today’s standards. But yes, in retrospect I agree that there’s nothing bad in ensuring that someone actually will see an improvement from new gear before spending serious cash on it.