10 Gigabit Ethernet?

[Doc69]

When do you think 10 Gigabit Ethernet will become standard in Mac Pros and iMacs?

I'm dreaming of having my main RAID data storage box connected to my server via Thunderbolt, and then having all my other computers being able to connect to the server via 10 Gigabit Ethernet.

A Thunderbolt breakout box with a 10 Gigabit Ethernet port would work as well, and then all the computers could connect to a 10 Gigabit Ethernet switch.

Any other ideas how to access large amounts of data fast from multiple computers? Xsan/Fibre Channel seems to be fantastic but is probably very expensive.

 

[paulrbeers]

I don't think you will see 10 gb standard on computers for quite some time. It wasn't until recently with SSD's, that we could even saturate a single gb connection (single drive of course). Most people who need/want 10gb+ connections, have huge budgets/IT staff to maintain a network with that kind of thoroughput. Most consumers are better off with a directly connected hard drive (i.e. eSATA and/or Thunderbolt).

P.S. the cheapest 10gb cards I can find that will fit in PCIE run over $500 and the cheapest switch I could find with any 10gb ports is almost $2000.

 

[cube]

I have been waiting for consumer 10GbE for some years now.

 

[Inconsequential]

I have been waiting for consumer 10GbE for some years now.

For what purpose can i ask?

I've only just got round to hitting 100 MB/sec let alone 1 GB/sec!

 

[nanofrog]

When do you think 10 Gigabit Ethernet will become standard in Mac Pros and iMacs?

As mentioned, this won't be seen for a long time. Think of it this way; you've already got 1G Ethernet ports, yet ISP providers are no where near saturating 100M Ethernet (i.e. DSL sold @ 6.0Mb/s, even formats such as FiOS Internet Service are only 150Mb/s/35Mb/s <down/up> at the top tier for a mere $200 per month ). In the case of FiOS, it's enough you can exceed a 100M port, but not a 1G port.

Now I realize that they can be/are used in networks as well (not sure if what you're doing is really consumer/home use, or business usage in your home), but 1G is overkill in most situations, and 10G is quite expensive. Combine both factors, and 1G is all you get OTB.

You have to pay for faster (FC, 10G Ethernet, or InfiniBand), and they come with a serious price tag to match that performance increase. $6k isn't unheard of just for a small switch, and they easily get over $25k. No cards, cables, or installation costs (running actual lines in the wall/ceiling, not just plugging the gear up properly and doing the software configuration). BTW, you need at least CAT6a (500MHz) if you mean to use copper (CAT7, 7a, or 8 will be even better, once the specs are finalized).

A Thunderbolt breakout box with a 10 Gigabit Ethernet port would work as well, and then all the computers could connect to a 10 Gigabit Ethernet switch.

I'm not sure we'll see such a unit, but it's possible. It'll be expensive to implement though as is any 10G system due to the cost (cost of the switches tends to be ugly).

Any other ideas how to access large amounts of data fast from multiple computers? Xsan/Fibre Channel seems to be fantastic but is probably very expensive.

FC is expensive, and so is 10G Ethernet (take a look at the cost of the switches in particular).

There isn't an inexpensive 10Gb/s networking solution yet. I was hoping LightPeak would be (had the potential if the networking layers were added to LP's protocol), but that's not what Intel has currently released. Then there's the distance issue of TB cables (3.0 meters).

I have been waiting for consumer 10GbE for some years now.

Even less expensive 10G (whatever the tech) would be welcome in the enterprise market, as it would open it up to additional users (those that could actually utilize it, but can't afford it at current pricing).

I was hoping LightPeak would do that, but alas, it looks like it's going to be a ways off yet (see above ).

For what purpose can i ask?

I've only just got round to hitting 100 MB/sec let alone 1 GB/sec!

100MB/s on an ISP connection (=800Mb/s)? What provider do you use, and how many body parts + children sold into slavery + monthly payment does it cost?

Seriously though, I assume you mean 100Mb/s Ethernet (= 12.5MB/s, and this is before losses due to overhead).

 

[Inconsequential]

100MB/s on an ISP connection (=800Mb/s)? What provider do you use, and how many body parts + children sold into slavery + monthly payment does it cost?

Seriously though, I assume you mean 100Mb/s Ethernet (= 12.5MB/s, and this is before losses due to overhead).

No, lol.

To and from a NAS I get 100MB/sec

 

[nanofrog]

No, lol.

To and from a NAS I get 100MB/sec

What are you doing that you need 10G speeds then?

 

[Inconsequential]

What are you doing that you need 10G speeds then?

I don't.

That's the point!

Nothing domestic can utilise it

 

[nanofrog]

I don't.

That's the point!

Nothing domestic can utilise it

Sorry, I was getting the impression you were interested in 1G as well as the OP (Doc69) in order to fulfill a throughput requirement.

 

[MovieCutter]

I don't.

That's the point!

Nothing domestic can utilise it

I move GBs of HD footage around the house for my client editing purposes...that's one.

 

[wafl iron]

Wirelessly posted (Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_5 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5)

its gonna be a long time before 10gb ethernet will be standard on consumer machines

i wouldnt be surprised if it was skipped for a greater speed

 

[nanofrog]

I move GBs of HD footage around the house for my client editing purposes...that's one.

That's not domestic/consumer use though, but rather commercial/professional use, even if it's at "home" for a company rather than renting office space.

Yes, I'm nitpicking.

BTW, examples on costs:

• 10G NIC's
• 10G Switches (managed & ready to go, not modules that need a backplane unit)
• CAT6a wire (1000' boxes, though ready-made are much easier if it's not to be installed in walls/ceilings)

Not exactly cheap per machine vs. 100M or 1G (i.e. no need for any NIC's). Just wire and a switch/router.

 

[Inconsequential]

I move GBs of HD footage around the house for my client editing purposes...that's one.

That is a business/professional use.

Gigabit ethernet for home users is more than enough.

All very well transferring GB's of data, but when hard-disks can barely supply that data quick enough in the first place

HOWEVER, given the Mac Pro is a professional machine, I could see what 10GbE could be useful to people.

Do any of the intel chipsets currently support 10GbE? Otherwise we're not getting it

 

[nanofrog]

Do any of the intel chipsets currently support 10GbE? Otherwise we're not getting it

Unfortunately, No.

Intel's 10G Ethernet controllers are separate parts, and only used on separate NIC cards (too expensive otherwise).

 

[deconstruct60]

BTW, examples on costs:

• 10G NIC's

One of the pricing issues for 10Gb Ethernet is that there hasn't been a > 10Gb Ethernet standard in production deployment. Since it was "as fast as you can go" the pricing stuck at higher levels. Now there are 40G Ethernet modules coming in some quantity the prices may move down slightly. It looks like that is going to be a slow process though.


You'd think the 10Gb Ethernet card vendors would be trying to push the

• 10G Switches (managed & ready to go, not modules that need a backplane unit)

For "home" usage an unmanaged switch would be better and typically cheaper. The only reason for managed switches is to do traffic shaping and fancy inter-switch networking/routing. If all of the computers/box on the 10G network are all plugged into one single switch then there is not much to "manage".

There aren't many unmanaged 10G switches because, so far, they tend not to be deployed often as direct user facing, "leaf" nodes. More backroom server and/or network backbone switches.

• CAT6a wire (1000' boxes, though ready-made are much easier if it's not to be installed in walls/ceilings)

Most of the copper 10Gb Ethernet solutions are not Class B ( rated for home). This is one of the primary problems with deploying > 5Gb copper wired LAN solutions to the home: lack of affordable fiber solutions .
A quick way to convert your Class B Mac Pro into a Class A device is to plug in a 10Gb Ethernet card.


Back when Lightpeak was going to push down home fiber network prices there was some hope that 10G Ethernet could piggyback off of that. Alas, "Lightpeak" became light-less and there is still now solution.

 

[nanofrog]

One of the pricing issues for 10Gb Ethernet is that there hasn't been a > 10Gb Ethernet standard in production deployment. Since it was "as fast as you can go" the pricing stuck at higher levels. Now there are 40G Ethernet modules coming in some quantity the prices may move down slightly. It looks like that is going to be a slow process though.

Unfortunately, I don't expect prices to fall quickly either.

For "home" usage an unmanaged switch would be better and typically cheaper. The only reason for managed switches is to do traffic shaping and fancy inter-switch networking/routing. If all of the computers/box on the 10G network are all plugged into one single switch then there is not much to "manage".

Generally speaking, I agree. But as you mention, there aren't a lot of them to begin with.

But the way things have been posed in this thread, I suspect 1x 10G up-link wouldn't be sufficient (under the impression they're after 1GB/s or so throughput per system, which means multiple NIC's in the server alone). So for a compromise, I looked for a 2x 10 up-link unit, which typically only become available in managed versions (and trying to avoid modules + backplane units due to costs).

Most of the copper 10Gb Ethernet solutions are not Class B ( rated for home). This is one of the primary problems with deploying > 5Gb copper wired LAN solutions to the home: lack of affordable fiber solutions .
A quick way to convert your Class B Mac Pro into a Class A device is to plug in a 10Gb Ethernet card.

Definitely.

But due to the cost constraints, Class A is all that's really feasible (fibre's just too insane right now).

Back when Lightpeak was going to push down home fiber network prices there was some hope that 10G Ethernet could piggyback off of that. Alas, "Lightpeak" became light-less and there is still now solution.

I know.

When LightPeak was being touted, it got my imagination going on the potential (cheap 10G fibre solution), but as you say, it's currently vapor. Maybe it will appear in the future (Intel claims they're still pursuing it), but that doesn't help anyone now, and may never happen (if they can't get the costs low enough). I suspect it may even have to wait for graphene to become commercially viable (thinking in terms of modulators in particular), and that's not going to happen within the next couple of years from available information.

 

[beaker7]

We're about to drop one of these into our filer.

 

[JavaTheHut]

We're about to drop one of these into our filer.

Let us know how the system works when you get it up and running - very interesting!

When people start to cry the blues over cost of the implementation of ThunderBolt in a year or so we can refer to the cost of this option -- BAAM

 

[nefan65]

When do you think 10 Gigabit Ethernet will become standard in Mac Pros and iMacs?

I'm dreaming of having my main RAID data storage box connected to my server via Thunderbolt, and then having all my other computers being able to connect to the server via 10 Gigabit Ethernet.

A Thunderbolt breakout box with a 10 Gigabit Ethernet port would work as well, and then all the computers could connect to a 10 Gigabit Ethernet switch.

Any other ideas how to access large amounts of data fast from multiple computers? Xsan/Fibre Channel seems to be fantastic but is probably very expensive.

It's still expensive on a cost/port basis. It's really for Enterprise Core Switches, with Physical hosts for Virtual environments, and Storage Area Networks. Also, to connect campus switches, and closets. It'll be a while before it's on the desktop. You'll see faster WiFi before you'll see 10GB desktop connectivity...

 

[deconstruct60]

When people start to cry the blues over cost of the implementation of ThunderBolt in a year or so we can refer to the cost of this option -- BAAM

Actually TB can't do this, since this is a multinode, network, NAS/SAN he is talking about. TB primarily oriented to direct attached storage (DAS) as is PCI-e. (and SATA and SAS) . TB has to meet or beat the per port and cable costs of external SATA/SAS to start claiming some cost saving victories. It may win in some price/peformance contests, but that is going to be rare unless push the price high.

It can go a bit longer than 3m too (for a price. ).


So if want to compare apples to oranges then yeah they will be different.

The more "apples to apples" comparison is to some XSan solution whose networking gear would likely cost just as much and also require a seperate metadata controller box. And yes, 10GbE setups have been getting increasing traction displaying Fibre Channel set-ups based on costs.

However, removing XSan's $999/seat cost factor may reverse that a bit for installations that are willing to the whole system to Lion. 10GbE needs a FCoE solution to crack that new price point. (XSan sending FC commands over FCoE transparently perhaps.)

At one point Small-Tree had a product GraniteStor FCoE-ST (http://www.small-tree.com/Articles.asp?Id=344), but it seems to be buried on the website now. I suspect most FC folks only want to very slowly unwind and switches haven't made it out of the OEM level.

 

[deconstruct60]

It's still expensive on a cost/port basis. It's really for ..... and Storage Area Networks..

Well, Mac Pros sit on SANs.

The Sparc T4s have two 10GbE ports built into the CPU package.
http://sun.systemnews.com/articles/108/3/hw/17688
(went to market as T2. http://en.wikipedia.org/wiki/UltraSPARC_T2 )

The new T4 will have two also (http://www.theregister.co.uk/2011/08/22/oracle_sparc_t4_hot_chips/ ). However, yes these will be storage network oriented boxes.

If Intel at some future stage were to put them into a Xeon Package that might cure the port price point problem. They'll still be stuck with how do you communicate cost effectively with less noise outside the box. They'd merge short distance TB before 10GbE .

Short term, most people will keep bonding multiple 1GbE ports to get something greater than 1 but less than 10.

 

[MovieCutter]

That's not domestic/consumer use though, but rather commercial/professional use, even if it's at "home" for a company rather than renting office space.

Yes, I'm nitpicking.

BTW, examples on costs:

• 10G NIC's
• 10G Switches (managed & ready to go, not modules that need a backplane unit)
• CAT6a wire (1000' boxes, though ready-made are much easier if it's not to be installed in walls/ceilings)

Not exactly cheap per machine vs. 100M or 1G (i.e. no need for any NIC's). Just wire and a switch/router.

That is a business/professional use.

Gigabit ethernet for home users is more than enough.

All very well transferring GB's of data, but when hard-disks can barely supply that data quick enough in the first place

HOWEVER, given the Mac Pro is a professional machine, I could see what 10GbE could be useful to people.

Do any of the intel chipsets currently support 10GbE? Otherwise we're not getting it

Um, ok. I guess those 20-50GB Bluray rips I move around don't count either...

 

[jsolares]

Um, ok. I guess those 20-50GB Bluray rips I move around don't count either...

But i doubt you're willing to pay the current prices for bluray rips, and it seems it'll be quite a while before prices start to come down

 

[Inconsequential]

Um, ok. I guess those 20-50GB Bluray rips I move around don't count either...

And? I do that too and I move them around at 100MB/sec.

My disks can't supply the data or write it quicker than that, and it only takes ~5 minutes to transfer 30GB anyway.

You can't rip faster than about 15-20MB/sec and once they are stored you don't need to move them about so I still don't see the need for 10GbE at home.

I think it's a case of "ooooh shiny" but doesn't actually benefit anyone.

It's like 500+MB/sec SSDs, everyone wants one, but won't be able to tell the difference between a 250MB/sec one.

I can't tell the difference between my X25-M and the Sandforce drives :/

 

[beaker7]

Short term, most people will keep bonding multiple 1GbE ports to get something greater than 1 but less than 10.

Unfortunately aggregating multiple ports together does not give you any more speed per socket. On a server, 2x 1Gb ports will supply 2 clients with 1Gb each, but not 1 client with 2Gb.

You also need a disk system that can handle it.

----------

Um, ok. I guess those 20-50GB Bluray rips I move around don't count either...

Nothing stopping you from putting a 10Gb NIC in your Mac Pro, getting 2 Thunderbolt breakout boxes for your Mac Mini and MBP, putting a 10Gb NIC in each breakout box, and hooking everything to a switch. It'll run about $5k though.