Is TB2 a real-world advantage?

[analog guy]

Hi!
I'm including this in the Mac Pro forum as the nMP is the only desktop with TB2 for the moment.

The limitations of many TB drive enclosures (over TB1) are pretty well-known. TB adds more bandwidth, but in thinking about this I have been unable to confirm the answers to a question:

If the nMP supports TB2 but one has two separate TB enclosures (say there are 2 SSDs in RAID0 in each), could the user essentially create a (software) RAID0 of those 2 enclosures and see a performance increase, or would that not materialize?

There are only a few enclosures shipping today that are TB2 (well, maybe only the R4), and further only a few will be upgradable (e.g. Sonnet Echo Express models). Some new ones (e.g. OWC Mercury Pro Dual) are being released as TB1.

I got to thinking that maybe TB2 isn't such a necessary thing right now...that one could buy a TB1 enclosure and when (/if ever) more speed is needed, a larger RAID0 could be created by daisy-chaining the TB1 devices. Not sure if in the case of the nMP it would be better to daisy chain the TB1 enclosures or hang them off of different TB lanes.

Any thoughts?

Also, will a TB2 device (eg R4) simply run at TB1 speeds if the host computer (MBA, iMac) only supports TB1?

Thank you!

 

[AidenShaw]

If the nMP supports TB2 but one has two separate TB enclosures (say there are 2 SSDs in RAID0 in each), could the user essentially create a (software) RAID0 of those 2 enclosures and see a performance increase, or would that not materialize?

Any kind of RAID between multiple external enclosures is a headache waiting to happen. If the cable drops from one enclosure, you could lose the whole RAID.

If all the drives are in one enclosure, then they all disappear at once if the cable is pulled. There's no chance of IO continuing on some disks but not others.

 

[CH12671]

They are essentially the same thing, except TB2 allows you to use the video bandwidth for data, as long as you don't plug in a DP device on that TB port....whereas TB1 always reserved 1/2 the bandwidth for display....whether or not you used it.

I'm not sure you would need TB2 in your example....but to answer your question you cannot make a stripe out of two arrays that are already striped. When you stripe the two "arrays" you will wipe everything out, and have one big stripe. So yes, you can stripe all four drives, but you will not have two striped arrays within one big striped array (if I understand your question). And you can't run a raid 10 with it either, because OSx doesn't do that. You would have to have a hardware controller for that.....But even that wouldn't give you what you're asking for.

 

[analog guy]

Any kind of RAID between multiple external enclosures is a headache waiting to happen. If the cable drops from one enclosure, you could lose the whole RAID.

If all the drives are in one enclosure, then they all disappear at once if the cable is pulled. There's no chance of IO continuing on some disks but not others.

excellent point. i guess this came up because in my recent research i am reading about people running a RAID comprised of multiple USB3 enclosures. that seemed odd to me.

They are essentially the same thing, except TB2 allows you to use the video bandwidth for data, as long as you don't plug in a DP device on that TB port....whereas TB1 always reserved 1/2 the bandwidth for display....whether or not you used it.

I'm not sure you would need TB2 in your example....but to answer your question you cannot make a stripe out of two arrays that are already striped. When you stripe the two "arrays" you will wipe everything out, and have one big stripe. So yes, you can stripe all four drives, but you will not have two striped arrays within one big striped array (if I understand your question). And you can't run a raid 10 with it either, because OSx doesn't do that. You would have to have a hardware controller for that.....But even that wouldn't give you what you're asking for.

thanks for that explanation.

one correction, though: OSX does allow software RAID10 (i've done it for years) -- unless your point was to say that it doesn't allow it with *external* drives. i tend to doubt that but haven't tried it to refute it with certainty.

but anyway, setting up one RAID0 of 4 SSDs in 2 enclosures would exceed TB1 bandwidth.

if two TB1 enclosures are daisy-chained on a single TB2 bus (each enclosure has a separate RAID0 of 2 SSDs), could one see maximum simultaneous performance from both of them, or does the connection drop down to TB1 at best (so each transfer would get throttled to total to the bandwidth limit).

thanks.

 

[CH12671]

Really? The only options I see are stripe or mirror in the dive utility.....I'm talking native osx support, not third party app. Is it there?

 

[analog guy]

Really? The only options I see are stripe or mirror in the dive utility.....I'm talking native osx support, not third party app. Is it there?

yes, native support. RAID10 is a stripe of mirrors. first, you create the two mirrored sets. each mirrored set appears as a volume.

then, you create a stripe of those two mirrors.

(one could do it the other way around -- a mirror of stripes -- but there is a difference.)

not that i would do it, but earlier you set nested RAID0 sets are impossible without HW RAID. i'm not sure if that is true. given the above re: RAID1+0, are you sure of your answer?

 

[analog guy]

here's a screenshot from disk utility. you can see that i've got two sets of 1TB mirrors (RAID1-A and RAID1-B) which are then striped to form a 2TB RAID10.

one slice of RAID1-A is degraded at the moment.

 

[wonderspark]

Never thought of doing a RAID 00. Isn't that what they call a "plaid" RAID? Sounds like an incredibly bad idea.

 

[analog guy]

Never thought of doing a RAID 00. Isn't that what they call a "plaid" RAID? Sounds like an incredibly bad idea.

it sure has risk. however, people who are doing a RAID0 of multiple accelsior (PCIe) SSDs are essentially doing just that. their 960GB version is a RAID0 of 2 480s.

anyway, my question was really meant to be more about a single RAID0 across multiple enclosures -- more in terms of total bandwidth.

i'm not particularly interested in doing that (and even less interested in doing a RAID00) -- but I wanted to get at the question of simultaneous disk access across multiple enclosures and how that was handled if there are multiple TB1 enclosures on a TB2 bus.

 

[VirtualRain]

it sure has risk. however, people who are doing a RAID0 of multiple accelsior (PCIe) SSDs are essentially doing just that. their 960GB version is a RAID0 of 2 480s.

anyway, my question was really meant to be more about a single RAID0 across multiple enclosures -- more in terms of total bandwidth.

i'm not particularly interested in doing that (and even less interested in doing a RAID00) -- but I wanted to get at the question of simultaneous disk access across multiple enclosures and how that was handled if there are multiple TB1 enclosures on a TB2 bus.

To answer your original question... Yes, a RAID0 of two enclosures should scale just like any other RAID0 array. Each TB bus is 4 lanes of PCIe. It would be exactly like putting two Accelsiors in the top two slots of a 4,1 or 5,1 Mac Pro (each of those slots shares a x4 PCIe bus connection to the system).

The only thing I don't know, is if OS X would let you RAID0 two RAID0 arrays. This works without question with a pair of Accelsiors because they each appear as a single volume to OS X.

 

[CH12671]

The only thing I'm sure of is that I wouldn't try out without hardware.

I didn't realize osx would let you combine the two raid sets....my apologies. I always assumed if one was to create a second array with the software then it would destroy the first. Apparently I was wrong!

 

[analog guy]

The only thing I'm sure of is that I wouldn't try out without hardware.

I didn't realize osx would let you combine the two raid sets....my apologies. I always assumed if one was to create a second array with the software then it would destroy the first. Apparently I was wrong!

n=1, but i've been running a software RAID10 for 5.5 years on my MP.

----------

To answer your original question... Yes, a RAID0 of two enclosures should scale just like any other RAID0 array. Each TB bus is 4 lanes of PCIe. It would be exactly like putting two Accelsiors in the top two slots of a 4,1 or 5,1 Mac Pro (each of those slots shares a x4 PCIe bus connection to the system).

thank you.

sounds like putting enclosures on different busses would lead to higher overall transfer speeds (regardless of whether it was 4 drives in a RAID0 in two enclosures or 4 single drives being accessed simultaneously).

thanks.

 

[CH12671]

n=1, but i've been running a software RAID10 for 5.5 years on my MP.

Good to know, thanks!

 

[deconstruct60]

Any kind of RAID between multiple external enclosures is a headache waiting to happen. If the cable drops from one enclosure, you could lose the whole RAID.

Depends upon how he redundancies are balanced. RAID-01 where the mirrors are on independent paths then the system is still up. whereas with one box the whole system is down.

However, striping over cables isn't all that much headache than striping over devices. There are still failure modes inside a single box also.

 

[deconstruct60]

Never thought of doing a RAID 00. Isn't that what they call a "plaid" RAID? Sounds like an incredibly bad idea.

...

...
The only thing I don't know, is if OS X would let you RAID0 two RAID0 arrays. This works without question with a pair of Accelsiors because they each appear as a single volume to OS X.

There isn't really a 00. It is far more straightforward to just bubble 4 drives up and stripe those.

Can use two different RAID controllers (and implementations ) but it is not really 00. If that's 00 then every SSD in a set can add some zeros or a digits are the SSD drives pragmatically use RAID inside the drives between the Flash storage chips.

----------

I got to thinking that maybe TB2 isn't such a necessary thing right now...that one could buy a TB1 enclosure and when (/if ever) more speed is needed, a larger RAID0 could be created by daisy-chaining the TB1 devices.

When run out of bandwidth increasing the stripe width doesn't do much. If have 6 80MB/s drives and only a RAID controller with 400MB/s throughput then not really buying much by striping 6 way.


Real world advantage is not just the present world. If the system is being bought for use over multiple years then even if can't completely choke the bandwidth with workload now doesn't mean won't have higher workload requirements later within the service life of the system. That is still "real world" issues.

Not sure if in the case of the nMP it would be better to daisy chain the TB1 enclosures or hang them off of different TB lanes.

Far more depends upon what drives are in the drive enclosures than TB version.

Also, will a TB2 device (eg R4) simply run at TB1 speeds if the host computer (MBA, iMac) only supports TB1?

Everything downstream of first TB v1 controller will run at TB v1 speeds. [ downstream == farther from host ] If host starts at TB v1 then that sets the tone for everything else.

 

[AidenShaw]

Depends upon how he redundancies are balanced. RAID-01 where the mirrors are on independent paths then the system is still up. whereas with one box the whole system is down.

It's up, but will have to be rebuilt when reconnected.

However, striping over cables isn't all that much headache than striping over devices. There are still failure modes inside a single box also.

I'm referring specifically to the Tube here - you have a bunch of non-latching cables connected to a cylinder that's made to rotate so that additional cables can be added or removed.

I usually stripe between enclosures on my $300K EMC array - an 8 disk RAID-6 volume on eight different shelves. It has 30 of these 8 disk volumes.

As you said, this keeps my data available even if two shelves (enclosures) die completely. That, however, is a system with latching cables (and fibres), dual power supplies, dual UPS, dual controllers all enclosed in a cabinet that's almost never disturbed.