advice needed re mini-SAS

[malch]

Hi there,
I need advice. I'm a video editor who's been using an 8-bay e-SATA enclosure. It's been o.k., but for whatever reason, the connection has been getting 'flaky': when I launch my drives, they sometimes don't show up on my desktop and this can then cause kernel panic. (sometimes the drives show up if I start one at a time, but this is getting boring).

So I'm thinking of switching to mini-SAS. Is it just as fast, or faster? Or more important, since I don't work a ton of effects and speed hasn't really been a huge issue... is the Mac Pro/mini-SAS/enclosure connection more dependable?

If it is, what enclosure would people recommend, and what card for my 2010 Mac Pro.
I don't really think I need to RAID the drives, by the way.
Thanks,
malch

 

[CaptainChunk]

Which enclosure and what controller card are you using? Properly designed eSATA enclosures shouldn't behave like that.

SAS drives really aren't particularly useful outside of server and enterprise environments. Their capacities are lower and they cost significantly more (especially the ones with 10-15k spindle speeds).

 

[Torster]

Mini-SAS enclosures are awesome, rock-solid, and very fast. Even though they use mini-SAS connectors, you can use standard SATA drives. They don't have to be SAS drives. We use some here that get 1000 MB/s both reads and writes.

PM for a recommendation.

TL

 

[malch]

Hmmm. Well, I have two FirmTek SeriTek enclosures, with the FirmTek card they recommended. The technician there has tried to help; he thinks it must be some 3rd-party app conflicts and has suggested what sounds like a long, involved process to try to figure out where the conflict is.
I admit I was hoping mini-SAS was a huge leap forward in terms of speed/reliability/etc., and that this would justify the cost of a new enclosure/card. If it's not that much better, maybe I'll just have to play detective.
The SAS enclosures I was looking at were about $800 for a unit that holds 8-12 TB of storage (four 2TB drives or four 3TB drives), though, which is less than what I paid for my eSATA enclosure(s).
malch

 

[malch]

Torster - - just posted, in response to the Captain, before seeing your reply.
Glad you like SAS. Will keep thinking about it (and I'll wait, in hopes of more info). I just want to be able to hit a switch and see all my drives appear on my desktop.

 

[wonderspark]

I'm curious about your needs, since you say speed hasn't been a huge issue and you don't think you'll need RAID, yet you are asking about speeds and reliability. Are you just planning to use it for a lot of slow storage, or do you really want to improve sustained data transfer rates and reliability?

Do you know what speeds your getting now? If you're not using some form of RAID, I'd say it's less than 100MB/sec (maybe 85MB/sec?) right now, since you're at the mercy of the speed at which one disk can read/write.

 

[malch]

All very good questions, which I'll try to answer (though I should admit off the top that the reason I seek advice about what hardware to buy is that I'm not good at the tech specs myself).

The enclosure is a SeriTek/5PM, five-bay hot-swap enclosure. From the website:
"It supports all Serial ATA hard drives including the newest 1TB drives with 3.0Gbps transfer rates as well as original 1.5Gbps Serial ATA hard drives"
I use Western Digital Black Caviar, 1 TB drives for the most part. One of my drives is 1.5 or 2 TB I think.
Also from the website: "The SeriTek/5PM is compatible with virtually every external SATA PM host adapter on the market and supports data transfer rates of up to 300MBytes/sec or 3.0Gbits/sec per drive".
That's from the website, as I said; I have no idea what speeds I'm actually gettiing.

The adapter card that goes with it (SeriTek/2ME4-E) "features a high performance, energy efficient, four channel, hot swap, native eSATA interface with PCIe 4x bandwidth designed to maximize hard disk performance" (this from the FirmTek site)

I shoot and edit HD, using Avid Media Composer on my Mac Pro (24 GB RAM).
I don't do much green-screen or multi-layered effects, but often work with five or six video channels. I've been tempted by RAID, but haven't figured out exactly what the benefits would be for me. For every important project I make sure I have a backup of all the original files, plus a backup of my Avid MediaFiles folders, so that if I were to have a crash I'd be able to rebuild the project (this is how it's supposed to work in theory, at least).

So yes, I want speed (maybe more speed would mean I don't have to render certain effects in order to play them back; most effects don't need to be rendered as things stand now, though), but most of all I want reliability. I hate turning on my hard drive units and not seeing the drives load. When this happens I have to turn the drive units off, which causes kernel panic, which means re-starting everything, which is really annoying.

So should I stick with eSATA or go mini-SAS? Should I go RAID or not? Any thoughtful advice is welcome!
malch

 

[wonderspark]

Well, I'm a huge fan of RAID. If you have a chunk of time, you can read about the process I just went through in this thread.

RAID will greatly improve playback, renders, and just general file transfers. The 300MB/sec transfer speeds they talk about are max in theory, but you have to RAID drives together to reach that speed, unless you start using SSDs.

For example, Western Digital's 2TB RE4 7200rpm drives have a sustained data transfer rate of 138MB/sec individually (if I remember the spec sheet correctly) but when you RAID them all together, you can multiply that speed by the number of disks in the array, minus any parity disks used. (Consider them as not providing any speed to the data.)

So if you have a RAID3 of six disks, you get pretty close to the speed of five disks. 138MB/sec x5=690MB/sec, and in my case the real world speed I've measured in the last few days is 603MB/sec write, 550MB/sec read. (I'm trying to figure out if I can get closer to that max speed, but I'm pretty happy with what I have now!)

 

[malch]

O.K., I guess I better think seriously about RAID.
I read through the other thread and saw a lot of good advice from nanofrog and others... Thanks for the link.
By the way, are you going with an eSATA connection or mini-SAS?
Regards, malch

 

[wonderspark]

O.K., I guess I better think seriously about RAID.
I read through the other thread and saw a lot of good advice from nanofrog and others... Thanks for the link.
By the way, are you going with an eSATA connection or mini-SAS?
Regards, malch

I went with an Areca ARC-1880ix-12 (16 Ports 6Gb/s SAS/SATA RAID Adapter)

So, SAS via PCI card. I like my card, but if I were not interested in future-proofing, I'd probably just go with the cheaper 1880x card with just two external ports. I've had some struggles with the internal ports.

 

[malch]

O.K., a couple more quick questions. With two ports, you could hook up two of those 8-bay Sans Digital enclosures, couldn't you? (sounds like a lot to me).
Also: are you using SATA drives in your enclosure?

 

[malch]

I went with an Areca ARC-1880ix-12 (16 Ports 6Gb/s SAS/SATA RAID Adapter)

another quick question: the tech specs on newegg say there are two external ports... for both the ARC 1800ix-8 and the ix012. What do the 8 and 12 refer to?

 

[nanofrog]

Hmmm. Well, I have two FirmTek SeriTek enclosures, with the FirmTek card they recommended. The technician there has tried to help; he thinks it must be some 3rd-party app conflicts and has suggested what sounds like a long, involved process to try to figure out where the conflict is.

A software conflict is the likely culprit, particularly if there have been recent changes to the system, such as upgrading to Lion (there have been a lot of random issues with Lion and storage systems from what I'm seeing).

I admit I was hoping mini-SAS was a huge leap forward in terms of speed/reliability/etc., and that this would justify the cost of a new enclosure/card. If it's not that much better, maybe I'll just have to play detective.
The SAS enclosures I was looking at were about $800 for a unit that holds 8-12 TB of storage (four 2TB drives or four 3TB drives), though, which is less than what I paid for my eSATA enclosure(s).
malch

SAS gear can be both a lot faster, and more reliable than what you're using now. But you need to realize that it's also a lot more money vs. simple eSATA cards, software RAID (if used), Port Multiplier based enclosures, and consumer grade disks.

SAS cards can run SATA disks, but they need to be enterprise grade for stability reasons (different firmware than their consumer counterparts), and it all can add up quickly.

But if you need speed, reliability, and earn a living with your system (makes it a justifiable expense and can be written off), it's in your best interest to seriously consider the move to such equipment.

O.K., I guess I better think seriously about RAID.
I read through the other thread and saw a lot of good advice from nanofrog and others... Thanks for the link.
By the way, are you going with an eSATA connection or mini-SAS?
Regards, malch

The card and enclosure use MiniSAS (aka SFF-8087 = internal connections, SFF-8088 = external connections).

O.K., a couple more quick questions. With two ports, you could hook up two of those 8-bay Sans Digital enclosures, couldn't you? (sounds like a lot to me).
Also: are you using SATA drives in your enclosure?

No.

Each 8 bay enclosure used in wonderspark's configuration (he's only running one) does not use a SAS Expander (SAS Expander = similar to a Port Multiplier = it allows the use of multiple disks on a single channel). Which means it's 1:1 (disk per port) = 8 disks means 8 ports, and it uses 2x external MiniSAS connections on the rear of the enclosure (2x SFF-8088's on the back).

So the ARC-1880X can only run 1x Sans Digital TR8X.

another quick question: the tech specs on newegg say there are two external ports... for both the ARC 1800ix-8 and the ix012. What do the 8 and 12 refer to?

The number of disk ports each card supports.

Only one of those external connectors actually connects to disks. The others are for Ethernet, and in some cases, they have another port that goes to external enclosures for monitoring data.

 

[wonderspark]

another quick question: the tech specs on newegg say there are two external ports... for both the ARC 1800ix-8 and the ix012. What do the 8 and 12 refer to?

i=internal, x=external ports, and the LP=low profile (if you didn't pick up on that.) That means ix=both internal and external ports. The odd card is the ixl, which is long but not tall. My card (ix-12) is both long and tall, aka "full height."
I was confused about this before, but after looking into it, the 12 refers to the number of internal ports, but not external ports.

There is only an 1880ixl-8, and that has 2 internal ports (which will run 8 disks) and one external port (which will run 4 disks) so you could have one 8-bay and one 4-bay off that card.

What I was saying earlier is that if you'd only ever run 8 disks off the card at once, you could just get the 1880x and use the two convenient external ports. It's an option I still wonder that I might have been better served by. I like my card, but the internal ports require special (expensive) cables to make it work, and they don't supply you with any in the packaging.

Hopefully, between myself and nanofrog, we've made it clearer, not murkier.

 

[wonderspark]

We live in a tropical climate and don't have things like Fabreeze ready to purchase; so I am looking for cheap home remedies...thanks.

You lost me at Febreze...

 

[malch]

SAS gear can be both a lot faster, and more reliable than what you're using now. But you need to realize that it's also a lot more money

O.K., thanks so much for all the notes.
If I go ahead with this, I'm pretty sure I'd be fine with one 8-bay enclosure... the Sans Digital TR8X ($400) you've recommended.
On top of that I understand that I'd need the Areca ARC-1880x card ($700).
So that's $1100, and for the sake of greater reliability and speed, I think I could swing it.
A couple questions, though:

1. could I not put my Black Caviar drives into the SANS DIGITAL TR8X?
2. is RAID hard to set up? Would my drives, in the Sans Digital enclosure, connected to my Mac Pro via the 1880x card, be hardware or software RAIDed? Would RAIDing these drives require more expense? My plan would be to keep my eSATA drives connected for storage of finished, consolidated edits. I would move finished projects from the Sans Digital (RAIDed) drives to the non-RAID eSATA drives, then erase the Sans Digital drives when I start another project. Does this sound do-able? Would eSATA and mini-SAS co-exist peacefully? Looking at the back of my Mac Pro, it seems I have two more 'slots' for cards, under the one with my eSATA ports. Would it matter if the Areca card went into the slot below the eSATA card?

Thanks for your advice on this...
Malcolm

 

[wonderspark]

I'd keep your WD Caviar Blacks for storage of projects like you mentioned, and get enterprise drives for the TR8X. I went with WD RE4 drives, but they were $204 each. For backups and long term storage, I just keep buying way cheaper external drives or bare drives to stick in my Voyager Q.

The Areca card is hardware RAID, and it's easy to do, just takes time if you set up a parity RAID.

As for the slots on the Mac Pro, you can get away with only using one. I got the battery backup module (BBM) which they tell you to mount into a slot, even though it doesn't connect to a PCI slot... it just sits there. What I did is unscrew the BBM from the PCI bracket and slipped it between the power cables for the GPU. It fits because the BBM cable is quite long, and there is all that space between my 5870 and the side panel to the Mac Pro. This allowed me to remove the PCI slot cover to run my internal cables out to the TR8X.

Since the 1880 requires an x8 lane, and the Mac Pro only has two x16 lanes and two x4 lanes, it means putting the GPU in the bottom x16 lane, the Areca above that on the second x16 lane, nothing in the third x4 lane (for me, this is where my internal cables run out) and my 6G eSATA card in the top x4 lane. It all works out perfectly.

If I needed to add another device, I'd have to figure out a new hole to run my internal cables out.

 

[malch]

I'd keep your WD Caviar Blacks for storage of projects like you mentioned, and get enterprise drives for the TR8X. I went with WD RE4 drives, but they were $204 each

It's a lot of money, but they must be safer, so it sounds good.

The Areca card is hardware RAID, and it's easy to do, just takes time if you set up a parity RAID

Which RAID should I go for, do you think (I'd like to keep it kind of simple if possible).

As for the slots on the Mac Pro, you can get away with only using one. I got the battery backup module (BBM)

Is this the BBU you're talking about? The ARC-6120-3 BATTERY BACKUP UNIT?
http://tekram.com/store/product_info.php?cPath=24&products_id=203

which they tell you to mount into a slot, even though it doesn't connect to a PCI slot... it just sits there. What I did is unscrew the BBM from the PCI bracket and slipped it between the power cables for the GPU. It fits because the BBM cable is quite long, and there is all that space between my 5870 and the side panel to the Mac Pro. This allowed me to remove the PCI slot cover to run my internal cables out to the TR8X

Hmmm... would I have to do this? I'm not sure I'd need as many cables as you've got going...

Since the 1880 requires an x8 lane, and the Mac Pro only has two x16 lanes and two x4 lanes, it means putting the GPU in the bottom x16 lane

O.K., that's where my 5870 graphics card is located.

the Areca above that on the second x16 lane

Yes, above a slot that seems to have a fan in it (or something with lots of vertical slots) I have a free slot. This is where the Areca card would go if I'm following you correctly.

nothing in the third x4 lane (for me, this is where my internal cables run out) and my 6G eSATA card in the top x4 lane

My set-up as well, with my eSATA card in the top slot. I guess I could put the BBU into the slot below the eSATA adapter, correct?

Thanks for all this advice...
One more thing: if I only get four of these WD RE4 drives at first, would it be a problem RAIDing them, and then RAIDing again, once more drives get added (when I can afford them)?
malch

 

[nanofrog]

1. could I not put my Black Caviar drives into the SANS DIGITAL TR8X?

Unfortunately, NO.

Not if you want the array to be stable anyway. The recovery timings programmed into consumer grade disks do not work properly when attached to a hardware RAID controller, and drop out randomly = unstable (OK one second, degraded, or failed another).

2. is RAID hard to set up?

It depends on the user (how much they know, and how fast they can learn from written resources, such as manuals and online sources).

Would my drives, in the Sans Digital enclosure, connected to my Mac Pro via the 1880x card, be hardware or software RAIDed?

You won't be able to use your existing drives in the enclosure. You'll need enterprise drives instead, and they are more expensive.

For example, if you were after 8x of the WD RE4's @ 2TB each, you're a bit over $1600 USD for drives alone. Add in the $1100 for the card and enclosure (Sans Digital will include the external cables, which isn't common), you're a bit over $3700 USD by the time you've bought the hardware, and you may still need a proper UPS and/or Battery Backup Unit for the card.

So it's not inconceivable to exceed $4k if you're missing the right type of UPS (stepped inverters aren't recommended, as they damage Active PFC based PSU's, which is what the MP uses).

My plan would be to keep my eSATA drives connected for storage of finished, consolidated edits. I would move finished projects from the Sans Digital (RAIDed) drives to the non-RAID eSATA drives, then erase the Sans Digital drives when I start another project. Does this sound do-able?

You can do this, but keep in mind, you will need a backup solution for both the working data (RAID system) and the archival system (SATA/eSATA you're currently using).

eSATA is brilliant for this, as it's inexpensive, particularly when using Port Multiplier based enclosures and Green drives (best cost/GB out there).

Would eSATA and mini-SAS co-exist peacefully?

Yes.

Looking at the back of my Mac Pro, it seems I have two more 'slots' for cards, under the one with my eSATA ports. Would it matter if the Areca card went into the slot below the eSATA card?

Put the RAID card in SLOT 2, as the card is 8x lanes.

 

[malch]

Unfortunately, NO.
Not if you want the array to be stable anyway. The recovery timings programmed into consumer grade disks do not work properly when attached to a hardware RAID controller, and drop out randomly = unstable (OK one second, degraded, or failed another)

O.K., I understand and accept this. I wonder, though, could I start out with just four WD RE4 drives (they're the right kind, I think), and RAID them, and then add more drives later, when I can afford them? Or would it be complicated to have to re-RAID the system?

It depends on the user (how much they know, and how fast they can learn from written resources, such as manuals and online sources)

This is a problem for me, no question. I'll try to figure some of this stuff out, but when I read posts on the subject, like the one wonderspark started, and that you're advising on, the whole subject seems almost nightmarish. I suppose I might have to hire someone local to set things up for me, and then I'll just leave it alone. I'd want to make sure I have it done right the first time, though.

if you were after 8x of the WD RE4's @ 2TB each, you're a bit over $1600 USD for drives alone. Add in the $1100 for the card and enclosure (Sans Digital will include the external cables, which isn't common), you're a bit over $3700 USD by the time you've bought the hardware, and you may still need a proper UPS and/or Battery Backup Unit for the card

I think you mean $2700 for the drives, the enclosure and the card. That's a lot of money, but I'm thinking I could afford it (and if I can't, perhaps I can add fewer drives at first). By the way, my UPS backup is an APC Smart-1500, purchased last year on your advice as I recall. I hope it would be fine for the Sans Digital as well as for my Mac Pro.

keep in mind, you will need a backup solution for both the working data (RAID system) and the archival system (SATA/eSATA you're currently using)

Well, if I back up my Avid MediaFiles folder (onto an eSATA drive) for every big project, wouldn't I be o.k.?

Put the RAID card in SLOT 2, as the card is 8x lanes.

Is slot 2 the second one from the top (just under my eSATA card)?

Thanks so much for your help, nano. Same goes for wonderspark.
Regards,
malch

 

[nanofrog]

O.K., I understand and accept this. I wonder, though, could I start out with just four WD RE4 drives (they're the right kind, I think), and RAID them, and then add more drives later, when I can afford them? Or would it be complicated to have to re-RAID the system?

Yes you can.

It's called Online Expansion, and it allows you to add new disks to the current set without loosing your data or functionality during the process (it takes time, but it's very useful for things like SAN's).

This is a problem for me, no question. I'll try to figure some of this stuff out, but when I read posts on the subject, like the one wonderspark started, and that you're advising on, the whole subject seems almost nightmarish. I suppose I might have to hire someone local to set things up for me, and then I'll just leave it alone. I'd want to make sure I have it done right the first time, though.

You can if you wish, but there are some of us that can walk you through it (and it's important IMO for users to learn first hand how it all works = you can fix problems your self faster and cheaper than calling someone).

I think you mean $2700 for the drives, the enclosure and the card. That's a lot of money, but I'm thinking I could afford it (and if I can't, perhaps I can add fewer drives at first). By the way, my UPS backup is an APC Smart-1500, purchased last year on your advice as I recall. I hope it would be fine for the Sans Digital as well as for my Mac Pro.

Sorry about the math. But at least you get the point (not exactly cheap, as it can approach or exceed the cost of the computer it's attached to rather quickly - imagine a storage system consisting of 128 or 256 enterprise disks... ).

Well, if I back up my Avid MediaFiles folder (onto an eSATA drive) for every big project, wouldn't I be o.k.?

So long as the backup is actually sufficient for your usage, it will be fine (BTW, backup solution is more than just the location - you have to consider scheduling via backup software, so don't forget this aspect).

Given your intent though, I can't help but think you'll have a cluttered mess due to the capacity usage in short order. So an eSATA card + Port Multiplier based enclosure would be something to consider (Green drives for their low cost/GB make wonderful backup disks).

Is slot 2 the second one from the top (just under my eSATA card)?

No, it's the second from the bottom (bottom most slot = SLOT 1, then count up from there).

 

[wonderspark]

This is a problem for me, no question. I'll try to figure some of this stuff out, but when I read posts on the subject, like the one wonderspark started, and that you're advising on, the whole subject seems almost nightmarish. I suppose I might have to hire someone local to set things up for me, and then I'll just leave it alone. I'd want to make sure I have it done right the first time, though.

Thanks so much for your help, nano. Same goes for wonderspark.
Regards,
malch

You've followed us this far... I think you'd be fine with a little more Q&A and research. It will click for you.

 

[malch]

I really appreciate the advice and support.
I don't know what it is, but my brain just seems unable to decipher let alone retain most of the technical information I read on these threads (if only I didn't hear the strains of the music from Jaws, or on other occasions, The Exorcist, when I read sentences like "As per other levels on a hardware RAID card (5/6/50/60), the rebuild times can be improved by changing a firmware setting in the card (Background Operation Priority or similar heading; push it up, and the rebuilds go faster - performance of data operations during the rebuild slow down").
I will try, but really, right now I'm just hoping I can get an enclosure, a few enterprise hard drives and the right 1880 card (and BBU card), and keep the number of cables (it seems to me that you have them sprouting out of your Mac Pro like vines out of a vase, wonderspark) to a minimum, and set up a simple but fast RAID system.
Two things: nanofrog: is the APC Smart UPS 1500 the right kind?
wonderspark: did I gather from your other thread that Hitachi drives might be a better bet than the WD RE4s? Is the BBU unit I linked to the right one for my simpler Areco card?
Thanks again,
malch

 

[nanofrog]

I really appreciate the advice and support.
I don't know what it is, but my brain just seems unable to decipher let alone retain most of the technical information I read on these threads (if only I didn't hear the strains of the music from Jaws, or on other occasions, The Exorcist, when I read sentences like "As per other levels on a hardware RAID card (5/6/50/60), the rebuild times can be improved by changing a firmware setting in the card (Background Operation Priority or similar heading; push it up, and the rebuilds go faster - performance of data operations during the rebuild slow down").
I will try, but really, right now I'm just hoping I can get an enclosure, a few enterprise hard drives and the right 1880 card (and BBU card), and keep the number of cables (it seems to me that you have them sprouting out of your Mac Pro like vines out of a vase, wonderspark) to a minimum, and set up a simple but fast RAID system.
Two things: nanofrog: is the APC Smart UPS 1500 the right kind?
wonderspark: did I gather from your other thread that Hitachi drives might be a better bet than the WD RE4s? Is the BBU unit I linked to the right one for my simpler Areco card?
Thanks again,
malch

As the old saying goes, "The Devil's in the details". Particularly when it comes to RAID.

So things like exact level, card used, disk count, how fast each disk is, ... all matter in how fast a particular implementation will perform. This is why it's impossible to just put numbers to it.

With specifics, it's possible (there are approximations that are rather accurate).

As per the UPS, the SUA1500 or SMT1500 are good ones to have (use the correct inverter type).

Hitachi drives - wonderspark's are consumer models which are not suited to a hardware RAID controller. Their SAS disks are fine, but too expensive for what you'll be doing, and WD has better reliability, so WD is the brand of choice for SATA IMO (consumer or enterprise variants).

 

[wonderspark]

The correct BBU is ARC-6120BA-T113. It looks like it could be the same thing, but to be safe, here's where I ordered mine.
http://www.provantage.com/areca-technology-arc-6120ba-t113~7AREC03X.htm
(I like Provantage.)

The Hitachi drives seem to have a faster sustained data transfer rate, but I'm pretty sure they're not enterprise grade, so no, don't get those for the RAID. They are what came from Apple in my Mac. They even have the Apple logo on them. I use three of them in an internal software RAID 0 stripe in my Mac Pro as my scratch/previews/renders/exports disk, for which they work great. I get a solid 315MB/sec speed, which is about half of what my RAID 3 does but faster than a single SSD will do, and that makes me happy. (Had I known what I was doing, I would not have ordered my Mac with extra hard drives... at least I didn't order the RAM from Apple!)

I have an APC SmartUPS 1500. (Older model SUA1500, not SMT1500) and it's great. I bought it used for $25 from a company going out of business, and just discovered the battery needs replacing and ordered a new one. Who knows how old it was before I got it a year and a half ago. Incidentally, I found the newer one locally (Denver, CO) for $195, and get this... an SUA2200XL for $400! Those are $1200 from APC, and the one this guy is selling is still in the original shipping packaging, never been opened. I wonder if it "fell off a truck." It's even still strapped to a pallet. I'm super tempted to buy it.

By the way, I only have two "vines" coming out the back of my Mac Vase, haha, and they loop straight into the back of the tower right next to it, so it's not so bad.