External storage shared between Mac pro and Macbook pro

[ernie81]

Hi, I've got a mid 2010 mac pro running HS which I'm currently upgrading (processor, RAM and SSD and USB 3.1 + eSATA card). I've also got a mid 2014 macbook pro running Yosemite (planning to upgrade OS too).

Both are connected on a Gb ethernet network. I do photoshop and capture one stuff exclusively (large image files in excess of 5GB) and want to be able to work from either machine onto a shared external drive, so for example I'd like to be able to process images on capture one on the macbook and while images are processing there, I could be working in photoshop on another job on the mac pro. But wouldn't need to copy files between the two machines.

So I'd need the storage device to be fast, at least 4Tb (it'll also hold an archive of last 3/6 months of work) and likely need some kind of RAID mirroring.

One nice to have ability would be to have Time Machine backups from BOTH computers go onto this one storage device.

What sort of solutions would you guys recommend?

Thanks in advance

 

[biker4mac]

Seems like an NAS drive would be perfect for your situation.

 

[ernie81]

Seems like an NAS drive would be perfect for your situation.

Thanks, that's great! I don't suppose you have any recommendations for a fast NAS drive that can be used to edit files on?

 

[bsbeamer]

What's your budget? NAS typically fits into the pick 2 of 3 categories - good, fast, or cheap.

 

[Zeke D]

I've had good luck with used time capsules on the mac refurb places. I initialized a WD red drive in it, then stuck it in my cMP to add a separate partition for NAS, and one for time machine backups.

 

[ernie81]

Budget around £400-500 for 2x4Tb in a dual bay NAS. Priority order is:
1. Fast
2. Good
3. Cheap

I want something that can also be used with exFAT so potentially could be accessed read/write by a Windows machine as well.

 

[h9826790]

Budget around £400-500 for 2x4Tb in a dual bay NAS. Priority order is:
1. Fast
2. Good
3. Cheap

I want something that can also be used with exFAT so potentially could be accessed read/write by a Windows machine as well.

Don't know how fast you want, but QNAP TS-251A may fit your need.

TS-251A £200
2x Seagate Ironwolf 4TB £ 190

So, total is around £400 (in my city, which is NOT UK).

This NAS has 2x 1Gbps network, if you use both together, that can provide ~200MB/s read / write via the cable network.

Even though it's easy for the cMP, but in general, we won't connect the NAS in this way, especially it's not that simple to connect 2x 1Gbps LAN cable to your MBP.

So, let's assume the "normal" connection in this way.

1) direct 1Gbps LAN cable connection from cMP to NAS (private channel for NAS, won't affect internet speed)
2) cMP 1Gbps LAN cable to the router (for fast internet)
3) NAS 1Gbps LAN to the router (for other computer to access in the intranet)
4) MBP wifi access to the router (for both NAS and internet)

In this case, your cMP can have stable 100MB/s transfer speed to / from NAS. MBP up to 100MB/s via wifi, but unstable. This is for "normal" use.

When you need faster connection. Either cMP or MBP can make a direct connection to the NAS's USB QuickAccess port (by using USB 3.0 A to micro B cable). So that, your NAS will work like a USB 3.0 HDD enclosure. In this case, the RAID 1 array speed will be the bottleneck. This should provide about stable 150MB/s.

For the cMP, this USB QuickAccess may not make huge difference, especially the real transfer speed via direct LAN cable usually cap at 110MB/s, but not 100MB/s. And direct USB access not necessary able to provide 150MB/s (depends on the data location on the disc). But for the MBP, this USB QuickAccess means a lot. Super easy connection via USB cable, much faster stable transfer speed, and release all bandwidth for internet access.

 

[bsbeamer]

100% agree with above. For your budget and your requirements, QNAPs are probably the best option. Unless you're going with SSDs or RAID5 options, you're not going to get much more than 175-200MB/s with these types of enclosures.

Even with a 8TB GRAID connected to MacPro5,1 and configured in RAID0 via USB3, speeds average around 220MB/s write & 230MB/s read with 16GB video test files in AJA System Test Lite. This drive also has TB2 port for when I need to use it with MacBookPro.

 

[ActionableMango]

I'd just connect the drive(s) to the Mac Pro and enable file sharing so the MBP can access it via the network. You won't have to pay for a NAS and you won't add to your complexity, clutter, and maintenance. The MP will have a full speed interface because the drives are local, and the MBP will have network access just as it would from a NAS. The only drawback is the MP will have to be on for the MBP to access the files, and the NAS is likely to always be on.

If this solution sounds good to you, it'll have to be eSATA or USB 3.0 since you want the drives to be external. Assuming you have a free PCIe slot, those cards are fairly inexpensive, much less so than a NAS. If you are going to use more than one external drive simultaneously, I'd either use eSATA or a USB 3.0 card with a controller dedicated to every port. A USB 3.0 card with a single controller shares bandwidth on all ports.

I'm not a huge fan of RAID, from both personal experience and what I've read right here on MR. Yes they protect against one kind of problem (hard drive failure), but they introduce complexity and cost, and they add other single points of failure. And RAID is not the same as a backup. Corruption, accidental action, or malware action occurring on the RAID can destroy your data.

I'm also not a fan of exFAT. I am never using that again. Yes, I know native support from both MacOS and Windows is a huge boon, but I had frequent corruption problems on two out of two drives. exFAT was intended for consumer devices like a camcorder to be able to access larger files than FAT32 could deal with. exFAT is not journaled, and you should never use a non-journaled file system for anything important. I think what really scared me though, is that when I experienced corruption problems I could run a chkdsk in Windows, it would find and fix errors giving me a clean bill of health, then I'd run a fsck in MacOS and it would find errors too. In my opinion exFAT support is BETA level at best.

 

[AidenShaw]

100% agree with above. For your budget and your requirements, QNAPs are probably the best option. Unless you're going with SSDs or RAID5 options, you're not going to get much more than 175-200MB/s with these types of enclosures.

Even with a 8TB GRAID connected to MacPro5,1 and configured in RAID0 via USB3, speeds average around 220MB/s write & 230MB/s read with 16GB video test files in AJA System Test Lite. This drive also has TB2 port for when I need to use it with MacBookPro.

Gigabit Ethernet has an absolute theoretical max of 125 MB/sec. A good implementation can do 110 MB/sec or so.

Do these USB3/TB2 connections actually support shared write access? A big issue with sharing is that the drive array has to manage the filesystem. With NAS that's a given, with DAS not so much.
[doublepost=1522182580][/doublepost]

I'm also not a fan of exFAT. I am never using that again.

+100. I don't consider "exFAT" to be "persistent store". It's "maybe you'll be able to read it" store.

When your starting point is a floppy filesystem designed for 1MB disks, and your goal is to come up with a more-or-less compatible filesystem for files larger than 4 GiB - the result is not pretty.

 

[bsbeamer]

Does GRAID support read/write via two separate connections at the same time? No, and never said it did. Was showing the speeds I was getting from RAID0 off that unit for comparison.

There are other connection protocols to look into, but given the budget available, it's useless going into them.

RAID is never a backup. Clone, and clone often to multiple sources if it's important data.

 

[AidenShaw]

RAID is never a backup

This can't be emphasized enough.

Most of my servers have RAID-60 drives with hot spares. And they're backed up to a backup server with RAID-60 drives with hot spares. (The backup server does daily backups with 30 day retention, and monthly backups with 180 day retention.)

And I have a separate "archive" file server with RAID-60 drives with hot spares for manual "backups" using robocopy or rsync.

RAID protects from the occasional drive failure. It doesn't protect from wetware failures, or from RAID failures (I had a 20 TB RAID-60 array dissolve after a power failure - but the backup saved almost everything).

 

[Blackey Cole]

I use a external raid connected thru esata that I share to my MBP. It will hiccup every now and then but I think it has to do with other issues.

I’m thinking of rebuilding my cMP since I screwed up the file system by deleting the wrong partition so I’ve got some newer better items sitting to go in but I go to check out which processors it has I know they are 5,1 12 core but I’m don’t recall the speed or the speed of the ram plus want to update the bat and WiFi card to bt4 I have a pci card for it now but I’m thinking of replacing the mb bt card to free up a slot for something else. I’m upgrading the guy from a GTX 680 to a 1080ti replace oue of the HDd with a sst and the other three upgrading to 4tb red from 2tb red that will give it two suds 1tb ea in a raid on the saga card and 1tb Ssd and three 4tb red HD’s, 64 gb ram ,the 1080ti gpu, usb3 pci card, and the bt4 pci card until replaced I’m wanting to max this cMP out since I’m not paying 5 grand for a basic iMac pro.

 

[ernie81]

Hi Guys, thanks for the advice, there's been a lot to read through and research! After having a long think about my workflow, I've ordered a TerraMaster "DAS" drive with RAID mirroring, and will share that from the mac pro to the macbook.