Want to set up a NAS and a backup system

[jrm27]

Hello all,

I'm on a 2016 MBP with 500G of space. This puppy is filling up and it's a daily chore to keep clearing out files and projects so I can keep enough room to keep working. I've got a handful of external hard drives that I swap around depending on the project I'm working on, but it's just a hassle. At one of my old jobs, I had a very nice drobo setup where I kept all my files in a RAID setup. It was great. But a job, new protocols, new WFH arrangments mean that I need to start from scratch.

Besides the plug and play nature of the drobo, I have no idea how to do any of this. Might anyone have a recommendation on how to set up a budget RAID NAS? Would it also be possible to use it as a Tiem Machine backup device, or scratch disk for my Adobe CC needs?

Any help one might be able to offer would be great. Actually, it doesn't have to be wireless or anything... if I can just plug it into a port on my laptop, that'd be preferable as my home network is sloooooow. Thanks for any direction!

-jon

 

[techwarrior]

NAS = Network Attached Storage

There are a number of NAS options, the more feature rich tend to be QNAP and Synology. Both make excellent NAS with 1Gbps Ethernet connectivity to your network, and both can host Time Machine backups. They are a bit involved in setting up, certainly not P&P.

Any NAS will tend to be slower than USB or Thunderbolt (TB) connected storage because 1Gbps network is less bandwidth than USB2, USB3, or TB.

When you say slow network, is that ethernet is slow, or WiFi? If WiFi, connect your Mac to the ethernet to improve performance to a NAS.

If speed is what you are after, OWC makes great RAID solutions that can connect by USB or TB2 or TB3. With USB or TB connected RAID solutions, it is plug and play. OWC sells their arrays on Amazon, or direct from the OWC site. Your Mac has USB-C\Thunderbolt 3 ports, so either a USB 3.1 or TB3 connected array would work. Drive I\O speed will be the bottleneck, so regardless of USB3.1 or TB3, file transfer speeds would be pretty similar with any of the OWC arrays.

When thinking about storage, the number of drives can help with capacity, speed and resiliency. For example, a 2 drive bay will allow RAID 1 which means data on D1 mirrors D2. You get the capacity of the smaller of the two drives. This tends to be a little slow because you write data to one drive and the RAID controller replicates it to the second, and since you typically put HDD in these, regardless of how fast the USB\TB connection is, the drive I\O is the bottleneck. A RAID 0 configuration will split the data between the two drives to give you faster read\write speeds, but not resilient to data loss. With RAID0, 1 drive failure will lose all data. With RAID1, 1 drive failure can recover the data from the healthy drive and rebuild the mirror image.

Jumping up to a 4 drive array allows RAID 5 which writes data to all 4 drives with a parity bit. Since you only write a fraction of the data to any given disk, the I\O is faster than a single drive (RAID1), or 2 drives (RAID0). Failure of any 1 drive allows recovery by simply putting a new drive in to replace the failed drive, and the other drives will rebuild the new drive. It is far better resiliency than a single drive, with faster read\write performance. Also with a 4 drive array, RAID 10 is possible, it is basically RAID 1 + RAID 0 so you write all data to two disks and they replicate the data to the second set of disks. With both RAID 5 and 10, one disk failure will not lose data.

As for capacity (assuming all drives are equal capacity):

RAID0 (2 disks) = capacity of D1 + D2 with no resiliency
RAID1 (2 disks) = capacity of D1 with resiliency
RAID5 (4 disks) = capacity of D1 + D2 + D3 with resiliency
RAID10 (4 disks) = Capacity of D1 + D2 with resiliency

You can speed up the USB\TB arrays by using faster drives, including SSHD or SSD.

 

[Erehy Dobon]

You are not the first person wanting to do this. There are plenty of tutorials online that explain how to do this, some written better than others.

techwarrior has given the names of two established NAS brands. Check their support sites for any official guides on doing this.

You mention no budget nor level of system administration load you are willing to endure. Reading the official documents should give you an idea about the effort needed to properly complete the task.

Old sysadmin’s adage: if it’s worth backing up, it’s worth backing up twice.

RAID-1 and RAID-10 get expensive with all the extra drives. They are still susceptible to complete data loss in the event of A.) a RAID controller failure, B.) hardware damage (fire, flood) and C.) theft.

Best practice would be to have a second set of backups on different hardware (covers A) and ideally stored offsite (covers B & C). Many of today’s IT staffers love quoting best practice as the only way but they are wrong. You can skip best practice but add risk while saving on time and money.

No one here can advise you what you must do, it’s your call on risk tolerance.

 

[jrm27]

Thanks all. Appreciate the info, and the fact that I'm not the first person who wants to set up something like this. Thanks for the resources, as it gives me a great place to start!

I think what I'm looking for is something that is physically attached. Ethernet isn't really an option in this old house as it would require running some wires which is not a possibility at the moment. I'm running some ethernet adapters at for now to feed a second wifi setup in my office, ut the speed in my office is 20mb on a good day, and 300mb on a normal day out in the living room. So accessing by physically attached storage would be best for now. I'll take a look at those options. Thanks for the leads!

 

[pldelisle]

As @Erehy Dobon said, you didn't provide enough clues on what you want to get.

Let's just start by saying Synology makes really good products. I have a DS1513+ (2013 model) and it's still rocking and fulfils all my needs even after 7 years. But first answer to the questions asked by @Erehy Dobon .

 

[s66]

Might anyone have a recommendation on how to set up a budget RAID NAS?

RAID: is a way to make multiple disks working together into a larger system and/or give you redundancy so that in case of a disk failure you do not lose data.

NAS: Network attached storage. Most (if not all) of the NAS devices will offer RAID as well.

Take a look at the offers of Synology to get an idea.

If I can just plug it into a port on my laptop, that'd be preferable as my home network is sloooooow.

NAS would not "plug into your laptop, but your home network being "slow" probably meant to say your connection from home to the Internet is slow, your actual local network would be unlikely to be slow unless it's old.

Would it also be possible to use it as a Tiem Machine backup device, or scratch disk for my Adobe CC needs?

NAS devices like synology mentioned above do offer a time machine service that can be configured to be used by your mac. It works great once you set it up, but you need to work a bit on it to get it going.

Scratch disk for Adobe ... might not be smart over a network (if it even works to start with). Adobe really likes local disks, and they can be faster as well.


Soem references to get you started:
RAID: https://en.wikipedia.org/wiki/RAID
NAS: https://en.wikipedia.org/wiki/Network-attached_storage
Time Machine on a NAS: https://nascompares.com/guide-to-synology-nas-with-time-machine/
Adobe Scratch Disk: https://helpx.adobe.com/uk/photoshop/using/scratch-disks-preferences.html

 

[jrm27]

Yep, I'm definitely realizing that I didn't provide enough info upfront. After looking through some of the info shared above, and having a little better idea of the options involved, perhaps I can update a little better.

So, I'm thinking:
Physically attached
4 bays
RAID-5 sounds right up my alley for storage (3 drives) and an extra dive for resiliency
Something with extra thunderbolt ports would be nice so I could hook up a monitor, or another accessory, etc...
I'd love to not spend more than $1000 (less is preferred, if possible)
Storage space, between 8 and 16 TB.
As far as admin load... I don't think I have the chops or knowledge to try and admin something tricky... so I'd like it to be "plug and play" as much as possible.

Thanks for the patience and the info suggested already. It is much appreciated!

 

[pldelisle]

I would suggest this one : https://www.qnap.com/en/product/tvs-472xt

It fits what you require (4 bays, physically attached via thunderbolt connection + network, storage space, HDMI, and is mostly plug and play with minimal configuration).

For the HDDs, consider Western Digital Red or Gold the size you want.

Keep in mind RAID 5 keeps you from using the equivalent of the storage of one drive for parity. For example, 3x 4 TB HDDs will result in 8 TB of usable storage.

 

[Erehy Dobon]

In addition to pldelisle’s helpful suggestion you should consider another Drobo since you had good experiences with it before.

Neither will check all of the boxes for an IT staffer’s best practice list but you can always revisit adding a NAS solution later.

 

[techwarrior]

Yep, I'm definitely realizing that I didn't provide enough info upfront. After looking through some of the info shared above, and having a little better idea of the options involved, perhaps I can update a little better.

So, I'm thinking:
Physically attached
4 bays
RAID-5 sounds right up my alley for storage (3 drives) and an extra dive for resiliency
Something with extra thunderbolt ports would be nice so I could hook up a monitor, or another accessory, etc...
I'd love to not spend more than $1000 (less is preferred, if possible)
Storage space, between 8 and 16 TB.
As far as admin load... I don't think I have the chops or knowledge to try and admin something tricky... so I'd like it to be "plug and play" as much as possible.

Thanks for the patience and the info suggested already. It is much appreciated!

I think you are headed in the right direction. To daisy chain TB ports, definitely go with a TB3 drive enclosure.

OWC can ship with or without drives, so price it both ways. If you can save a bit on drives bought elsewhere, not a bad idea. Also look at Amazon, they sometimes offer OWC a bit less than OWC itself.

As Erehy Dobon rightly points out, RAID is not an end all for data security. Off site storage, even cloud as a backup for important files you cannot afford to lose is important to think about. RAID may survive one disk failure, but 2 disks and controller fails can still leave you in trouble. Even if you just backup to another USB drive is wise for critical files, but off site has the added security of surviving a catastrophic site failure (house burns to the ground).

Ultimately, you have to decide how much risk you are willing to take.

As for the network issues, running Cat 5e\6 cabling is challenging in older homes. You may not be aware, but there are adapters that can use your house power lines to send ethernet to other rooms in the home. TPLink AV2000 units may give you close to 1Gbps, I have successfully used these on my parents home that was built about 75-100 years ago with minimal electrical updates. They are best when used on the same circuit, but do seem to work ok spanning circuits. Also, if you have coax cable (Cable TV) in your home, MOCA 2.x adapters can give you good results too.

Lots of worms in the can to sort out!

 

[Erehy Dobon]

16 TB of RAID storage at $1000 is going to be a stretch. Single 16 TB spinners are still around $400 retail at my last recent check.

One consideration is to add a fast NVMe SSD in an external TB3 enclosure for exclusive use as an Adobe scratch drive in conjunction with the RAID array.

I saw a 500 GB Sabrent PCIe 4.0 m.2 stick for $125 or so at Amazon a few days ago.

 

[pldelisle]

16 TB of RAID storage at $1000 is going to be a stretch. Single 16 TB spinners are still around $400 at my last recent check.

One consideration is to add a fast NVMe SSD in an external TB3 enclosure for exclusive use as an Adobe scratch drive in conjunction with the RAID array.

But even there, I would go with the Q-Nap. It won't be much of a difference I think and Q-Nap brings a lot more features than a simple enclosure. I would go with 4x 6TB Western Digital Red Pro WD6003FFBX, so 18 TB usable, 24 TB total. Having an NVME drive for caching would be a plus if we work directly on it.

But it's personal opinion. I love having network storage I can access from anywhere.

 

[Erehy Dobon]

Yes, that’s why I wrote “in conjunction.” The NVMe would just be for transient data, not long term non-volatile storage.

 

[jrm27]

I think you are headed in the right direction. To daisy chain TB ports, definitely go with a TB3 drive enclosure.

OWC can ship with or without drives, so price it both ways. If you can save a bit on drives bought elsewhere, not a bad idea. Also look at Amazon, they sometimes offer OWC a bit less than OWC itself.

As Erehy Dobon rightly points out, RAID is not an end all for data security. Off site storage, even cloud as a backup for important files you cannot afford to lose is important to think about. RAID may survive one disk failure, but 2 disks and controller fails can still leave you in trouble. Even if you just backup to another USB drive is wise for critical files, but off site has the added security of surviving a catastrophic site failure (house burns to the ground).

Ultimately, you have to decide how much risk you are willing to take.

As for the network issues, running Cat 5e\6 cabling is challenging in older homes. You may not be aware, but there are adapters that can use your house power lines to send ethernet to other rooms in the home. TPLink AV2000 units may give you close to 1Gbps, I have successfully used these on my parents home that was built about 75-100 years ago with minimal electrical updates. They are best when used on the same circuit, but do seem to work ok spanning circuits. Also, if you have coax cable (Cable TV) in your home, MOCA 2.x adapters can give you good results too.

Lots of worms in the can to sort out!

Yes, LOTS of worms to sort.

Interestingly enough, I have lost tons of tons of data in a house fire.. and that sucks a ton. Luckily offsite backups at my office helped me save completely irreplaceable data like a decade's worth of Lightroom libraries and video files. So, after this part of the equation gets sorted out, online/second-location systems are next. Some data I'm willing to risk, some, I'm not. So, hopefully the RAID can mitigate the risk a bit on the bulk of the files, while the most important ones get an extra dose of security.

I'm currently running a set of powerline adapters, and the speeds are still really rough. I'll take a look at thew MOCA adapters too. The previous owner had TV's in each room being fed from a media area under the stairs... so perhaps that wiring is better, more reliable than what is carrying my signals now. Or, maybe this old Airport Extreme is giving up the ghost finally. Lots of parts of the equation to figure out!

Thanks fore the direction.

16 TB of RAID storage at $1000 is going to be a stretch. Single 16 TB spinners are still around $400 retail at my last recent check.

One consideration is to add a fast NVMe SSD in an external TB3 enclosure for exclusive use as an Adobe scratch drive in conjunction with the RAID array.

I saw a 500 GB Sabrent PCIe 4.0 m.2 stick for $125 or so at Amazon a few days ago.

Running an SSD scratch sounds like a good idea too. Thanks for that! 16TB may not be reachable for me at the moment, but I'll see how close I can get for now! Thanks for the heads up on prices.
[automerge]1595282424[/automerge]

I would suggest this one : https://www.qnap.com/en/product/tvs-472xt

It fits what you require (4 bays, physically attached via thunderbolt connection + network, storage space, HDMI, and is mostly plug and play with minimal configuration).

For the HDDs, consider Western Digital Red or Gold the size you want.

Keep in mind RAID 5 keeps you from using the equivalent of the storage of one drive for parity. For example, 3x 4 TB HDDs will result in 8 TB of usable storage.

thanks for the link! Looking at it now.

 

[Erehy Dobon]

Normally, I buy a bare 3.5" HDD at a local bricks-and-mortar computer shop and shove it into the appropriate external enclosure.

I am currently waiting on delivery of one of these 16 TB external drives:

Seagate Expansion Desktop 16TB External Hard Drive HDD - USB 3.0 for PC & Laptop - 1-Year Rescue Service (STEB16000402), Black

Ideal for the home, office, or dorm, Seagate Expansion Desktop offers enormous desktop storage for photos, movies, music, and more. Backing up and transferring content is incredibly easy—just drag and drop! To get set up, connect the USB hard drive to a Windows computer for automatic recognition—...

www.amazon.com

and I never buy prepackaged, off-the-shelf name-brand drives.

However the current $309 price (discounted from $390) is unbeatable right now. Hell, I'd be happy to pay that now for the bare drive, forget about the enclosure. This could easily be a credible compact Plan B drive for offsite backup.

Here's that $120 Sabrent M.2 stick

SABRENT 500GB Rocket Nvme PCIe 4.0 M.2 2280 Internal SSD Maximum Performance Solid State Drive (Latest Version) (SB-ROCKET-NVMe4-500).

www.amazon.com

When I build my custom Windows PC hopefully in a few months I'll use one of these or a similar PCIe 4.0 NVMe M.2 drive. You'll still need to budget for a quality TB3 enclosure.

 

[jrm27]

So, perhaps something like this might fit the bill?
OWC Thunderbay 4: https://eshop.macsales.com/item/OWC/TB3IVKIT0GB/ - $394
4, 4 TB drives (12TB usable): https://www.newegg.com/seagate-ironwolf-st4000vn008-4tb/p/N82E16822179005 - $419

I mean, that's just a little over $800... which leaves some extra dough for soemthing like the sabrent stick and an enclosure.

 

[pldelisle]

So, perhaps something like this might fit the bill?
OWC Thunderbay 4: https://eshop.macsales.com/item/OWC/TB3IVKIT0GB/ - $394
4, 4 TB drives (12TB usable): https://www.newegg.com/seagate-ironwolf-st4000vn008-4tb/p/N82E16822179005 - $419

I mean, that's just a little over $800... which leaves some extra dough for soemthing like the sabrent stick and an enclosure.

Considering your budget, probably it will be fine. But I don’t like those dummy boxes.

 

[techwarrior]

I'm currently running a set of powerline adapters, and the speeds are still really rough.

Powerline adapters have evolved over time. First few generations were not all that great. AV2000 and AV1200 units from TPLink I have had good luck with. If your units are older generation, I agree they are not all that helpful.

Same goes for MOCA. MOCA 2.1 I recall is the latest standard, 1.x is kind of weak.

In the end, you get what you pay for, so if the network is a real problem, spend a bit more and make it right.

 

[jrm27]

Considering your budget, probably it will be fine. But I don’t like those dummy boxes.

I appreciate the feedack. I love the additional functionality that something like the Q-NAP offers, however, even empty that would push my budget to a peak. Is there something I should consider (besides the added functionality of a more robust box, ie. pcie card slots) that would make the dummy boxes less reliable?

Powerline adapters have evolved over time. First few generations were not all that great. AV2000 and AV1200 units from TPLink I have had good luck with. If your units are older generation, I agree they are not all that helpful.

Same goes for MOCA. MOCA 2.1 I recall is the latest standard, 1.x is kind of weak.

In the end, you get what you pay for, so if the network is a real problem, spend a bit more and make it right.

I definitely think I have old tech on my powerline adapters as they themselves are 4 years old. Sounds like it is time to upgrade. Thanks for the advice.

 

[HDFan]

that would make the dummy boxes less reliable?

If you are talking about mountable drives/enclosures you can run into problems with unexpected unmount. Safer to avoid 2nd tier vendors.

Seagate IronWolf (Pro optional) health management is supported by QNAP.

 

[Cyprusian]

NAS devices like synology mentioned above do offer a time machine service that can be configured to be used by your mac. It works great once you set it up, but you need to work a bit on it to get it going.

My own experience of using Time Machine to back up to a Synology DS218j (disk configuration: Synology Hybrid RAID 1) has not been good. It ran for a few weeks then Time Machine reported it had failed after verification and the entire backup had to be deleted then re-created to then fail again. I've researched the issue extensively and have found that I am far being the only one to experience it. I've attempted several solutions such as AFP and SMB protocols, Wi-Fi and LAN connections, running Time Machine on a legacy iMac on High Sierra and a MBP on Catalina, etc. All, sadly, to no avail. I also have participated in more than one MR thread on this subject. So, I've had to adopt alternative backup strategies while clinging to an optimistic hope that one day a future MacOS update will resolve the matter......

 

[pldelisle]

My own experience of using Time Machine to back up to a Synology DS218j (disk configuration: Synology Hybrid RAID 1) has not been good. It ran for a few weeks then Time Machine reported it had failed after verification and the entire backup had to be deleted then re-created to then fail again. I've researched the issue extensively and have found that I am far being the only one to experience it. I've attempted several solutions such as AFP and SMB protocols, Wi-Fi and LAN connections, running Time Machine on a legacy iMac on High Sierra and a MBP on Catalina, etc. All, sadly, to no avail. I also have participated in more than one MR thread on this subject. So, I've had to adopt alternative backup strategies while clinging to an optimistic hope that one day a future MacOS update will resolve the matter......

Time machine backup over 1gbps ethernet (wired) to my Synology DS1513+ has personally always worked nicely without any error for years.
SMB is fixed on Big Sur, but I used AFP before.

 

[pldelisle]

Is there something I should consider (besides the added functionality of a more robust box, ie. pcie card slots) that would make the dummy boxes less reliable?

It’s not really the hardware which is the problem. More the fact that it’s dummy. It doesn’t have a processor, file system is dependant from the Mac, unexpected cable disconnection can ruin your life... none of which are possible on a real NAS. A Synology/Qnap. Synology runs BTRFS which is very reliable and include many features for file integrity, checksumming... Qnap only runs EXT4 but certainly has an integrity verification too. CPUs are AES-NI, lets you create a VPN server to access your files from everywhere.

So it it’s really more on how it works than the hardware itself.

 

[HDFan]

My own experience of using Time Machine to back up to a Synology DS218j (disk configuration: Synology Hybrid RAID 1) has not been good. It ran for a few weeks then Time Machine reported it had failed after verification and the entire backup had to be deleted then re-created to then fail again.

Same here with QNAP. Failed verification yesterday for the Nth time, waiting on first backup now. Got Synology working for a while, but it stopped showing as an option in Time machine. Haven't been able to figure out why.

 

[pldelisle]

Same here with QNAP. Failed verification yesterday for the Nth time, waiting on first backup now. Got Synology working for a while, but it stopped showing as an option in Time machine. Haven't been able to figure out why.

Weird. It never failed on mine. Always detected correctly.