NAS and organizing files

[mavericks7913]

Hi I have some questions and concerns about NAS storage and file organizing.

I have two external HDD(Both are different brands) for storing all raw files by using lightroom catalogue. I just use one catalogue and then create folders base on year, topic, and date. Btw, I'm start thinking to use Capture one pro rather than lightroom so I'm not sure about organizing files if I do that. In fact, I use only one catalogue to store all files together. Few people made catalogue by years. The biggest problem is storages. Two different external HDD already massed up my desk with a lot of cables(2 usb, 2 power, and 1 usb hub.) I don't know anything about NAS btw. Adding new HDD to lightroom catalogue was so painful job to do. I really not sure how to organize files with NAS system since it uses several HDD together. Can anyone help me with NAS and organizing files?

 

[kenoh]

At a very simple level, the NAS allows you to take multiple drives and have either a big unprotected set of storage or, you can use RAID to mirror and stripe the data across multiple disks in the NAS. You then have access to "drives" that you mount on your Mac.

At that point it is exactly the same as an external hard drive except you access the NAS across the network so it isnt as fast to access.

I use a synology 2 bay enclosure with a pair of 3TB drives mirrored so I get roughly 3tb storage but if a drive fails i can recover with out data loss.

I access it across my network and since upgrading my camera i am using wired connection on network as it is too slow on wifi.

 

[OreoCookie]

NAS stands for Networked Attached Storage. Put simply, you connect one or more volumes as a network share. Technically, a NAS is a little server, and the most important component is actually the software, not the hardware. That's why it is wise to invest in a quality NAS (the two default options here are Synology and QNAP).

A priori that has nothing to do with RAIDs, but only with the interface. The advantage is that you can serve files to many devices and many users simultaneously in ways that you can't (easily) if you use external hard drives that connect via USB, FireWire or Thunderbolt. NAS can also run various services, e. g. a Dropbox-like service or a VPN that allows you to connect to your home network when you are away.

While there one single-drive NAS, most of them come with 2 or more drive bays. Some consumer models support up to 18 bays. I have a 2-bay Synology with a 4 and a 6 TB hard drive. I chose not to use a RAID, but to keep them separate. Basically you need to use a form of RAID when you want to have a volume that is larger than the capacity of your hard drives and/or you want protection from drive failure. Synology offers to make volumes with a dynamical RAID level where you can add drives on the fly. This way you can adapt to a growing library of images.

I'm extremely happy with my NAS, but let me mention some of the downsides:
- Transfer rates are slower than via USB or FireWire. With a GBit connection, I get between 50-70 MB/s tops. This doesn't have to be a deal breaker — especially if you keep the catalog file on your local drive.
- Complexity increases, because you have to configure your NAS and care about issues like multiple machines accessing the same files. (Honestly, it's not that hard, but it forces you to take an inventory of your data. My suggestion is to start small and easy, and then work your way up.)

Nevertheless, a NAS has many advantages such as being platform-independent and relatively future-proof. My Synology has been rock solid and the software has received a steady stream of updates.

 

[phrehdd]

There is NAS and then there is DAS. The latter is what you are using but with single drive enclosure. DAS is directly attached storage. You have the option of getting a more robust DAS that can handle multiple drives and either hardware or software RAID if you want or as single drives within the enclosure.

The advantage of DAS is two-fold - speed of access and your computer in most cases has more control if you use either individual drive access or software RAID. Of these enclosures, they can be relative low cost or go up into some serious dollars. Of the DAS options - both USB3 and Thunderbolt will give you the best options. Single drives don't benefit from Thunderbolt over USB3. However, when you start to use RAID , Thunderbolt takes the lead.

I use a 5 bay QNAP NAS and also DAS. For what you talk about, unless others have to have access, a DAS might be a better way to go. OWC has a nice 4 bay Thunderbolt DAS and Areca might be a really upper line fit if you need 6 or more drives via Thunderbolt. There are of course other makers and as always, best to investigate features/specs then look for reviews by both professionals and end users.

If you have Thunderbolt connectivity, you could also consider a docking station and attach your devices to it rather than directly to your computer (tidy up the desk as it were and leave ports on the computer available. OWC, Belkin, Caldigit, Elgato all make standard docking stations and there are more exotic ones that include single internal hard drives such as the Echo series by Sonnet.

LAST - if you have more than one computer that is networked, you can always add a DAS to one and "Share" across a network. This for some is the best of both worlds with respect to NAS vs DAS.

 

[AFEPPL]

Another vote for NAS, while you will only ever get 120MB/s (limit of 1Gbe) it's plenty fast enough for pretty much anything including streaming HD movies to 5 ATVs at the same time.

Sharing a DAS over the network is pretty much the worst of all options. It's limited to the uptime of the other device, it impacts the performance of the other device and it removes somewhat the port abilities of the serving device. Also recovery or future proof options are not as good as with the NAS devices.

DAS is fine for 1990s or USB drives, but the worlds moved on.
Don't forget whichever option you look at RAID doesn't do anything to stop loss of files by deletion, so consider backups too.

 

[phrehdd]

Another vote for NAS, while you will only ever get 120MB/s (limit of 1Gbe) it's plenty fast enough for pretty much anything including streaming HD movies to 5 ATVs at the same time.

Sharing a DAS over the network is pretty much the worst of all options. It's limited to the uptime of the other device, it impacts the performance of the other device and it removes somewhat the port abilities of the serving device. Also recovery or future proof options are not as good as with the NAS devices.

DAS is fine for 1990s or USB drives, but the worlds moved on.
Don't forget whichever option you look at RAID doesn't do anything to stop loss of files by deletion, so consider backups too.

If someone has a single computer requiring fast access and an occasional need on another computer then DAS is ideal. As stated, I have a NAS and have dealt with various storage facilities for personal, SOHO and corp. There is nothing 1990's about DAS. In fact, external drives are DAS and sell like hotcakes so I'll just have to disagree with you.

To the OP, if your goal is to have ready access for your library, speed for edit and such, DAS is more advantageous. As for NAS uses, there is an overhead of permissions, when things go wrong you are dependent on knowing how your NAS software app/OS work and more. It would be a shame to steer someone towards a NAS who isn't interested in learning how to run one and just wants more storage for a library. Please do understand that I am keenly aware of using a NAS for media files (I have several terabytes worth) and ability to play back which is not the same as a photo library need for storage and access.

** Rather interesting that QNAP is starting to manufacture NAS with also direct access (Thunderbolt). I guess QNAP sees the 1990's light still shimmering.

 

[OreoCookie]

The advantage of DAS is two-fold - speed of access and your computer in most cases has more control if you use either individual drive access or software RAID. Of these enclosures, they can be relative low cost or go up into some serious dollars.

About the speed advantage of a DAS, you are correct. Although I'd strongly caution against using a softraid: this only works if the DAS is permanently connected and you can't just unmount the softraid volume and plug it into another Mac.

I use a 5 bay QNAP NAS and also DAS. For what you talk about, unless others have to have access, a DAS might be a better way to go. OWC has a nice 4 bay Thunderbolt DAS and Areca might be a really upper line fit if you need 6 or more drives via Thunderbolt. There are of course other makers and as always, best to investigate features/specs then look for reviews by both professionals and end users.

Yup, Thunderbolt is clearly the fastest interconnect out there. However, it is not clear for how many years Apple will include Thunderbolt ports. USB-C in its current incarnation has 5 or 10 GBit/s throughput, and it seems reasonable to expect that Apple may phase out Thunderbolt in favor of USB-C.

LAST - if you have more than one computer that is networked, you can always add a DAS to one and "Share" across a network. This for some is the best of both worlds with respect to NAS vs DAS.

This is not the same thing: first of all, it assumes that the machine which is connected to the DAS is running 24/7 (e. g. when your primary machine is a notebook) and does not change, and secondly, a NAS typically does more than just serve files. It's an always on box that can run a variety of services for you in the background. A NAS is also more future proof in the sense that it relies on interconnects that will be around for a long time. Note that you can use any NAS also as a DAS if you use iSCSI.

@OP
phredd's argument regarding speed is quite important, but can be solved in different ways: you can use your NAS to store older projects that you do not access regularly. Here, the difference in speed between a NAS and your internal drive is not that important. Thus, IMHO the best strategy is to keep current photos on your Mac's internal volume and then move older projects onto the NAS or DAS. Keep in mind that neither a DAS nor a NAS is a backup, even if you use a RAID level that allows for disk failure.

 

[phrehdd]

About the speed advantage of a DAS, you are correct. Although I'd strongly caution against using a softraid: this only works if the DAS is permanently connected and you can't just unmount the softraid volume and plug it into another Mac.

Yup, Thunderbolt is clearly the fastest interconnect out there. However, it is not clear for how many years Apple will include Thunderbolt ports. USB-C in its current incarnation has 5 or 10 GBit/s throughput, and it seems reasonable to expect that Apple may phase out Thunderbolt in favor of USB-C.

This is not the same thing: first of all, it assumes that the machine which is connected to the DAS is running 24/7 (e. g. when your primary machine is a notebook) and does not change, and secondly, a NAS typically does more than just serve files. It's an always on box that can run a variety of services for you in the background. A NAS is also more future proof in the sense that it relies on interconnects that will be around for a long time. Note that you can use any NAS also as a DAS if you use iSCSI.

@OP
phredd's argument regarding speed is quite important, but can be solved in different ways: you can use your NAS to store older projects that you do not access regularly. Here, the difference in speed between a NAS and your internal drive is not that important. Thus, IMHO the best strategy is to keep current photos on your Mac's internal volume and then move older projects onto the NAS or DAS. Keep in mind that neither a DAS nor a NAS is a backup, even if you use a RAID level that allows for disk failure.

I guess we'll disagree on some and agree on others. I enjoy the exchange here and civility. Much thanks!

 

[OreoCookie]

I guess we'll disagree on some and agree on others. I enjoy the exchange here and civility. Much thanks!

Thanks. You raised good points, and to be honest, I was just playing devil's advocate for the most part.

 

[AFEPPL]

If someone has a single computer requiring fast access and an occasional need on another computer then DAS is ideal. As stated, I have a NAS and have dealt with various storage facilities for personal, SOHO and corp. There is nothing 1990's about DAS. In fact, external drives are DAS and sell like hotcakes so I'll just have to disagree with you.

To the OP, if your goal is to have ready access for your library, speed for edit and such, DAS is more advantageous. As for NAS uses, there is an overhead of permissions, when things go wrong you are dependent on knowing how your NAS software app/OS work and more. It would be a shame to steer someone towards a NAS who isn't interested in learning how to run one and just wants more storage for a library. Please do understand that I am keenly aware of using a NAS for media files (I have several terabytes worth) and ability to play back which is not the same as a photo library need for storage and access.

** Rather interesting that QNAP is starting to manufacture NAS with also direct access (Thunderbolt). I guess QNAP sees the 1990's light still shimmering.

Selling like hot cakes for what though? It's a generalisation as to where a market is going without tangible data.
External HDDs can be used on TVs, PVRs and ever Xbox ones and PS4s. They are also very useful to add to the NAS drive as a means of backup.

The QNAP device is aimed at doing point one, you want to use your Mac mini as a dedicated fileserver, thats fine, some people might have hardware laying around to do that. Or maybe the Intel NUC would be an even better option but thats a subject on its own but still a decreasing market which is set to decrease even more with the cloud.

Unless your devices are either an iMac i.e. a static workstation or a dedicated mini, i would always go with NAS.
The OSs installed on them are general child proof, so are very easy to do what you need and if its only 1 user you don't need to worry about permissions at all.

 

[phrehdd]

Selling like hot cakes for what though? It's a generalisation as to where a market is going without tangible data.
External HDDs can be used on TVs, PVRs and ever Xbox ones and PS4s. They are also very useful to add to the NAS drive as a means of backup.

The QNAP device is aimed at doing point one, you want to use your Mac mini as a dedicated fileserver, thats fine, some people might have hardware laying around to do that. Or maybe the Intel NUC would be an even better option but thats a subject on its own but still a decreasing market which is set to decrease even more with the cloud.

Unless your devices are either an iMac i.e. a static workstation or a dedicated mini, i would always go with NAS.
The OSs installed on them are general child proof, so are very easy to do what you need and if its only 1 user you don't need to worry about permissions at all.

Some reality - most users of Macs don't want to think about a big set up for a home eco system. If they can plug in an external drive and get the results they want, they will do it. NAS (as I have said I am more than familiar with them and have had several over the years) brings on both a whole new set of options and overhead. The latter, along with price, often discourages the average user from going this path.

The original poster would get exactly what is needed for photo library from DAS. In this case, moving to a more robust external storage solution. One cable to handle everything whether it is USB or Thunderbolt. There is no mention of requirement to "share" data or access it from another computer. Everyone has a different work flow and way of storing their media files (photo, music and video/movie). In my case, I use NAS for all my movies and non AAC music. I DAS for projects to gain a library and access speed. As one person pointed out, one might take in progress projects and have them local and the rest stored on a NAS and the same logic also could apply towards a DAS.

Network storage - very slick and with a few pitfalls included when a network has issues, you have no connectivity to your files. When a drive fails and is swapped, the reconstitution often doesn't provide access to the drives until completed (large drives take many hours) or you can access at a turtle's pace. If someone gains access to your network, they gain access to the NAS and all your files if you don't secure the NAS (another overhead). While to someone like myself that would be business as usual, again the typical user doesn't want to worry or have the overhead. Just plug in a drive and go is the most common and popular way.

 

[AFEPPL]

While you have one version of "some reality" many others have a completely different one...

Why not simply replace the internal drive with a bigger capacity one? No info is provided as to what the actual device is - thats a million time better than external device. As for the comment about NAS cost, they can be had for the same cost or even less of that of a thunderbolt enclosure with some discs as you previously mentioned.

As for file access, well if someone got access to your external HDD they would also have access to your files so your point really doesn't make any sense if you are talking about security of data. Your previous point was permission add complexity, i disagree, you can simply use the default single user and save it in your keychain. if you have multiple users and wont to break it down further, its just point an click simplicity.

When a drive fails "your" typical user loses all their files, has to have cables and crap all over the place and has to compromise mobility when using a device like a laptop.

Horses for courses... I see no scenario in my workflows where a DAS device would or be more useful.

 

[phrehdd]

Horses for courses... I see no scenario in my workflows where a DAS device would or be more useful.

Plug in formatted DAS and use. - DONE.
Move DAS to another computer or along with laptop outside of home - DONE.

The list goes on. However, as stated, NAS has its place and as someone who has used both for years, I'll respectfully disagree with you. My only concern is providing food for thought to the original poster.

 

[kenoh]

OK, I am an IT guy... I design this crap for banks and hospitals.Thats my curse. You both make valid points. Bottom line...

Both right on most points but it comes down to:

1. Desired access - single or multiple
2. Access speed needs for read and writes
3. Fault tolerance needed
4. Volume of data needed
5. Backup strategy

 

[phrehdd]

OK, I am an IT guy... I design this crap for banks and hospitals.Thats my curse. You both make valid points. Bottom line...

Both right on most points but it comes down to:

1. Desired access - single or multiple
2. Access speed needs for read and writes
3. Fault tolerance needed
4. Volume of data needed
5. Backup strategy

It appears we are on the same page. I can only go by what the original poster offered as far as criteria or what is hoped for as a solution.

As for me - I wore many I.T. hats ranging from project management, technologist for road maps and quality assurance and of course hands on with hardware and networks, as well as working as a business systems analyst for an I.S. department. I tend to keep things oriented (here) towards Apple users and typical needs/wants.

 

[kenoh]

It appears we are on the same page. I can only go by what the original poster offered as far as criteria or what is hoped for as a solution.

As for me - I wore many I.T. hats ranging from project management, technologist for road maps and quality assurance and of course hands on with hardware and networks, as well as working as a business systems analyst for an I.S. department. I tend to keep things oriented (here) towards Apple users and typical needs/wants.

Yep. I laugh as a pro photog I know bangs on about digital ruining the industry as it lets "muppets who cant" take photographs... I laugh because here with commoditisation of NAS tech, we get end users discussing it. Unheard of 10 years ago!

By the sound if it we may have walked a similar path to arrive here...

Lets take some pictures!!!

 

[AFEPPL]

OK, I am an IT guy... I design this crap for banks and hospitals.Thats my curse. You both make valid points. Bottom line...

Both right on most points but it comes down to:

1. Desired access - single or multiple
2. Access speed needs for read and writes
3. Fault tolerance needed
4. Volume of data needed
5. Backup strategy

Strange, me too for some of the largest companies in the world..
25 years with Apple, Microsoft, Cisco and SAP including a world record project

But you missed variable change and mobility

 

[kenoh]

Strange, me too for some of the largest companies in the world..
25 years with Apple, Microsoft, Cisco and SAP including a world record project

But you missed variable change and mobility

Lol... not a competition. My biggies were the National Programme for IT, the deregulation of the electricity market systems and redesign of london stock exchange systems.

Varianle change and mobility are implicit not explicit sorry... i didnt miss them...

 

[phrehdd]

Yep. I laugh as a pro photog I know bangs on about digital ruining the industry as it lets "muppets who cant" take photographs... I laugh because here with commoditisation of NAS tech, we get end users discussing it. Unheard of 10 years ago!

By the sound if it we may have walked a similar path to arrive here...

Lets take some pictures!!!

Dang - you really don't have to work hard at all to get my agreement and consensus. (grin) Seriously, I fully understand the entirety of the change from film to digital in photography. I admit missing those days from time to time. I kept one of my old cameras just as a reminder. A simple manual camera - Nikon FM2. I don't have medium or 4x5 anymore but very tempted to get back into 4x5 for my own enjoyment. The only thing "digital" might be the light meter and timer for film development and contact sheet time heh.

 

[OreoCookie]

Some reality - most users of Macs don't want to think about a big set up for a home eco system. If they can plug in an external drive and get the results they want, they will do it. NAS (as I have said I am more than familiar with them and have had several over the years) brings on both a whole new set of options and overhead. The latter, along with price, often discourages the average user from going this path.

I think you are underestimating the choices you have to make when you buy a DAS as well: you have to choose RAID levels, hard- vs. software RAID, interconnect. But that doesn't mean it is the best. Setting up a NAS isn't that much more involved, you just need to follow a setup assistant, and you're done. The easiest solution for most is to get a larger external drive with one single hard drive.

Your argument that the OP doesn't need the extra functionality (reminded me of people wanting faster horses rather than cars): the OP doesn't necessarily know what he is missing. It's nice that my NAS box also acts as a print server and I can connect USB harddrives to it to share it across the network (or just quickly copy files). Media sharing is nice, too. And I can access its files from anywhere with an internet connection with my iOS devices.

I don't necessarily disagree with your conclusion (a DAS sounds like it could solve the OP's storage needs, but perhaps so does a single 8 TB external hard drive), I just think that the argument you're making is wrongheaded: some problems just require a more complex solution.

 

[phrehdd]

I think you are underestimating the choices you have to make when you buy a DAS as well: you have to choose RAID levels, hard- vs. software RAID, interconnect. But that doesn't mean it is the best. Setting up a NAS isn't that much more involved, you just need to follow a setup assistant, and you're done. The easiest solution for most is to get a larger external drive with one single hard drive.

Your argument that the OP doesn't need the extra functionality (reminded me of people wanting faster horses rather than cars): the OP doesn't necessarily know what he is missing. It's nice that my NAS box also acts as a print server and I can connect USB harddrives to it to share it across the network (or just quickly copy files). Media sharing is nice, too. And I can access its files from anywhere with an internet connection with my iOS devices.

I don't necessarily disagree with your conclusion (a DAS sounds like it could solve the OP's storage needs, but perhaps so does a single 8 TB external hard drive), I just think that the argument you're making is wrongheaded: some problems just require a more complex solution.

A fair enough response. We'll disagree at some level. Of course both NAS and DAS could be set up as simply a bunch of drives and be done with it. <G>

 

[mtbdudex]

Cool - IT geeks to help photo geeks
(I'm no way a IT geek, just a photo hobbyist)

Seriously I had my own "journey" on a NAS, in my case a WD EX2, which turned out was faulty and now I have a WD EX2100 , with (2) 6 TB Red drives in Raid1 config, thread below.
https://forums.macrumors.com/thread...hine-failures-to-nas-wd-my-cloud-ex2.1946452/

In my situation, having cat5e inwall enabled me to use the NAS as TimeMachine for all 4 of my Mac's on 3 floors of the home.

 

[mtbdudex]

Hi I have some questions and concerns about NAS storage and file organizing.

I have two external HDD(Both are different brands) for storing all raw files by using lightroom catalogue. I just use one catalogue and then create folders base on year, topic, and date. Btw, I'm start thinking to use Capture one pro rather than lightroom so I'm not sure about organizing files if I do that. In fact, I use only one catalogue to store all files together. Few people made catalogue by years. The biggest problem is storages. Two different external HDD already massed up my desk with a lot of cables(2 usb, 2 power, and 1 usb hub.) I don't know anything about NAS btw. Adding new HDD to lightroom catalogue was so painful job to do. I really not sure how to organize files with NAS system since it uses several HDD together. Can anyone help me with NAS and organizing files?

You asked this question 1 week ago - multiple replies, yet none from you.
Did your issue get solved?

 

[cmm]

OK, I am an IT guy... I design this crap for banks and hospitals.Thats my curse. You both make valid points. Bottom line...

Both right on most points but it comes down to:

1. Desired access - single or multiple
2. Access speed needs for read and writes
3. Fault tolerance needed
4. Volume of data needed
5. Backup strategy

I'm very sorry and I feel your pain. I work in government and feel bad for the infrastructure team. I sometimes bring them cookies with a smile, once, the team lead say that is the first time someone has smiled at them all week.

 

[phrehdd]

You asked this question 1 week ago - multiple replies, yet none from you.
Did your issue get solved?

Skipping over the topic - I saw your AVS spread and I admit my envy. A very classy theater room!