How do people handle large photo libraries? (few TB+)

[TwoBytes]

Does anyone have any tips?
One big library that could get corrupt worries me...but i like searching through the years and having ALL my photos in one library.

EDIT Yes i have backups. Opening photos is very sluggish as you can imagine with over a TB of data!

 

[chabig]

You won't lose anything, nor will the library get corrupted. It's not really one big library anyway. It's a database. The Photos app manages it well. My library is 41GB.

But if you think it has a problem, you can rebuild the database by launching Photos while holding cmd-opt.

 

[cthompson94]

Backup backup backup is how. I am not sure what software you use, but I personally use Lightroom. How you want to go about having your photos is up to you as far as catalogs go and whatnot. Since yours is quite massive you could maybe have the last year in a new catalog and go from there? and have the rest of your photos organized how you have it and import on a case-by-case basis so that your catalog isn't massive (mainly in load times). You could also go ahead and create separate catalogs for various year groups/by year/etc.

Personally since most of my shooting is of my daughter, family, friends now (at least until my daughter gets older), I just operate using one catalog that just has everything in that. I edit/store all of my photos on a M.2 SSD and that gets backed up to my Synology NAS (which is nice because the photos app for the Synology allows access to RAWs even on my phone and allows me to create albums/links to send to family/friends) then I use backblaze for off-site storage.

 

[Ray2]

I sat down and started to cull a few years ago. Unless the image was “published” in one manner or another it got trashed. That’s one approach to handling huge libraries. Not nearly as large as yours, from 150+ Gb down to 56 Gb with the task still not completed.

 

[mlblacy]

Does anyone have any tips?
One big library that could get corrupt worries me...but i like searching through the years and having ALL my photos in one library.

EDIT Yes i have backups. Opening photos is very sluggish as you can imagine with over a TB of data!

I have well over 1tb and I try and cull it down to the 125k photos or less (less is always better). IF you can easily split a large library down into two, that is helpful but not very practical for many who have the large libraries. I have noticed what seems to be a ram leak or soft crash, usually evidenced by a single image export taking longer than usual, and at the first sign of anything amiss I just quit the app and restart it. If it appears to be frozen or hung on a task, don't be too quick to to force quit the app, which is never a good idea on many database apps. It may sort itself out, and I have had tasks take 30-45mins. You can use Activity Monitor to check, but even if it is showing errors, it may still sort itself out in time.

I would approach rebuilding the database with caution. It may be slightly better than in Aperture, which seemed more prone to database corruptions, but still not trouble free. It can be a risky activity and should be a last, not first resort. I have had one total library meltdown and had to go back to a slightly older copy (and rebuilding was useless).

I would also make redundant backups. Time Machine, and also make multiple full library backups on multiple hard drives. By far the quickest and safest way I have found is to use Carbon Copy Cloner. When Aperture incremental backups were taking 8-12 hours, and full library copying could literally take days (my longest copy was 5 days)... CCC was night and day. About 8 hours to a day to do the initial copy and then incremental updates within minutes.

I keep the active library on a fast external drive (Thunderbolt3), and max the ram out on the machine too. Always look for little clues that the library is getting problematic or that the drive is getting flaky. Hard drives are relatively cheap, compared to the risks of lost work or material.

 

[cSalmon]

have a library consisting of many TB, my belief: File naming is the absolute most important aspect. I use to want my file names short and tidy no more make them descriptive so "you" can easily scan a list and find manually. Seems counter productive but time and time again we come back to this
My naming: YRMnDay_category_2ndaryCategory_subject_subjectDetail_0003

In my experience (hard knocks) DO NOT RELY ON SOFTWARE

In that every single piece of software I have every used is gone. Think about it, it's true
Or in the one singular case Adobe has morphed to the point of not usable for archived "work" (this is actually true as well but I'm not going to waste the time to explain) I will say Lightroom was dead in the water when it wasn't designed to deal with video clips and why I Know no software out there today will stand the test of time.

We're not talking bells and whistles we're talking time and evolution and still having access to past work. One of my first jobs was working with archeologists and deciphering notes from the 1800s even photo-notes from the 1900 and how hard they were to decipher because we had changed. What once was obvious, insignificant details left out, how we wrote... (we're talking highly educated scientists of their time extremely detailed notes yet time always has a way of getting it's cracks in there)

File naming
Folder naming
keep it away from software as much as possible
Your own exterior notes for your system

 

[Ben J.]

I use Lightroom Classic. Subscription, yes, but I think it's worth it, and it's been very solid for me the last couple of decades. Has tons of tools too, really everything I need in a photo editor and library app.

I want to mention one thing I really like about Lightroom; it has two basic ways to view photos; 1. as they are sorted in folders on disk, and 2. as collections that I create in lightroom.

So my image folder has been bulding up over the years, and is organized in the way I've found natural at any time. There are folders for scans of old prints, scans of slides, digital photos by year, etc.

I rarely re-organize my basic photos folder - although I could, inside Lightroom, and it would be reflected in Finder view. Instead I use the collections view in LR to organize stuff. And the hierachy is totally different; i.e I might have a collection called "Family" and it might contain both old prints, slides, and raw files, just as an example. Of course, collections only references the photo files, nothing is duplicated.

I find this concept works very well for me. Another approach is to use keywords/tags, but I never really got into that .

 

[dimme]

I use once every few months is Integrity checker which checks for data corruption. The one I use is at https://diglloydtools.com/integritychecker.html. But there are others out there.