Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

On-Board and Off-Board iCloud Photos Storage?

purdnost

purdnost

macrumors 6502a
Original poster
Dec 2, 2018
500
132
I use iCloud to sync my photos and videos across devices as well as serve as a short-term backup. I don’t want to use iCloud as a long-term backup, however. I like to keep my iCloud library small as I’ve found that Photos runs much more efficiently than hosting my entire vast library.

So, I move all of my media from my phone to an external hard drive every month or so. The Photos library on my external drive seems to be corrupted now, maybe because it’s too big, or because the hard drive is failing.

I don’t think it’s possible to use iCloud on my phone for just new captures, while keeping everything else off my devices. If there is some workaround, I’d like to hear it.
 
I’m pretty sure I just discovered what happened to all of my photos to cause me to have to recover the data. One little setting in the Photos app: “Importing: Copy items to the Photos library.”

I just spend the last few hours ”importing” recovered photos my new library on the SSD and then deleting the recovered files, only to discover that the thumbnails were still displaying, but the originals couldn’t be referenced.

Fingers crossed for smooth sailing from here on out. 🤞🏼
 
  • Like
Reactions: lkalliance, Kimcha and BigMcGuire
rajs said:
What machine is your primary library on (machine type and how much RAM)

How large is your vast library (in gigs or terabytes). How many photos and videos in total stored within it ?

Thx
Click to expand...
2020 MacBook Pro, 2.3 GHz Quad-Core Intel Core i7, 16 GB RAM
Over 200,000 photos, around 20,000 videos
 
  • Wow
Reactions: BigMcGuire and rajs
purdnost said:
HDD (portable). I’ve thought about upgrading to a desktop drive for better speed and efficiency (and longevity, maybe). And importing photos has been taking extremely long, not sure why. I have also considered an SSD, but isn’t HDD better for long-term storage (and less risk of corruption)?
Click to expand...
I think that's your problem.

HDDs are WAY too slow for Apple Photos. Even much smaller libraries than yours won't run well on an HDD.

But an SSD could probably handle even a huge library like yours.

I'm using the Samsung T7 and am happy with it. But it's not the fastest SSD. For the highest speed you can buy an NVME drive and a separate USB 4 or even thunderbolt case to get the highest performance.

I also think that external HDDs are at a higher risk of corruption, because they have mechanical parts. If you move or drop your HDD while it's connected, it can easily damage it and lead to data loss.

With SSDs that's not a risk.

Either way, I wouldn't recommend to rely on an SSD or HDD not breaking. The risk for both is too high and it's extremely important to have a solid backup strategy.

Personally, I create client-side-encrypted cloud backups using ArqBackup. I also create a non-proprietary backup by exporting all photos and metadata using osxphotos and then also backing it up to the cloud using Arq.

I just documented my entire photos backup workflow here:
 
  • Like
Reactions: BigMcGuire and purdnost
x2 of what @Kimcha told you.

IMO you definitely need to:
  1. move your entire library onto a SSD (External - 2TB or 4TB is going to be needed based on your # of photos and videos. I've got ~ 130K photos and only ~5K videos and I am around 1.3 TB. On an SSD - external one - it is pretty fast.
  2. Definitely do a FULL backup on a regular basis of the above -- whether it be to another distinct and separate storage device at home or onto the cloud (ArqBackup, Backblaze, etc..)
 
  • Like
Reactions: Kimcha and purdnost
You’re correct that you couldn’t use your iPhone to just take pics if you wanted it to upload to iCloud automatically without also having your full library represented on the phone. If you wanted to, though, you could turn off iCloud Photo Library on your phone, and then sync manually to your Mac. Then your Mac could upload to iCloud.

You could even place a middle-man in there: connect your iPhone to your Mac, and instead of syncing right into Photos, use Image Capture to take the photos from your phone to a folder somewhere. Then you could go through those photos at your leisure, and drag the ones you want into Photos, where they’d be uploaded to iCloud.
 
  • Like
Reactions: purdnost
Kimcha said:
I would start with the entire library and see how it performs.

I think it will perform just fine.

If it doesn't, you can split it up into multiple libraries. But I don't think its necessary to do one per year. Splitting it at most into two libraries should be enough.
Click to expand...
I just picked up a SSD. I assume I should be formatting it to APFS since I’ll be using it with a Mac, right?
 
  • Like
Reactions: BigMcGuire
So, the data recovery scan found around 3.8 TB of photos and videos. Not sure I want to spend nearly $400 for a 4 TB SSD. The bulk of the photo space is used up my RAW files (old client wedding photos). Can’t bring myself to delete them even though I may never view them again. I could keep all my personal photos and videos on a 1 TB SSD… and upload my client files to a university/alumni drive (Box, OneDrive, or Google Drive). Decisions, decisions… What would you do?
 
Last edited:
  • Like
Reactions: BigMcGuire
purdnost said:
So, the data recovery scan found around 3.8 TB of photos and videos. Not sure I want to spend nearly $400 for a 4 TB SSD. The bulk of the photo space is used up my RAW files (old wedding photos). Can’t bring myself to delete them even though I may never view them again. I could keep all my personal photos and videos on a 1 TB SSD… and upload my client files to a university/alumni drive (Box, OneDrive, or Google Drive). Decisions, decisions… What would you do?
Click to expand...
If a 4TB SSD is out of the question, what about a nice little spinner? Every time I‘ve cut down data to save space, I’ve regretted it. But your call on that one. But yeah, what would I do? Buy a 4-6TB drive and save that stuff. :D

Definitely go the route of having your offline saved photos and your online personal photos/videos. My wife does this with her hobbyist photography - her iCloud Photos album is for personal photos and her REALLY good birding photos, then she has a photos album on a 4TB spinner to dump the hundreds of photos she takes while birding.

Glad to hear the recovery worked!
 
  • Like
Reactions: purdnost
The 4TB ”spinner“ is still a good drive. I think I’ll keep my RAW client files on it, and put all of my personal files on the SSD.
 
  • Like
Reactions: BigMcGuire
purdnost said:
The 4TB ”spinner“ is still a good drive. I think I’ll keep my RAW client files on it, and put all of my personal files on the SSD.
Click to expand...
I would do the same.

And I’m really happy to hear that you were able to restore your photos. That must have been so stressful!
 
  • Like
Reactions: purdnost
purdnost said:
Is the driver slower with APFS encrypted? My drives never leave my home, so I‘m think I may not need the encryption.
Click to expand...
I've done both and haven't noticed a difference in speed. I use it on a 4TB spinner and a 1TB SSD. I encrypt everything by default. But logically, it should be a tiny bit slower due to the need to encrypt/decrypt. But in my unscientific tests - I couldn't tell the difference.

Doing some sleuthing : https://apple.stackexchange.com/questions/336184/slow-write-speeds-with-apfs-encryped-external-ssd - Says that there's definitely speed differences when writing. Not sure if that above link is due to an OS problem back then (it is an older link).

I've done Blackmagic Disk Speed tests and haven't seen much of a difference between encrypted/unencrypted (on both my SSD/Spinner).
 
  • Like
Reactions: TheYayAreaLiving 🎗️
purdnost said:
Still having some major issues. I keep coming to my computer to find that only a small percentage of photos have transferred to the Photos app, with the progress bar paused any my portable spinner hard drive (source of the the photos) asleep. I have to force quit photos as it says it’s not responding. I then get the spinning pinwheel and am forced to hard power off my MacBook Pro because Finder stops responding. Running the latest version of Ventura.
Click to expand...
Don’t import directly from the spinner.

Copy the photos to the SSD first. Then import.

I imagine that will solve a lot of issues.

And make sure in photos settings, the “Copy photos to library” is enabled.

Otherwise photos will keep the originals in the original location
 
  • Like
Reactions: purdnost
purdnost said:
... Photos runs much more efficiently than hosting my entire vast library.

... The Photos library on my external drive seems to be corrupted now, maybe because it’s too big, or because the hard drive is failing.
Click to expand...

What machine is your primary library on (machine type and how much RAM)

How large is your vast library (in gigs or terabytes). How many photos and videos in total stored within it ?

Thx
 
purdnost said:
2020 MacBook Pro, 2.3 GHz Quad-Core Intel Core i7, 16 GB RAM
Over 200,000 photos, around 20,000 videos
Click to expand...
Wow, that IS a lot of photos.

You don’t need to have any devices storing your full-res images. In fact I don’t think you literally NEED to have your iCloud Photo Library on ANY device: the full-resolution images live in iCloud. I might be wrong about that, though. But if you don’t have at least one device or computer synced with iCloud then adding photos to the library becomes a manual browser-based process.
 
Is your external drive a HDD or SSD?

On HDDs Apple Photos it's painfully slow even with smaller libraries. But on SSDs it's very fast. My library is half the size of yours (at "only" 97,000 items), but the performance is excellent.
 
Another thing you could do:
  • Sync the library using iCloud.
  • Use osxphotos to export all the photos from the library with all metadata to a folder structure.
  • Delete the photos in the library
You can tell osxphotos to store all metadata in xmp files. And if you use APFS on your external drive, they will be cloned and won't use up any additional space.
 
Kimcha said:
Is your external drive a HDD or SSD?

On HDDs Apple Photos it's painfully slow even with smaller libraries. But on SSDs it's very fast. My library is half the size of yours (at "only" 97,000 items), but the performance is excellent.
Click to expand...
HDD (portable). I’ve thought about upgrading to a desktop drive for better speed and efficiency (and longevity, maybe). And importing photos has been taking extremely long, not sure why. I have also considered an SSD, but isn’t HDD better for long-term storage (and less risk of corruption)?
 
rajs said:
x2 of what @Kimcha told you.

IMO you definitely need to:
  1. move your entire library onto a SSD (External - 2TB or 4TB is going to be needed based on your # of photos and videos. I've got ~ 130K photos and only ~5K videos and I am around 1.3 TB. On an SSD - external one - it is pretty fast.
  2. Definitely do a FULL backup on a regular basis of the above -- whether it be to another distinct and separate storage device at home or onto the cloud (ArqBackup, Backblaze, etc..)
Click to expand...
About to order a portable SSD. Wondering if I should create a brand new library and import the photos by year, rather than attempting to copy the entire library over at once. Or does it even matter?
 
purdnost said:
About to order a portable SSD. Wondering if I should create a brand new library and import the photos by year, rather than attempting to copy the entire library over at once. Or does it even matter?
Click to expand...
I would start with the entire library and see how it performs.

I think it will perform just fine.

If it doesn't, you can split it up into multiple libraries. But I don't think its necessary to do one per year. Splitting it at most into two libraries should be enough.
 
purdnost said:
I just picked up a SSD. I assume I should be formatting it to APFS since I’ll be using it with a Mac, right?
Click to expand...
Yes. Make sure it’s encrypted so that if someone steals or finds your SSD, they can’t see your private photos.
 
I attempted to copy the library file over, but it seemed to be stuck on “preparing to copy”. So, after several minutes, I open the library package contents and navigate to originals and there’s nothing in there. Folder size is zero KB. Open the Photos library and there’s only 21,000 items, and they all reference the files on the source folder that I was copying from this morning. Talk about perfect timing… Do I need to do a library repair? Disk recovery? 😩
 
Did you just create this library this morning?

Were you using folders before that?

If that’s the case, I would copy all the folders to the SSD and then create a new library and import all the photos.

Be patient with the copying. I imagine you have over 1tb in photos. That’s going to take forever to copy.

Even the preparing step can take a long time.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back