Photos Library Too Large?

[g0ld2k]

My wife has an iPhone X with a photo library stored in iCloud Photos of roughly 140,565 photos, 2,340 videos taking up about 655 GB. The photos are set to “Optimize iPhone Storage” and there is 64 GB of free space on the device. She is having issues with apps that access her photos.


When posting to Instagram it can take about 5 minutes for the photos to load photos and another several minutes on selection.


Uploading to Facebook usually just hangs.


In the Photos app, recently captured videos take several minutes to load and photos can take over a minute to clear up.


She’s had this issue for a while (since before iOS 11) and on her previous phone (iPhone 7 Plus); we have tried doing a full reset and starting off fresh so I don’t think it’s related to something in her current configuration.


Also I don’t think it’s a bandwidth issue since Speedtest shows her getting about 200 Mbps down and 200 Mbps up.


I’m assuming it’s an issue with having such a large library. Is anyone else experiencing similar issues or have libraries this large or larger? Any thought on how to resolve this?

 

[Enjoylife1788]

My wife has an iPhone X with a photo library stored in iCloud Photos of roughly 140,565 photos, 2,340 videos taking up about 655 GB. The photos are set to “Optimize iPhone Storage” and there is 64 GB of free space on the device. She is having issues with apps that access her photos.


When posting to Instagram it can take about 5 minutes for the photos to load photos and another several minutes on selection.


Uploading to Facebook usually just hangs.


In the Photos app, recently captured videos take several minutes to load and photos can take over a minute to clear up.


She’s had this issue for a while (since before iOS 11) and on her previous phone (iPhone 7 Plus); we have tried doing a full reset and starting off fresh so I don’t think it’s related to something in her current configuration.


Also I don’t think it’s a bandwidth issue since Speedtest shows her getting about 200 Mbps down and 200 Mbps up.


I’m assuming it’s an issue with having such a large library. Is anyone else experiencing similar issues or have libraries this large or larger? Any thought on how to resolve this?

140 k photos and 2.3 k videos. Woah.

I mean I don’t think even Apple would have tested this case scenario in their stress testing of the iOS. This is a big exception.

 

[BrianBaughn]

140 k photos and 2.3 k videos. Woah.

I mean I don’t think even Apple would have tested this case scenario in their stress testing of the iOS. This is a big exception.

2TB of iCloud storage is $9.99/month. I don't know why not!

 

[Chazzle]

My wife has an iPhone X with a photo library stored in iCloud Photos of roughly 140,565 photos, 2,340 videos taking up about 655 GB. The photos are set to “Optimize iPhone Storage” and there is 64 GB of free space on the device. She is having issues with apps that access her photos.


When posting to Instagram it can take about 5 minutes for the photos to load photos and another several minutes on selection.


Uploading to Facebook usually just hangs.


In the Photos app, recently captured videos take several minutes to load and photos can take over a minute to clear up.


She’s had this issue for a while (since before iOS 11) and on her previous phone (iPhone 7 Plus); we have tried doing a full reset and starting off fresh so I don’t think it’s related to something in her current configuration.


Also I don’t think it’s a bandwidth issue since Speedtest shows her getting about 200 Mbps down and 200 Mbps up.


I’m assuming it’s an issue with having such a large library. Is anyone else experiencing similar issues or have libraries this large or larger? Any thought on how to resolve this?

I believe you nailed it. Library is simply too large.

 

[BrianBaughn]

Another thread on Photos Library size but doesn't really mention iOS:

https://forums.macrumors.com/threads/apple-photos-library-size.2060226/

The size of the library could be the issue. Another possibility I can imagine is corrupt image(s) in the library...and with that many your chances are high.

Also, I think I remember Photos having issues with some video formats. In my case they were videos taken on an old digital camera.

 

[Enjoylife1788]

2TB of iCloud storage is $9.99/month. I don't know why not!

Yes 2 tb is available. But that normally is for very large backups. Even a lot of big files for work. For you the number of files is an issue not the size of those files.

 

[cynics]

If she looks at photos in the photo app does it take the same amount of time for them to download from the cloud?

Some apps struggle with pulling photos from iCloud however I’ve never known Instagram to exhibit that behavior though. I have an app I use for work and it can’t even see iCloud photos in it. I need to view the photo and zoom in and out before it downloads and the app can see it.

 

[g0ld2k]

If she looks at photos in the photo app does it take the same amount of time for them to download from the cloud?

Some apps struggle with pulling photos from iCloud however I’ve never known Instagram to exhibit that behavior though. I have an app I use for work and it can’t even see iCloud photos in it. I need to view the photo and zoom in and out before it downloads and the app can see it.

Normally the Photos app is faster than Instagram and Facebook but there is definitely a delay.

I’m beginning to think there may be a corruption in the library. When we made calendars via the photos app for Christmas a handful of them had screwed up orientations (even though the library showed them in the correct orientation) and I worked with Apple to remove the photos and reimport them, which fixed those photos.

I’m thinking about trying to create a new iCloud account with a new Photos library and import my wife’s exported photo library into it and see if the same issues occur on my development iPhone 8 Plus running a restored backup of her phone. I think this might show if the library is corrupted or if there is a size/number of files limitation on iCloud Photos.

 

[HDFan]

Although the Photos app is very convent, the way it integrates with things like your desktop background, Apple TV, and other devices, etc. as your library grows storing everything in one file package can become problematic. If a file is lost or corrupted, as you discovered, it can be a problem. If you open the library package you can see the database file and the Masters file. If anything goes wrong, even if the file is there, it is difficult to find it given the obscure names that photos gives them.

You might want to consider using referenced files. In photos preferences/General if you unclick "Copy items to the Photos library" checkbox the photos will be left in their original location. As such they aren't as vulnerable to catastrophic failure if something happens to the library.

Advantages:

1. If you rename your photos when you import them (IMG 00007 renamed to "Big Sur Trip 2018 IMG 00007" you can always quickly find them via the finder if you put your keywords in the photo name. There are a number of utilities which will allow you to do this, such as A Better Finder Rename.

2. Fixing corrupted photos is as simple as finding the backup of those individual files and restoring it. Since the names are tagged this process is relatively simple.

3. Since you have thousands of files, versus one library, if a single file gets corrupted then its an easy fix as above. If the library gets corrupted then you pray that your backup is good.

Disadvantages

1. You lose the integration between devices and the web that Photos offers. The messy workaround is that you simply import into another iCloud photos library (not by reference) the ones that you want to have generally available. If Apple allowed iCloud to work with referenced photos this problem would go away.

If your library is going to get even bigger, you might want to consider a program such as Adobe Lightroom. Much more complex, but has many more tools for managing your pictures. As I have ~3TB of pictures this is the only viable option for me. If there are pictures that I want to use with iCloud integration, it is simple enough to get them into Photos. I select all photos with a 5 star rating, export them to a temporary directory, and then import them into an iCloud Photo library. So Photos only has a subset of all my photos, the ones that I want to have available everywhere.

 

[zorinlynx]

I'm amazed at the size of that library. My photo library dates back to 2006 and only has 25,000 or so photos in it. Is she a trigger-happy SLR user?

The times I use an SLR I don't even bother importing all the photos into my Photos library. I can come back from a day of shooting with over 500 frames, but most of those are bursts and I end up only saving the best photos to my Photos library.

 

[g0ld2k]

I'm amazed at the size of that library. My photo library dates back to 2006 and only has 25,000 or so photos in it. Is she a trigger-happy SLR user?

The times I use an SLR I don't even bother importing all the photos into my Photos library. I can come back from a day of shooting with over 500 frames, but most of those are bursts and I end up only saving the best photos to my Photos library.

Yeah, she takes a lot of photos, things were pretty manageable with lots of shots from photo walks and family events, but then she went photo crazy when our niece was born a few years ago and then 2.5 years ago when our son was born the number of photos shot everyday went through the roof.

 

[gigapocket1]

Is she shooting/uploading in raw? If so. That probably the problem as far as downloading/loading pictures.. iCloud tends to be slow in downloading large files

 

[g0ld2k]

Is she shooting/uploading in raw? If so. That probably the problem as far as downloading/loading pictures.. iCloud tends to be slow in downloading large files

Very little RAW, maybe a couple thousand out of the 140k photos.

 

[Small White Car]

I don't use iCloud photos so I don't have any advice, but just wanted to say that my personal library is at 129,389 right now so not everyone here thinks your wife's habits are unusual!

 

[zorinlynx]

Yeah, she takes a lot of photos, things were pretty manageable with lots of shots from photo walks and family events, but then she went photo crazy when our niece was born a few years ago and then 2.5 years ago when our son was born the number of photos shot everyday went through the roof.

I was just thinking about that. I don't have kids, but my parents have a whole bunch of big massive albums of baby photos of me, and this was back in the 70s and 80s when you needed to use film and have it developed, so it was a pain.

I can't imagine how many photos parents must take of their kids these days, when it's so effortless and costs nothing.

 

[Enjoylife1788]

I don't use iCloud photos so I don't have any advice, but just wanted to say that my personal library is at 129,389 right now so not everyone here thinks your wife's habits are unusual!

Honestly, I am sure a lot many of us would have huge numbers of photos and would even click as many. The problem is that many in the icloud. If you are click so many pictures then you got to have better manageability of photos and not rely on any cloud. Because I am sure those pictures mean so much to her and a single error on icloud and gone.

 

[TheSkywalker77]

Woah, that is quite the collection of photos and videos. I think the library is simply too large. Try cleaning up the photos when you have time on your hands. That might help you.

 

[unobtainium]

Repair your photo library. Hold down the option key when opening Photos on the Mac, then choose the option to repair. Back up BEFORE you do that.

You may need to turn off and re-enable iCloud Photo Library on your iDevices. Turn it off before you begin the repair on your Mac, turn it back on once repair is completed.

Hope that helps, if not try Google Photos.

 

[royaltaste]

Same here with 115k pictures. The delay you mentioned occurs in my case mainly with pictures I have just taken and which are not uploaded to iCloud yet.

 

[NazgulRR]

To OP, I'm seeing the same delays in these apps in a 50k item / 400GB library and it has been there across iOS 9, 10, 11 and iPhones 6, SE and 7. I think it's probably Apple not optimizing for such large libraries.

If a file is lost or corrupted, as you discovered, it can be a problem. If you open the library package you can see the database file and the Masters file. If anything goes wrong, even if the file is there, it is difficult to find it given the obscure names that photos gives them.

Are you referring photos on Apple servers getting corrupt? I.e., Apple's storage (and not your own Mac / iPhone drive) corrupting the images? Surely once the image is in the iCloud, Apple is making sure it does not get corrupted and any time you re-do the Photos Library (either optimized version or full versions) you get un-corrupted images?

 

[1711.tony]

Same here with 115k pictures. The delay you mentioned occurs in my case mainly with pictures I have just taken and which are not uploaded to iCloud yet.

I experience this problem too, with 105K photos.
However, I found that only occur after iOS 11.3 Beta 3 or 4... which I think this more than a bug instead..

Reported through Feedback app at that time, however, I gave up any thought that think Apple will fix it.

 

[redman042]

My wife and I also have large photo libraries in iCloud. My account has 167 GB of photos and videos, including some RAW DSLR images and 4k videos. The count is 7,393 photos and 1,409 videos, so while the count is a lot, it's about a factor of 20 less than your wife's (!). I don't have my wife's phone in front of me, but I believe she has 3-4 times the photos that I do (say 25-30,000) but less total storage used.

The Photos app on our phones has performed flawlessly and quickly with this quantity. Both phones are iPhone X 256 GB phones, set to "optimize storage".

On our Macbook, Photos works fine with my library, but in my wife's account it often has glitches (mainly large swaths of missing thumbnails and huge delays downloading the full-size image). Both are also set to "optimize storage" (the Mac's HD is not big enough for both our libraries). I've had to repair her Mac photos database a few times to resolve this issue, and at one point completely remove the library and re-download it. She rarely uses the Mac, and what seems to happen is that if her Mac account is not logged on for a while, the local library gets too far out of sync from the iCloud one, and they can't reconcile, causing these glitches. So I try to keep her logged in to keep it current, which is weird to have to do. This problem has persisted for two years through all the MacOS updates.

You and I are in the same boat in that we've committed our photo libraries to iCloud. There are huge conveniences associated with that, but it's nerve-wracking when something goes wrong and we have to start troubleshooting, because Apple gives us VERY few tools to manage the iCloud database or really understand what's going on. Thankfully, so far all the issues I've had are with the local copies, which in the worst case can just be wiped out and re-downloaded (over days or weeks, but at least it works). iCloud seems to be good enough to know that, unless you actually issue a delete command to a photo on your local device, you do not want the iCloud copy removed.

Your wife's issues also seem to be confined to the local copy of her photos database (in her case, on her phone). My suggestion is to make sure you have a current backup and full sync of Photos (a backup by itself does not sync all photos - go to All Photos and scroll to the bottom and make sure the status indicates you are synced). Then wipe the phone (erase all content and settings) and restore from the cloud. This will build a fresh copy of the Photos database downloaded from iCloud. My guess is this will take care of it.

It is possible she has stretched the database capabilities of the local Photos app too far. If so, she'll have to do some major culling of items or offload a big chunk from iCloud. I can tell more stories about the challenges of downloading batches of files from iCloud on the Mac or PC, but I'll save that for another thread. But definitely DO keep a local backup of all your iCloud photos. If you need tips, PM me.

Edit: I just re-read your post. You already did the wipe and restore. Given that, your best option might be to have her remove the first third or so of all the photos from iCloud after downloading to a local drive. Sorry, I'm not sure what else to suggest.

 

[Kognito]

Normally the Photos app is faster than Instagram and Facebook but there is definitely a delay.

If you get mostly normal behaviour with the Photos app on your wife's iPhone then it might be worth reporting this issue via Instagram as a bug. No doubt a huge library will slow down the Photos app a little, but 3rd party apps might not be tested to work with such large libraries given I imagine it's a bit of a rarity.

Might be best to think about offloading some of the older photos to reduce the size of the library in the long run.

 

[HDFan]

Are you referring photos on Apple servers getting corrupt? I.e., Apple's storage (and not your own Mac / iPhone drive) corrupting the images? Surely once the image is in the iCloud, Apple is making sure it does not get corrupted and any time you re-do the Photos Library (either optimized version or full versions) you get un-corrupted images?

I don't know. My guess would be that if you are propagating your photos library to iCloud and a photo becomes corrupted, there's a very good chance that the iCloud version will become corrupt as well.

I always have a "master" copy in disk directories. If your photos library gets corrupted you have, I think, just the photos repair tool. If a disk file is corrupted there are a lot of very good 3rd party tools for disk repair.

In short, never rely on one storage solution. As with backups be sure to have at least 3 copies on different media of your critical files.

 

[coneybeare]

Just wanted to throw in my $0.02, seeing the same problem with a library of only about 85k photos. Most evident in Instagram, when trying to post a same photo I just captured from the library.

Admittedly, the Instagram developers should not be trying to load the metadata for the entire library, but rather focus in batches of say, the first couple hundred. But this is definitely a problem exacerbated by a recent iOS update which seems to be causing a delay in the Photos api as it needs to grab info from the cloud instead of just locally. Anecdotally, I have found that the delay grows in areas of weak or slow network connectivity.