Please help...Need Advice for Photos App Migration to M1 Mac

[Spotlighter9]

I am finally replacing my 2009 15" Intel Macbook Pro 1TB running MacOS El Capitan with the new M1 16” Macbook Pro running MacOS Monterey.

I currently use the Mac Photos App as our central library (system library), combining/uploading family photos and movies taken on both my and my wife’s iPhones and a family DSLR. For backup, we use iCloud Photo Library and also backup the 2009 MBP and photos to our Airport Time Capsule (which requires the Photo Originals all be kept on the Mac).

This Photo library of 17 years has grown to 740GB on the MBP. This includes an inelegant co-mingling of my original iPhoto library from when Apple replaced the iPhoto App with the current Photos App. For example, A "Daisy Disk" scan still sees these as two separate libraries (a 315GB iPhoto library and a 427GB Photos App library), even though they are effectively combined in the Photos app. I would like to get away from/shed this legacy file organization and clean up this all into one streamlined photo library upon my migration to the new M1 MPB.

Notably, my iCloud Photo Library is only 613GB compared to the 740GB Mac Photos library that it backs up. I assume this discrepancy is because there are certain iPhoto/Photos App files/duplicates/redundancies/etc that iCloud does not need/use.

Questions:

Can I simply turn on iCloud Photo Library on the new MBP and download the Full Originals to populate the Photos App on my New M1 MPB or is this unwise?

Would the new MBP download all these photos into nicely organized "Photos App" subfolders?

Any guess on what might be in that extra 127GB of "photo" data on the 2009 MBP that is not in iCloud?

Are there any other issues/concerns that I should be mindful of?

Many Thanks!

 

[HDFan]

we use iCloud Photo Library and also backup the 2009 MBP and photos to our Airport Time Capsule

Are there any other issues/concerns that I should be mindful of?

If you care about your photos you need a 3-2-1 backup strategy. iCloud doesn't count. Only 1 of the 3 should be a Time Capsule since TM backups tend to fail. Consider one of of the inexpensive backup services (BackBlaze, Crashplan), and a cloned backup made with something like CCC. A cloned backup is much more reliable than TM.

 

[James_C]

Can I simply turn on iCloud Photo Library on the new MBP and download the Full Originals to populate the Photos App on my New M1 MPB or is this unwise?

The simple way to copy your library is to copy your photos library from your old Mac to your new one as this will be a lot quicker than downloading from iCloud. Apple Photo’s stores all photos in the Library file container. You don’t need to worry about the iPhoto library as when you upgraded your previous iPhoto library to the new Apple photos format, it should have moved all existing photos at the time into the new Photo’s library. This is why your iPhoto Library legacy is smaller than your current photos library.

Once you have copied the library you can then point the Photo’s app on your new Mac by holding the Alt key down while opening the photos app. This will allow you to select the Library file you copied over.

Would the new MBP download all these photos into nicely organized "Photos App" subfolders?

If you copy the library as suggested above, then it should look the same as your current Library structure. Syncing the library from iCloud photos instead will not add or change the subfolder structure to what you already have. Just to be clear it is unlikely that the Photos app on your old Mac is looking at old photos in your old iPhoto library as these should have been copied into your current Photo’s library.

Any guess on what might be in that extra 127GB of "photo" data on the 2009 MBP that is not in iCloud?

The most likely reason is photos that you have deleted in the Photos app, but you have not yet emptied the deleted photos folder.

Are there any other issues/concerns that I should be mindful of?

Firstly make sure you have a valid backup of your main photos library.

once you have copied the master library folder over, you will need to go into Photo’s preferences on your new Mac and turn on sync to iCloud ( it will not replace the photo’s already there, if they match). You should also designate your new Library to be Master Library by turning on “use as system photo Libary “

last item to be aware of are any photos that are externally referenced in the photos app. These are any photos that you added to the Photos app, but you changed the settings so that they were not automatically added to the library. You will know if you did this or not. Externally referenced photos are obviosly not in the Photos libary, but in another folder on your Mac, and they also will not be in your iCloud photos library either as they are not synced.

 

[123kev]

Hi, I have a very similar situation

The simple way to copy your library is to copy your photos library from your old Mac to your new one as this will be a lot quicker than downloading from iCloud. Apple Photo’s stores all photos in the Library file container. You don’t need to worry about the iPhoto library as when you upgraded your previous iPhoto library to the new Apple photos format, it should have moved all existing photos at the time into the new Photo’s library. This is why your iPhoto Library legacy is smaller than your current photos library.

Once you have copied the library you can then point the Photo’s app on your new Mac by holding the Alt key down while opening the photos app. This will allow you to select the Library file you copied over.



If you copy the library as suggested above, then it should look the same as your current Library structure. Syncing the library from iCloud photos instead will not add or change the subfolder structure to what you already have. Just to be clear it is unlikely that the Photos app on your old Mac is looking at old photos in your old iPhoto library as these should have been copied into your current Photo’s library.


The most likely reason is photos that you have deleted in the Photos app, but you have not yet emptied the deleted photos folder.


Firstly make sure you have a valid backup of your main photos library.

once you have copied the master library folder over, you will need to go into Photo’s preferences on your new Mac and turn on sync to iCloud ( it will not replace the photo’s already there, if they match). You should also designate your new Library to be Master Library by turning on “use as system photo Libary “

last item to be aware of are any photos that are externally referenced in the photos app. These are any photos that you added to the Photos app, but you changed the settings so that they were not automatically added to the library. You will know if you did this or not. Externally referenced photos are obviosly not in the Photos libary, but in another folder on your Mac, and they also will not be in your iCloud photos library either as they are not synced.

Hi James_C, I have a very similar situation....and need help!

I have a 2013 MBP with a 250Gb photo library (46,959 items) synced nicely to iCloud with originals stored on the MBP. I am retiring this MBP soon and have a new M1 16" MBP. It's a clean install so I've copied my 250Gb library over from the 2013 MBP to the new M1 MBP, pointed the Photos app to the new library, set it as the system library and turned on sync to iCloud.
So now, the library on the M1 MBP has ballooned to 402Gb (and rising), looking in the photo library I can see many (certainly hundreds so far) of duplicates, triplicates (even quadruplicates) and the text at the bottom of Photos app says 'Uploading 39,567 items' (but is decreasing) and the number of photos/videos is now 45,104 for some reason.
iCloud on the web still tallies with the original - 46,959 items
Also, there is now a new 'folder' in the sidebar next to Recents and Imports called 'Unable to Upload' and this has a few thousand photos in there, increasing by the minute.

Any idea what is going on? Is it really uploading that much and virtually duplicating my library? I am sorely tempted to stop it and delete the library (I have the original still on the 2013 mbp.

Any help appreciated!

Kevin

 

[James_C]

Hi James_C, I have a very similar situation....and need help!

I have a 2013 MBP with a 250Gb photo library (46,959 items) synced nicely to iCloud with originals stored on the MBP. I am retiring this MBP soon and have a new M1 16" MBP. It's a clean install so I've copied my 250Gb library over from the 2013 MBP to the new M1 MBP, pointed the Photos app to the new library, set it as the system library and turned on sync to iCloud.
So now, the library on the M1 MBP has ballooned to 402Gb (and rising), looking in the photo library I can see many (certainly hundreds so far) of duplicates, triplicates (even quadruplicates) and the text at the bottom of Photos app says 'Uploading 39,567 items' (but is decreasing) and the number of photos/videos is now 45,104 for some reason.
iCloud on the web still tallies with the original - 46,959 items
Also, there is now a new 'folder' in the sidebar next to Recents and Imports called 'Unable to Upload' and this has a few thousand photos in there, increasing by the minute.

Any idea what is going on? Is it really uploading that much and virtually duplicating my library? I am sorely tempted to stop it and delete the library (I have the original still on the 2013 mbp.

Any help appreciated!

Kevin

Not sure to be honest. Firstly I would suggest that you don't open the photos app on your old Mac while the new library is syncing back to iCloud.

If you did open or keep open Photos on you old Mac I and you still have your Original intact Photos Library on external media that you used to copy to your new Mac, I would be tempted to cancel and redo.

If the original libraries may have been compromised due to they may have been syncing to iCloud then I would wait until the process is complete, and use a tool to remove any duplicates.

 

[123kev]

Not sure to be honest. Firstly I would suggest that you don't open the photos app on your old Mac while the new library is syncing back to iCloud.

If you did open or keep open Photos on you old Mac I and you still have your Original intact Photos Library on external media that you used to copy to your new Mac, I would be tempted to cancel and redo.

If the original libraries may have been compromised due to they may have been syncing to iCloud then I would wait until the process is complete, and use a tool to remove any duplicates.

Thanks James,
The original mac was and is offline and the original library still on there.
I've left it all overnight and this morning it's 'updating' as opposed to 'uploading xx items' and is apparently settled at 385Gb (compare to the original 250-ish). The iCloud on the web is still the same and correct numbers of items (46,128 photos) but the new 385Gb library that is 'updating' is saying 44,276 photos, with 3,449 that it is 'unable to upload'.
Interestingly, I checked and some of (assuming all) of the 'unable to upload's are in iCloud, so maybe it won't upload them because of that. That said, every single item in the original library was uploaded by the old MBP so they should really all be 'unable to upload'...
I'm quite p'd at the whole thing. I'm going to leave it and see if it resolves itself today but I've a feeling I'll have to delete the entire library and iCloud and start again.

Cheers,

 

[Spotlighter9]

Sorry to hear you're having trouble. I too am not sure I can make any sense of it for you. iCloud Photo Library can be really finicky, especially when it comes to synching entire libraries. I recall when I originally uploaded my library to iCloud a few years ago it took over three weeks to fully upload it and was so temperamental that it kept starting over from the beginning each time I would pause, shut down, or disturb it in any way. The size of the library in the Photos App Synch Window also oddly grew as it uploaded photos to iCloud. A few calls to Apple support indicated that it was likely analyzing all the photos from the beginning to ensure they were the same and that the size expansion was due to it temporarily generating different thumbnail sizes for use by the cloud.

In your case, I suspect that iCloud is now busy uploading/comparing what it sees as a new library on a new device to ensure all photos on all devices/icloud are accurately synched. I agree with your approach to just let it work through until it is finished and then assess things. With that said, I would keep the old MBP offline and not delete anything anywhere without first making another backup/clone of your entire original library on the old MBP onto a separate external hard drive (assuming haven't done this already)...especially if you are retiring or wiping the old MBP. You don't want to discover a few years from now that you somehow lost photos in the migration process and not have an original library backup to check.

Thanks for sharing your situation, as it is affirming my decision to just populate my ordered 16” MBP with originals from iCloud Photo Library rather than risk dealing with the temperamental iCloud re-synching of a large ported library….especially since I am going from HFS to the new APFS.

 

[James_C]

Thanks James,
The original mac was and is offline and the original library still on there.
I've left it all overnight and this morning it's 'updating' as opposed to 'uploading xx items' and is apparently settled at 385Gb (compare to the original 250-ish). The iCloud on the web is still the same and correct numbers of items (46,128 photos) but the new 385Gb library that is 'updating' is saying 44,276 photos, with 3,449 that it is 'unable to upload'.
Interestingly, I checked and some of (assuming all) of the 'unable to upload's are in iCloud, so maybe it won't upload them because of that. That said, every single item in the original library was uploaded by the old MBP so they should really all be 'unable to upload'...
I'm quite p'd at the whole thing. I'm going to leave it and see if it resolves itself today but I've a feeling I'll have to delete the entire library and iCloud and start again.

Yep, probably best to let it do its stuff. How is it going is it complete yet ?

 

[123kev]

Yep, probably best to let it do its stuff. How is it going is it complete yet ?

Hi Spotlighter9 and James_C,
Thanks both for your replies. It finally finished doing its thing yesterday afternoon and....so far so good. The library size ballooned but then started reducing, I suspect Spotlighter9 was right and that it was holding temporary files, thumbnails and so on because it's finally settled on a size within 1Gb of the original size.
And on that note, the new library has exactly the same number of photos and videos as the old library now and the web iCloud, which is great. I'm still a little suspicious so I'm going through albums and randomly checking the size, resolution, metadata and so on of some photos against the original and again, I've looked at about 100 so far and every one has been the same as the original library.

A few interesting things though, it seemed to hang a lot on 'Updating'. After an overnight of no progress I looked at Activity Monitor and the Cloudphotod process was at 0% for a long time, even though it was plugged into the charger. A reboot seemed to kickstart it and then Cloudphotod and Photolibraryd would then spring into life in the 20-70% range each. Photos app would then provide a timeline, show tickers on uploads and so on. This hanging happened 3 or 4 times but each time a reboot seemed to solve it.
The 'unable to upload' folder had almost 4,000 photos in it, but when the thing finally finished, that folder disappeared. The status bar at the bottom said there were photos that could not be uploaded, with a View link but when clicked, it just showed a blank page and said 0 photos.

So, although I'm not sure on the duplicates yet (I've got Duplicate Photos Fixer Pro, need to use it) I'd tentatively say it appears to have worked. Maybe I was just being hasty and should have trusted it but I've never moved a library before to a new install (outside of Migration Assistant) and it just seemed to not be too informative (it's a big library, will take time) or demonstrate that it was actually doing something and not hanging.

Cheers,

 

[James_C]

@123kev, Great thanks for letting us know that it all appears to have gone successfully.

 

[kooky]

I also had the 'Unable to upload' blue icon appear under the main Apple Photos menu next to Imports/Recently Deleted after moving my system library to an external drive, which resulted in having to do the iCloud upload sync again. I had about 40k photos and it was progressing something like 20k to go, with 15k unable to upload. This seemed scary.

I also created the Smart folder with rule 'Photo is unable to upload to iCloud Photos' and this was 0 items matched.

I logged a call with Apple support and they indicated the reason it's unable to upload in this case is because the file already existed in iCloud, otherwise it would be duplicated. And I should just wait until it finishes (which was going to be another 2 days or so).

So I waited, and then once the updating was complete, the Unable to Upload icon started rapidly counting down and emptying, and eventually disappeared completely, and all was good.

So in summary it seems this is just the 'logic' the program goes through, however it is an unfortunate implementation in that there is no explanation to the user what on earth is happening, and the choice of words is poor.

 

[Spotlighter9]

So I waited, and then once the updating was complete, the Unable to Upload icon started rapidly counting down and emptying, and eventually disappeared completely, and all was good.

I'm glad things went well for you. I agree that the best practice with iCloud, is just let it finish synching...despite that being measured in days and not hours. I am also convinced that most issues are related to slow/choppy internet speeds, which is why I chose to simply turn on "download originals to this Mac." Most internet services provide crap "upload" speeds even with gigabit service. So with no photos already on my Mac, most of the transfer was done in the faster "download" mode. My system photo library definitely feels "cleaner" having done this...then if I had used migration assistant or a local transfer, I suspect.

 

[kooky]

To me my hard disk is always the master, iCloud always the copy, of my photos. I would never trust iCloud to be my master. Hence I strictly use 'download originals to this Mac', always. And ensure enough free HD space.

Therefore:

I noticed after setting the new system library and turning on iCloud Photos:

-if you turn it on inside System Preferences > Apple ID > Photos, it defaults to "Download originals to this Mac" (this was the option I had set previously in the library, so perhaps it maintains it)

-if you turn it on within Apple Photos > Preferences > iCloud , it defaults to "Optimize Mac storage". I think (gut feel) this is bad because: IF you don't trust the cloud and always rely on the cloud as a backup, not a master, then turning it on this way could make you very nervous indeed (that you have now severed some form of integrity you can never recover).



On another topic, of feeling cleaner, I did notice I had issues when I first switched my library and made a mental note to Repair the library first, before switching it in future.