Managing Large Aperture Library

[boilerdam]

Hey folks!

I'm sure this topic has been beaten to death and beyond. I've spent quite some time researching this myself but I wanted an answer to the doubts/questions I now have. But first, basics:

I have a 3yr old Aperture 3 library which is 373gb in size (on a 750gb MBP). I do not have a backup and all pictures are managed.

My general workflow is:
1) Shoot landscapes or astro (star trails/stacks), holiday trips [normal pictures are JPEG, astro pics are RAW)
2) Import them from the SD card into Aperture directly
3) Delete ones that I don't want to reduce the number of pictures
4) Export JPEG versions for star stacking, time lapsing or panorama stitching
5) End of workflow.

So, now, I'm running out of space on my drive. I gather that the best option to free up space is to relocate the masters.

But, I haven't backed up my Aperture library ever and want to start it.

Questions:
1) Can I treat the "relocate the masters" option as a backup?
2) After relocating the masters, can I still use Vault as another option to backup the referenced images/versions left over on my MBP?

Any help would be great!

 

[flynz4]

Questions:
1) Can I treat the "relocate the masters" option as a backup?
2) After relocating the masters, can I still use Vault as another option to backup the referenced images/versions left over on my MBP?

Any help would be great!

1) Relocating the Originals is NOT a backup. You are just moving them from inside the library, to outside the library. You still only have 1 set
2) The vault ONLY backs up the library (including any managed originals).

I STRONGLY recommend that you pick up a couple of external drives. Use one for Time Machine backups of your entire system... use the 2nd as your "home" for your referenced originals.

You must have a minimum of two copies of all your data to be backed up.

Personally... I would also subscribe to a cloud backup service and use that for a 2nd backup destination. I personally use (and love) Crashplan. It is very inexpensive and gives you off-site disaster recovery.

/Jim

 

[MCAsan]

You are pointing out one of the major drawbacks of using a managed library vs a referenced library. The master/oriignal files in a referenced library can be on one or more external drives.

I have both Aperture and Lightroom referencing the same set 56,000 masters/originals that set on an external LaCie drive. I have Aperture do Valult backups to the LaCie drive and I have Lightroom do catalog backups to the Lacie drive. Naturally the LaCie drive is included in my Time Machine backups to my Time Capsule.

http://support.apple.com/kb/PH7625 http://www.lifeafterphotoshop.com/managed-vs-referenced-files-in-aperture/
https://www.video2brain.com/en/lessons/managed-vs-referenced-file-storage
http://archive.bagelturf.com/aparticles/qanda/files/8992c352f4d1429747200b3e06c215fe-42.php

 

[ChrisA]

Hey folks!

I'm sure this topic has been beaten to death and beyond. I've spent quite some time researching this myself but I wanted an answer to the doubts/questions I now have. But first, basics:

I have a 3yr old Aperture 3 library which is 373gb in size (on a 750gb MBP). I do not have a backup and all pictures are managed.

My general workflow is:
1) Shoot landscapes or astro (star trails/stacks), holiday trips [normal pictures are JPEG, astro pics are RAW)
2) Import them from the SD card into Aperture directly
3) Delete ones that I don't want to reduce the number of pictures
4) Export JPEG versions for star stacking, time lapsing or panorama stitching
5) End of workflow.

So, now, I'm running out of space on my drive. I gather that the best option to free up space is to relocate the masters.

But, I haven't backed up my Aperture library ever and want to start it.

Questions:
1) Can I treat the "relocate the masters" option as a backup?
2) After relocating the masters, can I still use Vault as another option to backup the referenced images/versions left over on my MBP?

Any help would be great!

First off if you have to ask these questions then you need to let Aperture manage the files, just like you are doing now. "Referenced" files are best left to people who understand a lot more about how Aperture works.

What you are going to have to do is first before you do anything else make at least TWO backups. Buy an external disk drive that is twice as big as all the data you are likely to have in the next three years. 4TB or 3TB should do it. Let Time Machine use it. Let TM use the entire drive and don't put anything else on that drive. It might take 24 hours for TM for backup everything. TM is not fast. Make it a habit to connect the Time Machine drive when ever you can. (You can also use a write less connection to an Apple router. This will save you from having to remember to connect the TM drive but it is slower and costs more.)

Then later get a second external drive and make a simple copy of the Aperture library. Just drag the files over to the new drive and wait. Then unplug this drive and place it in another room inside a small water proof fire safe. Yes, spend the $50 on the safe. The drag it over copy is easy but there are better ways starting with Aperture "vault" system but this is easy to do with no setup.

These disks can be slow USB drives because backs run in the background and you don't care how long they take.

OK now you have two independent backups and one is in a safe place that you can't mess with. This is the dead minimum backup plan. Do not mess with anything until you are here.

Now how to fix your problem with file space. It's easy buy a larger and FAST external drive. The fastest one you can afford and make it big At least 2TB. Create a NEW Aperture library on that drive. In Aperture a "Project" is a self contained set of files. Projects can be moved between libraries. You need to make some plan for which projects go in the library on the Macbook and which go in the external disk. DOn't do al the moving in one day. You can sort th
is out over time.

Make sure that Time machine is backing up your new external disk. It should be making a backup every hour. Because TM runs every hour the TM disk needs to be very large so it can retain the old versions of all your files. Should you make a mistake TM can roll back any change, if it has space to keep all the old versions. Get the biggest disk you can and let TM use tall of it.

Next subscribe to an on0line backup service like Crashplan or Backbllaze. It might take a month for the initial backup to complete but you do need an off site backup.


Still got that disk in the fire safe. Good don't ever touch it. Buy test another external drive and use it to hold Aperture "vaults". These are copies of the library but in a format that Aperture can quickly update or "refresh" and they are good ways to move rejects between libraries. Keep these some place as far from your computer as possible. The office at work is a good place. Periodically connect to Aperture and refresh the vault(s) then unplug and move it back to your safe place.


The rule i you need both
1) three copies of the data at al times and
2) the data to be in at least two different buildings as far apart as practical.

Make sure this is always the case even during a backup operations

Time Machine is the first line of defense and it runs automatically. It's a good system but not enough the copy in the fire safe and the on-line service complement TM.

 

[MCAsan]

Time Machine is the first line of defense and it runs automatically. It's a good system but not enough the copy in the fire safe and the on-line service complement TM.

I suggest TM is second line of defense for Aperture. First line is to use the Vault backups and make sure they are going to another drive. Then use TM to backup the internal drive with the library and also the external drive with the Vault backup. Having backups off site would be icing on the cake.

 

[boilerdam]

ChrisA, thanks for the extensive reply! Thanks everyone else as well.

I think I've come up with a solution that'll keep me safe & "backed up" for the short term. First of all, my backup needs aren't anything important. I'm single and my MBP is the sole computer and I'm the sole user. This means, my present daily life is worth the 600gb that I can carry in my MBP, that's it or

There are 3 types of files in there: pictures, videos/movies & documents. All of this for absolutely ALL data I ever have/had is approx 2TB tops (internal MBP drive & a few external drives with data from my old VAIO). Out of these, my most precious is the Aperture library which is about 375gb by the time I type this, which has the max importance. Fortunately, my OCD keeps the 3 types of files in 3 different folders on my internal drive. So, all pics (iPhoto-Aperture libraries, wallpapers, random pics etc) in exactly 1 folder, docs in 1 folder & videos in another; it's easier to manage/backup now. I have a 4th folder that I store downloads or any projects that I'm working on.

So, here's what I plan to do:

1) Create an Aperture Vault (Already did on a spare 1TB Seagate External drive)
2) Move Aperture Masters (Almost done -- to 1TB WD Passport)
3) Copy all docs (2GB) to a dedicated "Documents" drive [basically a copy of the docs folder] (Done)
4) Copy all movies to a movies drive (currently on a non-NAS drive connected to a thin client as a pseudo-NAS, might move it over to a proper NAS drive) (Done)
5) Copy the Pictures folder to a 2TB drive (Done, the Aperture library got backed up as well)
6) Clone the system as it is now (probably Carbon Copy Cloner, yet to do)

This will keep the 3 types of files in 3 different drives. The cloner will also keep a copy. The Aperture library, therefore, is backed up as a vault on 1 drive, as a library on the another drive AND by CCC on a separate drive.

I'm not really warm on the Time Machine idea. I don't change much on my MBP on a daily basis; for me, a sensible backup interval could very easily be once a month. If it's gonna back up every hour, it's not the solution for me, it's overkill; I would rather stick to manual copying like a caveman. If the TM backup interval can be altered, I have a spare 3TB drive sitting idle which can be thrown into use.

It's a bit different in a few aspects from the solutions all of you proposed but I think this might work for me with some diligence in keeping up a manual backup schedule.

 

[flynz4]

You are pointing out one of the major drawbacks of using a managed library vs a referenced library. The master/oriignal files in a referenced library can be on one or more external drives.

I have both Aperture and Lightroom referencing the same set 56,000 masters/originals that set on an external LaCie drive. I have Aperture do Valult backups to the LaCie drive and I have Lightroom do catalog backups to the Lacie drive. Naturally the LaCie drive is included in my Time Machine backups to my Time Capsule.

If you are using a referenced library in Aperture... and backing it up via a vault... do you realize that Aperture Vaults do NOT back up referenced originals? Only managed originals are backed up by an Aperture vault.

/Jim

----------

I think this might work for me with some diligence in keeping up a manual backup schedule.

Manual backup systems almost always fail... unless of course, you are not human.

/Jim

 

[MCAsan]

If you are using a referenced library in Aperture... and backing it up via a vault... do you realize that Aperture Vaults do NOT back up referenced originals? Only managed originals are backed up by an Aperture vault.

Agreed. That is why I use Time Machine to backup the entire file system including external library drives.

 

[boilerdam]

If you are using a referenced library in Aperture... and backing it up via a vault... do you realize that Aperture Vaults do NOT back up referenced originals? Only managed originals are backed up by an Aperture vault.

/Jim

----------



Manual backup systems almost always fail... unless of course, you are not human.

/Jim

Jim, if you were referencing me about the Aperture backup condition, then yes, I am aware that Aperture does not backup originals. I plant to counteract this by updating the vault before I move the originals for any new project I add.

And, haha, I know that the hard part would be to manually backup stuff. I just read a guide on solutions on stopping TM from making hourly backups. Therefore, I might add a TM drive and include the Aperture Masters drive to it.

 

[pezdaddy]

Regardless of backup solution, you should reconsider time machine for your entire system backup. If OSX fails, you can restore from your time machine backup.

 

[boilerdam]

Regardless of backup solution, you should reconsider time machine for your entire system backup. If OSX fails, you can restore from your time machine backup.

Yup, I found a tool called TimeMachineScheduler which is apparently available on the App Store. I'll use that to schedule monthly Time Machine backups and also disable its hourly snapshots. I think that would work fine for me.

I'm planning to get Carbon Copy Cloner as well, which will give me a bootable clone of the system. A monthly update should be good.

 

[flynz4]

Yup, I found a tool called TimeMachineScheduler which is apparently available on the App Store. I'll use that to schedule monthly Time Machine backups and also disable its hourly snapshots. I think that would work fine for me.

I'm planning to get Carbon Copy Cloner as well, which will give me a bootable clone of the system. A monthly update should be good.

I am curious... why do you want to restrict Time Machine backups... or better said... why would you care? They are essentially impact free. Isn't that like complaining that my wrist watch needlessly moves the second hand over 86,000 extra times a day since I barely ever look at it?

/Jim

 

[glenthompson]

Yup, I found a tool called TimeMachineScheduler which is apparently available on the App Store. I'll use that to schedule monthly Time Machine backups and also disable its hourly snapshots. I think that would work fine for me.

I'm planning to get Carbon Copy Cloner as well, which will give me a bootable clone of the system. A monthly update should be good.

Automated backups always work better. Read the various posts from people trying to salvage files because they haven't made a backup for months.

Remember, Time Machine doesn't backup everything every hour, just the files you've changed. Most of the time I never notice it running.

 

[Keebler]

Updated….not sure if you've seen this, but time to check out the front page thread - no more future development for Aperture

Apparently they're working on a new app for the masses to manage photos.

Cheers,
Keebler

 

[boilerdam]

I am curious... why do you want to restrict Time Machine backups... or better said... why would you care? They are essentially impact free. Isn't that like complaining that my wrist watch needlessly moves the second hand over 86,000 extra times a day since I barely ever look at it?

/Jim

LOL, good question and a fair analogy... If the wrist watch uses some limited resource apart from the battery charge 86,000 times a day, I will take it off. [] Similarly (based on my understanding of TM), if TM is going to make backups every hour of the 600gb+ data that I have on an MBP with limited RAM and then turn around and store it (thereby eating up some free space) on my MBP, I want to avoid it because I might make a change on barely a few docs in the space of a week.

I know my usage and the data that I have... Apart from the Aperture library, if my MBP crashes now, I don't mind losing the data... I have nothing that critical on there anyway. 'course, I don't want it to happen... Thanks to all of you, Aperture is backed up and the rest of my stuff is still in the process.

My most critical data is a 20MB set of docs that have always been on an encrypted 256mb flash drive from 2004 and a physical copy at my parents' house on the other side of the world (literally) LOL

Having said that, I've read up on what TM does and apparently it backs up just the delta in the docs and local snapshots every hour. Now, I'll give it a whirl for some time and if these local snapshots eat up whatever's left of my MBP, I'll disable it.

I do highly appreciate everyone's concern and I would share the same concern if I was helping someone else. Down the road, when I have data that I cannot lose, I'll setup a triple redundancy "fool-proof" system.

----------

Automated backups always work better. Read the various posts from people trying to salvage files because they haven't made a backup for months.

Remember, Time Machine doesn't backup everything every hour, just the files you've changed. Most of the time I never notice it running.

Thanks Glen. I'll give it a test run. I agree that automated backups are the best method but in the end, my TM drive is not going to be wireless but a USB connection. No matter how much automation is built in, if I don't plug it in, it just won't backup.

If the hourly local snapshots which TM makes up for not having a constant connection to the TM drive takes too much space, I'll disable it.

----------

Updated….not sure if you've seen this, but time to check out the front page thread - no more future development for Aperture

Apparently they're working on a new app for the masses to manage photos.

Cheers,
Keebler

Yup, I read it earlier in the day... quite unfortunate. But does it make any difference? Aperture hardly got any sort of update since 2010 except for small RAW image updates, which will continue to trickle in according to Apple. So, Aperture will continue to behave as it always has! It's been good all this while and we can expect it to work exactly the same way in the future (with Apple's promised compatibility in Yosemite)... Seems like a good enough situation to me []

 

[Mike Boreham]

If the hourly local snapshots which TM makes up for not having a constant connection to the TM drive takes too much space, I'll disable it.

Local snapshots are only made if space is available.

If the space used by local snapshots is needed, it is given up automatically. There is no need to disable it.

Of course it means local snapshots cannot be relied on because they are regarded as expendable.

All explained better here:

http://support.apple.com/kb/ht4878

 

[ChrisA]

.
Having said that, I've read up on what TM does and apparently it backs up just the delta in the docs and local snapshots every hour. Now, I'll give it a whirl for some time and if these local snapshots eat up whatever's left of my MBP, I'll disable it.

Yes you are correct that TM only backs up what has changed urging the list hour. If you were not using the computer than likely nothing changed.

But then yo seem to think TM is using up space on the MBP. What? You think it is backing up files onto the internal hard drive? That would be completely pointless. The data are written to an external disk, either one that is directly connoted with USB, zThunderbolt or a networked disk.

TM will use up whatever space you give it. It will write to the external disk until it is full them it will delete the older backups to make room for new. The TM disk is used ONLY for TM and you should expect it to fill up eventually. If you need to buy a TM disk, get one that is much laster (2x) then all the data you have. That way there s room for oder copies of the data.

----------

.. will continue to behave as it always has! It's been good all this while and we can expect it to work exactly the same way in the future (with Apple's promised compatibility in Yosemite)...

Aperture might work in "yosemite" but I doubt it will work in future Mac OS X releases after that. Apple is giving you maybe a year or so to find something else.

I suspect by then Adobe and others will have migration tools that can read the Aperture library and make the move easy. That is my plan, to stick with Aperture until I find some easy way to move and to keep my non-destructive edits.

 

[Mike Boreham]

But then yo seem to think TM is using up space on the MBP. What? You think it is backing up files onto the internal hard drive? That would be completely pointless. The data are written to an external disk, either one that is directly connoted with USB, zThunderbolt or a networked disk.

Yes, local snapshots are written by Time Machine to the internal hard drive on portable macs, if the external TM drive is disconnected, as it might often be on a portable mac. The local snapshots are transferred to the external drive when it is connected.

See my post #16 above and the link to the Apple article.