Command Line pause/resume of Time Machine?

[Darf Nader]

Hi, first time poster after many years of reading....

I am trying to come up with a way to safely backup my Time Machine backup volumes offsite and I chose to try ARQ which uses s3 Glacier as its storage which is pretty cheap compared to other solutions which seem to offer a host of caveats that make it unsuitable for my needs. The point is to cover my bases in case I have a catastrophic failure that not only destroys my Mac but also my Time Machine drive as well, like if I had a fire. The goal is to backup just the Time Machine data offsite and nothing else. I found what appears to be the obvious way to do that (backup the Time Machine drive but exclude .Trashes as well as anything in a folder that ends with "inProgress" as this represents a Time Machine backup that is underway. This seems like it should safely just backup all other Time Machine Backups which are completed, but I am not sure if there are other files the TM touches and are shared that might befoul my offsite backup as a result. Therefore, what I would like to do is run a command line program that pauses TM backups before ARQ starts and then resumes TM backups when it is complete. (ARQ has this function built in so I could run whatever script I want before and after a backup job.) The problem is, I cannot find any documentation about touching Time Machine with the command line in any way. I know TM backups are very fussy and it doesn't take much to make them behave like they are corrupt, which is why I want to take this extra precaution that they are pristine. Anyone have any ideas?

 

[BrianBaughn]

I'm not going to do your research for you but I'll just mention that TM backups are mostly made of "hard links". I don't know how ARQ works, if it can or how it would handle those.

If a TM backup folder is, for example, 300GB in size then in most cases subsequent backup folders will be just about the same size. Being made mostly of hard links, if you try to copy the backup folders to another drive they'll be 300GB each. You can see that it adds up quickly. 20 TM backup folders would be 6TB of data.

 

[leman]

All aspects of TM can be controlled via tmutil command line tool. If you want to backup a TM backup, make sure that that backup is stored within a sparse bundle. Technically, a sparse bundle consists of many small files which together encode a filesystem volume. This is the way the issue Brian describes can be escaped. I backup dozens of TM backups to tape using IMB Spectrum Protect this way and it works like a charm.

 

[Darf Nader]

I'm not going to do your research for you but I'll just mention that TM backups are mostly made of "hard links". I don't know how ARQ works, if it can or how it would handle those.

Ah, that would be a problem... I don't know if ARQ is smart enough to detect hard links to the same inodes or not. Thanks for that bit of critical info.
[doublepost=1476202002][/doublepost]

All aspects of TM can be controlled via tmutil command line tool. If you want to backup a TM backup, make sure that that backup is stored within a sparse bundle. Technically, a sparse bundle consists of many small files which together encode a filesystem volume. This is the way the issue Brian describes can be escaped. I backup dozens of TM backups to tape using IMB Spectrum Protect this way and it works like a charm.

I will look into tmutil to see if that has functions to extract backups from it or not. ARQ's biggest problem is that copying files to S3 is just pitifully slow and will never be able to keep up with Time Machine.

What I might do is simply attach a 4 TB drive to a mac or linux box at my office or somewhere else and see about doing an rsync of the entire TB drive every after a successful TM backup is ran and suspend future TM backups until the rsync finishes. Subsequent rsync jobs would be just updates so it should only be as much data as a single TM backup and as long as I allow rsync to do deletes it will prune the remote copy when TM expires backups due to space considerations. One thing about rsync is at least it is smart enough to copy an inode only once regardless of the number of links to it.

But, the more I think about it, what you are doing makes a lot more sense.

So without looking at the details of tmutil, can you tell me how are you putting the TM backups to tape? Are you using dd or tar or are you using something a little more advanced? It's a lot cheaper to use "sneaker net" to carry tapes offsite to a safe location rather than pump massive amounts of data over both the source and target's internet link. I think you have the right idea because it's simple, reliable, and inexpensive. All I need to do is remember to rotate my tapes.

Thanks!

 

[leman]

So without looking at the details of tmutil, can you tell me how are you putting the TM backups to tape? Are you using dd or tar or are you using something a little more advanced? It's a lot cheaper to use "sneaker net" to carry tapes offsite to a safe location rather than pump massive amounts of data over both the source and target's internet link. I think you have the right idea because it's simple, reliable, and inexpensive. All I need to do is remember to rotate my tapes.

I am using this enterprise IBM software which is part of the university service. It simply performs daily incremental backups to redundant tape libraries at the central IT department. The TM backups secured this way are already in sparse bundles because the computer I back up is our server.

So unfortunately, I can't really give you any useful details, the tools I use are more or less predefined. Didn't need to set up much. I did set up hooks that disable server TM backup prior to tape backup start and then reenable it when the backup is done.

 

[Darf Nader]

I am using this enterprise IBM software which is part of the university service. It simply performs daily incremental backups to redundant tape libraries at the central IT department. The TM backups secured this way are already in sparse bundles because the computer I back up is our server.

So unfortunately, I can't really give you any useful details, the tools I use are more or less predefined. Didn't need to set up much. I did set up hooks that disable server TM backup prior to tape backup start and then reenable it when the backup is done.

Actually, knowing what those hooks are would be helpful no matter what solution I go with, honestly. Thanks.

 

[leman]

The hooks are just IBM-specific stuff. But if you want to disable TM backup without the GUI, its just

tmutil disable|enable

Look at man tmutil

 

[Darf Nader]

Thanks. I imagine the enterprise backup software is spendy so I now need to find a way to tapes cheaply. Too bad I don't have my old reel-to-reel anymore!

 

[KALLT]

Also:

tmutil startbackup
tmutil stopbackup