I'd like to understand Timemachine Backups operating in greater detail if one day I have to recover my MBP from a TM backup.

[spnc]

I currently have a MBP M1 with 1TB disk space.

And a 2TB partition out of a larger hard drive for TimeMachine. This 2TB partition has 540Gb available.

Here's the screenshot of my TM partition content.

https://imgur.com/TE1pKeb

There's 29 snapshots since october 2022, which all equal between 850GB and 920GB each.

1. so how come their size don't add up, it should be like about 900GB times 29? Or they only add new files each time? but in this case why the size of each snapshot is the entire size of the backup, and not only the size of each incremental data of that backup?

2. and most importantly, in case one day I need to restore a TM backup on, let's say, a new Macbook, what happens if:

a. I delete let's say the very first backup made on 14 October, will the system be unable to recover part of, or even the entire, snapshots/TM backup?

b. I delete any of the other following backups in this list, will it mess up the restoration in any way (I suppose so)?

So in other words:

3. WHERE IS THE WHOLE OF THE TM DATA located across all 29 backups? mostly in the first/oldest (October 2022) snapshot?

4. DO YOU CONFIRM I SHOULD NEVER TOUCH A TM DISK CONTENT AND LET THE SYSTEM DELETE THE OLD SNAPSHOTS BY ITSELF

5. WHEN RESTORING TM ON A MAC, DOES IT RESTORE FROM SPECIFIC SNAPSHOT(S) OR DOES IT RESTORE FROM THE WHOLE OF THE TM DISK AS A SINGLE UNIT OF DATA (which includes, in turn, all the snapshots)? Do you have to choose the TM disk only to restore, or do you have to select a specific snapshot by the snapshot date of your liking?

Thank you.

 

[xraydoc]

I currently have a MBP M1 with 1TB disk space.

And a 2TB partition out of a larger hard drive for TimeMachine. This 2TB partition has 540Gb available.

Here's the screenshot of my TM partition content.

https://imgur.com/TE1pKeb

There's 29 snapshots since october 2022, which all equal between 850GB and 920GB each.

1. so how come their size don't add up, it should be like about 900GB times 29? Or they only add new files each time? but in this case why the size of each snapshot is the entire size of the backup, and not only the size of each incremental data of that backup?

2. and most importantly, in case one day I need to restore a TM backup on, let's say, a new Macbook, what happens if:

a. I delete let's say the very first backup made on 14 October, will the system be unable to recover part of, or even the entire, snapshots/TM backup?

b. I delete any of the other following backups in this list, will it mess up the restoration in any way (I suppose so)?

So in other words:

3. WHERE IS THE WHOLE OF THE TM DATA located across all 29 backups? mostly in the first/oldest (October 2022) snapshot?

4. DO YOU CONFIRM I SHOULD NEVER TOUCH A TM DISK CONTENT AND LET THE SYSTEM DELETE THE OLD SNAPSHOTS BY ITSELF

5. WHEN RESTORING TM ON A MAC, DOES IT RESTORE FROM SPECIFIC SNAPSHOT(S) OR DOES IT RESTORE FROM THE WHOLE OF THE TM DISK AS A SINGLE UNIT OF DATA (which includes, in turn, all the snapshots)? Do you have to choose the TM disk only to restore, or do you have to select a specific snapshot by the snapshot date of your liking?

Thank you.

The backups are incremental, meaning each time a file is added or changed, only those files get added to the backup (and older versions are kept as long as you have available storage space), so you won't really be able to add things up - the backup will always be larger than the total of the data you have on your main drive and will continue to get larger. Additionally, those folder sizes are not "real" - they're images of how big your drive is at the time, and can change depending on cache files, logs, off-line Time Machine incremental backups (like when your laptop is disconnected from the TM drive; snapshots will be saved until you next attach your TM drive). If you've got a 2TB Time Machine drive, there's no way to fit 29 "real" 900GB images.

Do not remove backups manually or edit the contents of those folders. Time Machine is not designed to be manipulated manually. It's not user-manageable. Just let it do its thing. If you need to recover a file, use the interface built in to Time Machine to navigate back to the version of the file you need.

If you lose your whole main drive, Time Machine can restore it to the state it was at the time of loss (well, up to the most recent hourly backup, technically). But if you mess with those folders on your Time Machine drive, you're screwed.

 

[MisterSavage]

Why would you want to delete some of the backups? Don't touch them. One time when an Apple Genius had to wipe my hard drive I just took my Mac home and restored from Time Machine. It was almost as if nothing had happened.

 

[TorbenIbsen]

Adding to xraydoc’s comment: The “incremental” backup method also means that your backup will keep files which you have deleted on the computer. And it will keep all changes made to files. So you can recreate any situation (more or less) by going back in time and restore a deleted file or a specific version of a file. The specific file will be created on the fly by Time Machine based upon the original, first backp plus/minus the changes made on til the date you specify as the restoration date. - But is has to be done by the Time Machine as things are built up to the point you want when you want it

 

[MajorFubar]

I currently have a MBP M1 with 1TB disk space.

And a 2TB partition out of a larger hard drive for TimeMachine. This 2TB partition has 540Gb available.

Here's the screenshot of my TM partition content.

https://imgur.com/TE1pKeb

There's 29 snapshots since october 2022, which all equal between 850GB and 920GB each.

1. so how come their size don't add up, it should be like about 900GB times 29? Or they only add new files each time? but in this case why the size of each snapshot is the entire size of the backup, and not only the size of each incremental data of that backup?

2. and most importantly, in case one day I need to restore a TM backup on, let's say, a new Macbook, what happens if:

a. I delete let's say the very first backup made on 14 October, will the system be unable to recover part of, or even the entire, snapshots/TM backup?

b. I delete any of the other following backups in this list, will it mess up the restoration in any way (I suppose so)?

So in other words:

5. WHEN RESTORING TM ON A MAC, DOES IT RESTORE FROM SPECIFIC SNAPSHOT(S) OR DOES IT RESTORE FROM THE WHOLE OF THE TM DISK AS A SINGLE UNIT OF DATA (which includes, in turn, all the snapshots)? Do you have to choose the TM disk only to restore, or do you have to select a specific snapshot by the snapshot date of your liking?

To supplement the correct information and warnings you have already been given, when you restore from a TM backup it will usually show the date and time of the latest incremental backup, and if you choose that date/time, it will restore the files backed-up at that time, plus the latest backups of everything else, from whatever earlier backups are required.

There used to be a warning on the underside of tech which said, "Do Not Remove Base. No User Serviceable Parts Inside." Pretend your TM backups carry the same warning and just leave them alone, otherwise if you do ever need to fully restore from a backup, your life will be just one sorry tale of misery and woe.

 

[spnc]

The backups are incremental, meaning each time a file is added or changed, only those files get added to the backup (and older versions are kept as long as you have available storage space), so you won't really be able to add things up - the backup will always be larger than the total of the data you have on your main drive and will continue to get larger. Additionally, those folder sizes are not "real" - they're images of how big your drive is at the time, and can change depending on cache files, logs, off-line Time Machine incremental backups (like when your laptop is disconnected from the TM drive; snapshots will be saved until you next attach your TM drive). If you've got a 2TB Time Machine drive, there's no way to fit 29 "real" 900GB images.

Do not remove backups manually or edit the contents of those folders. Time Machine is not designed to be manipulated manually. It's not user-manageable. Just let it do its thing. If you need to recover a file, use the interface built in to Time Machine to navigate back to the version of the file you need.

If you lose your whole main drive, Time Machine can restore it to the state it was at the time of loss (well, up to the most recent hourly backup, technically). But if you mess with those folders on your Time Machine drive, you're screwed.

Perfectly clear cheers yeah it's what I thought, thank you!

"The backups are incremental, meaning each time a file is added or changed, only those files get added to the backup (and older versions are kept as long as you have available storage space), so you won't really be able to add things up - the backup will always be larger than the total of the data you have on your main drive and will continue to get larger."

So it will only add newer versions of the same modified/added 'X', 'Y', etc. files, while keeping the older ones (to the limit of the remaining space available on the TM disk). This is why the overall disk space on my external drive is slowing growing (approx 1.5TB for a 1TB MBP after 6 months). Understood now.

By the way, when I upgrade my next MBP I wanna get a maxed out 8TB internal SSD. Sandisk apparently will finally release the first 8TB external SSD dongle drive in the coming months, would you say that if I only use about 7TB max on my internal MBP drive, 8TB external SSD dongle will be enough to make enough snapshots? I don't need that much history, something like 3/4 months snapshot history is definitely enough.

Regards.

 

[spnc]

Adding to xraydoc’s comment: The “incremental” backup method also means that your backup will keep files which you have deleted on the computer. And it will keep all changes made to files. So you can recreate any situation (more or less) by going back in time and restore a deleted file or a specific version of a file. The specific file will be created on the fly by Time Machine based upon the original, first backp plus/minus the changes made on til the date you specify as the restoration date. - But is has to be done by the Time Machine as things are built up to the point you want when you want it

Excellent information thank you, I was wondering this regarding deleted files.

1. Does it mean it keeps all deleted files 'FOREVER'? (I guess not since it would grow the TM backup way too large, for instance if I keep deleting a huge file lol it will every deleted version of it in the TM backup)

2. or does it only keep all deleted files indiscriminately, until the first date of the first available snapshop (October 2022 in my case), meaning any deleted file after this date (say September 2022) would no longer be available in the /Bin TM location?

 

[spnc]

To supplement the correct information and warnings you have already been given, when you restore from a TM backup it will usually show the date and time of the latest incremental backup, and if you choose that date/time, it will restore the files backed-up at that time, plus the latest backups of everything else, from whatever earlier backups are required.

There used to be a warning on the underside of tech which said, "Do Not Remove Base. No User Serviceable Parts Inside." Pretend your TM backups carry the same warning and just leave them alone, otherwise if you do ever need to fully restore from a backup, your life will be just one sorry tale of misery and woe.

Excellent indeed so I will just never touch anything, and when a restore situation occurs, will simply select the most recent date, which will quite logically restore the state of the files and system at this most current date (while still having the possibility later to have the entire snapshots at disposal in TM, and browse older versions of any of the files until the oldest snapshot in time, just the way it it now).

Cheers.

 

[xraydoc]

Excellent information thank you, I was wondering this regarding deleted files.

1. Does it mean it keeps all deleted files 'FOREVER'? (I guess not since it would grow the TM backup way too large, for instance if I keep deleting a huge file lol it will every deleted version of it in the TM backup)

2. or does it only keep all deleted files indiscriminately, until the first date of the first available snapshop (October 2022 in my case), meaning any deleted file after this date (say September 2022) would no longer be available in the /Bin TM location?

It will keep deleted files forever as long as there is space on your backup volume. If/when it runs out of space, it will purge older backups, so if a file has long since been deleted, it's possible it will be removed. But if you've got space, this could be years.

My personal rule is the Time Machine volume has 3-4x the amount of space of which Time Machine is backing up. So, if you had a 1TB internal drive, I would use a 3-4TB Time Machine drive (under ideal circumstances). But in reality, the amount of space used all depends on the frequency of changes to files already on your drive plus new files that get written.

For me, most of my old files are archival -- like old lectures I've given, old papers I've written, reference PDFs that I keep on hand -- that don't really get edited over and over again. So those usually don't get re-written to TM backups too much. It's mostly current projects and new files, so most of my TM backups are well under 500MB per hour. So I've got TM backups going back like 7 years for one of my machines (the others I replace too frequently to have more than a year or two of backups).

That said, I exclude my applications folder, virtual machine images and downloads folder from my Time Machine backups, since I'd rather just re-install apps from scratch than dig out an old copy, so app updates don't get backed up over and over again, ultimately saving Time Machine space. Same with VMs. And I don't keep anything important in my downloads folder, so I don't need to back up crap that I download then ultimately delete; if I keep it, I'll move it somewhere else appropriate.

 

[TorbenIbsen]

Excellent information thank you, I was wondering this regarding deleted files.

1. Does it mean it keeps all deleted files 'FOREVER'? (I guess not since it would grow the TM backup way too large, for instance if I keep deleting a huge file lol it will every deleted version of it in the TM backup)

2. or does it only keep all deleted files indiscriminately, until the first date of the first available snapshop (October 2022 in my case), meaning any deleted file after this date (say September 2022) would no longer be available in the /Bin TM location?

Time Maching simply keeps adding changes to the original (complete) first backup as long as there is storage enough. You will see that there is also some form of compression as time goes by. - For the first 24 hours there are individual increments hourly. They will later be compressed to day/week/month and possibly year if there is room enough. At some distant point in time the Time Machine will tell you that there is no more room and you will be asked to begin with a kompletely new total backup. When you do that you will no longer have the old backup unless you save that somewhere else first (!) or simply start on a new external harddisk. I have not bothered with the finer details about the compression but expect it to be pretty smart. These things are not made by idiots. So I guess that what is left after a compression is the state of the file on the end of the compression period.

 

[spnc]

Time Maching simply keeps adding changes to the original (complete) first backup as long as there is storage enough. You will see that there is also some form of compression as time goes by. - For the first 24 hours there are individual increments hourly. They will later be compressed to day/week/month and possibly year if there is room enough. At some distant point in time the Time Machine will tell you that there is no more room and you will be asked to begin with a kompletely new total backup. When you do that you will no longer have the old backup unless you save that somewhere else first (!) or simply start on a new external harddisk. I have not bothered with the finer details about the compression but expect it to be pretty smart. These things are not made by idiots. So I guess that what is left after a compression is the state of the file on the end of the compression period.

Thanks for the clarification. This is quite amazing to be honest! and why I've been using Mac for 15 years and never turning back. Time Machine only is such a game changer indeed...

 

[PauloSera]

There is nothing to understand. When you want to restore, you'll pick a date and time that is offered, and it will restore to that point. The end.

 

[gilby101]

Read my response to q 3 first.

Adding to what has already been said:

1. so how come their size don't add up, it should be like about 900GB times 29? Or they only add new files each time? but in this case why the size of each snapshot is the entire size of the backup, and not only the size of each incremental data of that backup?

The size of the snapshot (in Finder) is the size of the backup for that exact point in time.

If you want to see the size of data only in a single snapshot, use Disk Utility. Select the TM disk, Menu View > Show APFS snapshots. You may need to modify which columns are shown. Look for the "Private Size" column. That is the size of the data which is in that snapshot only.

My first backup was about 140GB in size (in Finder), but the private size (files only in that first backup) was only 20 GB.

2. and most importantly, in case one day I need to restore a TM backup on, let's say, a new Macbook, what happens if:

a. I delete let's say the very first backup made on 14 October, will the system be unable to recover part of, or even the entire, snapshots/TM backup?

If you delete the first backup: 1) you remove the snapshot, 2) delete the data (files) only in that snapshot. You do not mess any other backup. You can restore from any subsequent backup.

b. I delete any of the other following backups in this list, will it mess up the restoration in any way (I suppose so)?

If you delete the a backup: 1) you remove the snapshot, 2) delete the data (files) only in that snapshot. You do not mess any other backup. You can restore from any other - newer or older.

So in other words:

3. WHERE IS THE WHOLE OF THE TM DATA located across all 29 backups? mostly in the first/oldest (October 2022) snapshot?

This is not the way to think of it. The data from all 29 backups is all stored in the APFS file system. This is not directly visible. Each snapshot is a view of a specific subset of that data - the new and pre-existing files that were present when that one backup was created.

When you delete a single snapshot the storage freed up is the "private size" of that snapshot (see 1 above).

4. DO YOU CONFIRM I SHOULD NEVER TOUCH A TM DISK CONTENT AND LET THE SYSTEM DELETE THE OLD SNAPSHOTS BY ITSELF

There are ways to safely delete snapshots: a) using the `tmutil` command in Terminal, b) Disk Utility, or c) Finder. Mostly I prefer the `tmutil` command. You can only delete complete snapshots. Take care!! If you look for advice, make sure it relates to APFS format backups and not to the old HFS+ format.

5. WHEN RESTORING TM ON A MAC, DOES IT RESTORE FROM SPECIFIC SNAPSHOT(S) OR DOES IT RESTORE FROM THE WHOLE OF THE TM DISK AS A SINGLE UNIT OF DATA (which includes, in turn, all the snapshots)? Do you have to choose the TM disk only to restore, or do you have to select a specific snapshot by the snapshot date of your liking?

As others have said, you choose which snapshot to restore.

 

[gilby101]

For the first 24 hours there are individual increments hourly. They will later be compressed to day/week/month and possibly year if there is room enough.

Strictly this is not compression. The better description is "deletion" and "retention". Most hourly backups are deleted with one backup being retained each day. In turn (after a month) those are deleted with one being retained for each week.

At some distant point in time the Time Machine will tell you that there is no more room and you will be asked to begin with a kompletely new total backup.

When the disk fills up, Time Machine will first delete the oldest backups to make room for new ones.

When deleting backups can not longer provide space, that is your "distant point in time" and you would need to start a new backup - on a bigger disk.

 

[gilby101]

1. Does it mean it keeps all deleted files 'FOREVER'? (I guess not since it would grow the TM backup way too large, for instance if I keep deleting a huge file lol it will every deleted version of it in the TM backup)

2. or does it only keep all deleted files indiscriminately, until the first date of the first available snapshop (October 2022 in my case), meaning any deleted file after this date (say September 2022) would no longer be available in the /Bin TM location?

Sort of "forever".

If a files was present on date x, it will be included in date x's backup. Suppose you then delete it, or replace it with a new version (think spreadsheet you are modifying daily). The version which is in date x backup will kept forever with a few exceptions.
a) Old backups are purged by the system (to free up space for new backups), may be all the way up to date x - the version from date x will be deleted. Newer versions that file will still be present in newer backups.
b) As backups age, specific backups gets deleted whilst others are retained (once per hour, once per day for a month, once per week). If that file was only in a specific deleted backup - it will be gone. But any versions of that file in older or newer backups will still be there.

Versions of deleted and replaced files are kept so long as the backup when they were present is retained. This is great, but: There is a serious problem created by large files which are frequently (or continuously) being updated. Virtual Machines are the prime example for which there is no decent solution using Time Machine.

 

[gilby101]

he “incremental” backup method also means that your backup will keep files which you have deleted on the computer. And it will keep all changes made to files. So you can recreate any situation (more or less) by going back in time and restore a deleted file or a specific version of a file. The specific file will be created on the fly by Time Machine based upon the original, first backp plus/minus the changes made on til the date you specify as the restoration date. - But is has to be done by the Time Machine as things are built up to the point you want when you want it

It is simpler than this. TM does not keep changes to files. Rather it keeps the whole file whenever it changes. Restoring just involves coping each complete file from the chosen backup. There is no adding or deleting changes.

 

[gilby101]

To supplement the correct information and warnings you have already been given, when you restore from a TM backup it will usually show the date and time of the latest incremental backup, and if you choose that date/time, it will restore the files backed-up at that time, plus the latest backups of everything else, from whatever earlier backups are required.

I suggest that is the wrong way to think of it. Each backup snapshot includes all files which were present when the backup was done. So when restoring from a snapshot it restores all files in the snapshot. And only the files that are present in that snapshot.

 

[kitKAC]

https://eclecticlight.co/?s=time+machine

Howard's site has years of articles about Time Machine.