Big Sur finally uses APFS snapshots for Time Machine backups!

[canhaz]

I think what's confusing folks, well certainly me that is, most people view HFS+ and APFS as two competing filesystems, with APFS being newer and supposedly better.

"Apple's new shiny APFS replaces HFS+" is kinda a well known known in the public's mind.

Thus when we talk about embedded APFS inside HFS+.. well I dunno, but that all just seems rather odd.

Like I wonder if there are additional performance tradeoffs with that sort of thing. APFS I imagine would probably be happiest talking to bare metal.

 

[chrfr]

I think what's confusing folks, well certainly me that is, most people view HFS+ and APFS as two competing filesystems, with APFS being newer and supposedly better.

"Apple's new shiny APFS replaces HFS+" is kinda a well known known in the public's mind.

Thus when we talk about embedded APFS inside HFS+.. well I dunno, but that all just seems rather odd.

Like I wonder if there are additional performance tradeoffs with that sort of thing. APFS I imagine would probably be happiest talking to bare metal.

The underlying filesystem on a network-connected disk is invisible to the client system. The client just knows whether or not it can write a file to that drive. The "translation" is handled by the file sharing software, loosely speaking. Time Machine knows that it's able to save a disk image onto the Time Capsule drive, and it formats that disk image as APFS when you're using Big Sur and making a new backup. Existing backups are not converted from HFS+, and there's no way to convert them.

 

[viettanium]

in Big Sur, my backup tasks to AirPort Time Capsule is very slow except the first complete one.
updated to 11.1 beta but still the same!

 

[canhaz]

The underlying filesystem on a network-connected disk is invisible to the client system. The client just knows whether or not it can write a file to that drive. The "translation" is handled by the file sharing software, loosely speaking. Time Machine knows that it's able to save a disk image onto the Time Capsule drive, and it formats that disk image as APFS when you're using Big Sur and making a new backup. Existing backups are not converted from HFS+, and there's no way to convert them.

Thanks. I did know about that, but needed a reminder. The key to me was that APFS (and HFS+) can be a real or a virtual file system. When virtual it's can present itself as sparse images/bundles. A similar concept to sparse images (.sparseimage) is the very common .dmg image file. A key difference is .dmg uses and equal real and virtual space, where a sparse image can be smaller than the space it represents. But the general ideas is the same, it's virtual file system withing a filesystem.

Sparse images and sparse bundles are just regular files, and hence @chfr and others are correct, if APFS is exposed via a networking protocol like AFP, SMB or NFS, then it doesn't matter what underlying file system is.

Good refresher here. https://eclecticlight.co/2020/04/27/sparse-bundles-what-they-are-and-how-to-work-around-their-bugs/

That said, does Big Sur still use APFS sparse bundles for locally attached USB mounted drives?

Or, does Time Machine in Big Sur instead switch strategies and use APFS directly including its clone features?

 

[chrfr]

Or, does it instead use it switch strategies and use APFS directly including its clone features?

A new Big Sur backup to a local drive doesn't use sparsebundles but does use APFS snapshots. Howard has an article about this too: https://eclecticlight.co/2020/06/29...r-how-time-machine-backs-up-to-apfs-and-more/

 

[estesbubba]

I have a 2TB external HDD that I want to format APFS to use for data and backups for 2 MacBook Pros. What's the best way to format/partition/volume it? Here is what the sizes of each partition currently are but I want to erase and start clean.

Data - 500MB
Home Backup - 1GB
Work Backup - 500MB

It would be nice for Data to use what it needs and decease the backup sizes if needed. Is this possible? Do I create a single partition with multiple volumes? Should I use reserve and/or quota for any of these?

 

[StellarVixen]

How well does APFS work with spinning rust?

 

[Z3man]

I wondered why my backups were faster using Big Sur.

Makes no difference now anyway as i have gone back to Catalina.

 

[gilby101]

I have a 2TB external HDD that I want to format APFS to use for data and backups for 2 MacBook Pros. What's the best way to format/partition/volume it? Here is what the sizes of each partition currently are but I want to erase and start clean.

I would erase the disk (creating a single partition) with an APFS volume. Then add more APFS volumes as needed. You will end up with Disk Utility showing one partition with one container and multiple APFS volumes. Make sure you have enabled Disk Utility > View > Show All Devices (in the menus).

Time Machine, when to a directly attached disk, uses an APFS volume which is used just for Time Machine. Put data into its own APFS volume.

When you say backup for two Macs, do you mean the disk is physically connected to each Mac as required? If so, each requires its own Time Machine APFS volume.

You then end up with two APFS volumes for Time Machine and one for data. Very like your current partition setup, but without the need to allocate fixed space to each.

Of course, this requires wiping the disk, so think about how you will keep the data and whether you need to keep any old backups. Easiest to buy a new disk.

On quote/reserve, I would not bother. But I do like to keep an eye on disk space used.

 

[canhaz]

I wondered why my backups were faster using Big Sur.

Makes no difference now anyway as i have gone back to Catalina.

Why did you go back out of interest?

 

[estesbubba]

You then end up with two APFS volumes for Time Machine and one for data. Very like your current partition setup, but without the need to allocate fixed space to each.

Since I've been WFH both my home and work MacBooks alternate being on my dock so each gets backed up when on it. I essentially did what you explained, but gave each backup partition a quota so they don't grow over a certain size and delete old backups. All my important data is also on OneDrive so wiping the HDD wasn't that big of deal.

 

[estesbubba]

On quote/reserve, I would not bother. But I do like to keep an eye on disk space used.

I was just thinking about this and won't the backup volumes keep growing until they take up all the remaining space if there isn't a quota? If this is the case and I suddenly need 200GB for Data is there anything I can do?

 

[yangm]

@estesbubba I would still recommend giving the full drive to TM and storing other backups somewhere else. TM disks spin far more often than an usual backup, since it has to prune older snapshots to keep going and other maintenance tasks, so they're more likely to fail sooner.

 

[gilby101]

I was just thinking about this and won't the backup volumes keep growing until they take up all the remaining space if there isn't a quota? If this is the case and I suddenly need 200GB for Data is there anything I can do?

Yes, it will grow.
You can manually delete old backups in Terminal with the command:
sudo tmutil delete <path>
But make sure you understand this and confirm how it works with APFS TM backups (I have only used it with HFS+ TM backups). Understandably it is not a quick command. You may well prefer to use quotas.

BackupLoupe app can be used to give a view of what space is being used for each backup.

 

[tyron]

I can't remove old backups anymore in the same way I did with Catalina and prior versions. Do you know if this is a bug or "by design" (aka immutable backups)? (someone started a thread about it)

Sometimes you may have a large file (dmg) that will end up using disk space and it's not really needed. I liked the ability to remove these when I realized they ended up on my TM...

 

[gilby101]

You can manually delete old backups in Terminal with the command:
sudo tmutil delete <path>
But make sure you understand this and confirm how it works with APFS TM backups (I have only used it with HFS+ TM backups).

I don't like my earlier comment!

Does anybody understand how to use tmutil delete with the new APFS TM backups? I don't either understand it or have a recipe to delete stuff from APFS backups.

Edit: You can't.
See the very last few comments in https://eclecticlight.co/2020/06/29...ur-how-time-machine-backs-up-to-apfs-and-more

 

[reptarwilleatu]

anyone have any ideas on why i’m getting this error when trying to backup with TM to an APFS formatted drive? i’ve tried this one which worked in Catalina (i got the same error, but plugging in the flash drive directly instead of thru my OWC Thunderbolt dock fixed it), and several spinning drives where i added a new volume to the container. i’ve tried just about every combination in terms of APFS. Encrypted, non encrypted, case sensitive, non case sensitive. it always gets to about 8-10 GB before it throws up the error. i looked thru that article, but i’m probably missing something or just not understanding

 

[canhaz]

anyone have any ideas on why i’m getting this error when trying to backup with TM to an APFS formatted drive? i’ve tried this one which worked in Catalina (i got the same error, but plugging in the flash drive directly instead of thru my OWC Thunderbolt dock fixed it), and several spinning drives where i added a new volume to the container. i’ve tried just about every combination in terms of APFS. Encrypted, non encrypted, case sensitive, non case sensitive. it always gets to about 8-10 GB before it throws up the error. i looked thru that article, but i’m probably missing something or just not understanding View attachment 1680975

Not sure. You might want to take a look in Console though. Often it has additional error information.

 

[gilby101]

You might want to take a look in Console though

Essential I would say in this case. If Console frightens/confuses you (like it does everyone else), Time Machine Mechanic can pick out TM stuff from the logs. https://eclecticlight.co/consolation-t2m2-and-log-utilities/

 

[reptarwilleatu]

Not sure. You might want to take a look in Console though. Often it has additional error information.

Essential I would say in this case. If Console frightens/confuses you (like it does everyone else), Time Machine Mechanic can pick out TM stuff from the logs. https://eclecticlight.co/consolation-t2m2-and-log-utilities/

this is what i am seeing

here is Time Machine Mechanic

it doesn’t report any errors in TM Mechanic when i click ”Check Time Machine” which is interesting

 

[gilby101]

it doesn’t report any errors in TM Mechanic when i click ”Check Time Machine” which is interesting

Interesting - that is putting it politely.

You might get some help extracting data from the logs using Consolation (same link as before) but it is that little bit harder to use. At least you know to look at around 09:47:21.

In your screenshot of the Console app, you are looking at system.log. That will probably not help. For the last few years, system.log has become almost useless. Unified logging has replaced it.

You can watch the log in real time by streaming the log from Casey's Mac Mini.

 

[reptarwilleatu]

Interesting - that is putting it politely.

You might get some help extracting data from the logs using Consolation (same link as before) but it is that little bit harder to use. At least you know to look at around 09:47:21.

In your screenshot of the Console app, you are looking at system.log. That will probably not help. For the last few years, system.log has become almost useless. Unified logging has replaced it.

You can watch the log in real time by streaming the log from Casey's Mac Mini.

well Apple Support is of no help. originally they told me it was a hardware issue with the flash drive i am using which is nonsense. then someone else just told me there was no way running diagnostics or looking at logs to try and figure out what’s causing it 🙄

i haven’t been able to piece anything out of logs myself. to be honest, i’m not sure what i am looking for. but the drive did fill up to about 60 gb after a week or so with backup automatically checked so clearly it’s getting hung up on something near the beginning. i just don’t know what. i wish i could try HFS+ but it converts the disk to APFS any time that i try that.

Edit: i actually got it to try HFS+ with my Airport Extreme AC and exact same thing happened. filed a radar so we’ll see i guess

 

[luckystar101]

Hello. I want to setup a partition of my external SSD that is APFS encrypted with time machine. Should I also flag "encrypt backup" option in time machine or APFS encrypted is enough?

Thanks

 

[reptarwilleatu]

Hello. I want to setup a partition of my external SSD that is APFS encrypted with time machine. Should I also flag "encrypt backup" option in time machine or APFS encrypted is enough?

Thanks

Usually you would want to check the box when selecting the disk in Time Machine for encrypting it.

 

[KALLT]

Hello. I want to setup a partition of my external SSD that is APFS encrypted with time machine. Should I also flag "encrypt backup" option in time machine or APFS encrypted is enough?

Yes, you have to check that option. If you turn it off there, TM will first decrypt your APFS volume. If it prompts you for a new encryption password, then the APFS volume is not in the correct format and has to be reformatted by TM.