Questions about APFS

[TwoLaneHighway]

I am going to upgrade the OS on my MacBook Pro to probably Big Sur, and I have questions about the file system.

Since my Macs are all older, I believe they all use the Mac OS Extended (HFS+) option.

Questions:
1.) What exactly is the difference between HFS+ and APFS?

2.) With Big Sur or newer, do I have to use APFS?

3.) Is there any reason why I would not want to use APFS?

4.) If I format my MBP with APFS, then will I still be able to access data on old drives that use HFS+?

5.) Will old Macs using HFS+ be able to access data on my newly formatted MBP running APFS?

 

[rpmurray]

I can answer some of these:

1) HFS+ is Apple's old, documented, mostly stable, file system and APFS is their new, partially documented, still being worked on, file system.

2) With Big Sewer and newer you do need to use APFS for the boot drive. Apple gives you no choice in that, but you can use HFS+ for non-bootable drives.

3) Most casual users will say that APFS is the best thing since sliced cheese. I really prefer there to be a nice supply of third-party utilities available to fix things when they go wrong but Apple seems hell-bent on keeping that from happening, so be aware that if things go wrong and Disk First Aid can't fix it then you'll be SOL. So make sure you backup, backup and backup. APFS really works best on SSDs, and less so on spinning disks, since it's optimized for SSDs.

4) Yes, macOS of the newer persuasion can read (and write) to HFS+ disks.

5) Macs running macOS earlier than High Sierra won't be able to use drives formatted in APFS.

 

[TwoLaneHighway]

I can answer some of these:

1) HFS+ is Apple's old, documented, mostly stable, file system and APFS is their new, partially documented, still being worked on, file system.

I guess I am late to the game and APFS has been around for years?

2) With Big Sewer and newer you do need to use APFS for the boot drive. Apple gives you no choice in that, but you can use HFS+ for non-bootable drives.

Big Sewer? *LOL* You should join my other thread asking what people think about Big Sur! ;-)

3) Most casual users will say that APFS is the best thing since sliced cheese. I really prefer there to be a nice supply of third-party utilities available to fix things when they go wrong but Apple seems hell-bent on keeping that from happening, so be aware that if things go wrong and Disk First Aid can't fix it then you'll be SOL.

What 3rd party utilities would you have used with HFS+?

If a disk is not encrypted, is there a way to recover data easily with HFS+? How about APFS?

So make sure you backup, backup and backup.

I use Carbon Copy Cloner to backup my Macs.

APFS really works best on SSDs, and less so on spinning disks, since it's optimized for SSDs.

To save money, I clone my Macs to traditional HDDs so will that be a problem using an OS like Big Sur with APFS and Carbon Copy Cloner?

4) Yes, macOS of the newer persuasion can read (and write) to HFS+ disks.

So if I need to migrate or recover data from old disks that have backups of my old Macs (e.g. Mountain Lion, Sierra, etc) then I iwll be able to see the old files and migrate them over to my newer Big Sur/APFS Mac?

What about migrating data off of Windows disks? (Really this question applies to both HFS+ and APFS Macs.)

I have a ton of data from my Windows days on numerous hard-drives that I would like to get onto my Macs. Many years ago, I believe you could read data from a PC on a mac, but I am hoping that has changed. Most of my data of interest consists of photos, music, videos, etc. so I hope that would come over just fine. However some things like old Word docs and spreadsheets might be limited by the applications on my mac and hopefully not the HFS+/APFS issue.

Can you talk about how that would play out with all of this?

5) Macs running macOS earlier than High Sierra won't be able to use drives formatted in APFS.

I guess that isn't an issue, beause you are unlikely to need to take data from a newer Mac and need to read/use it on an older Mac, right?

 

[Fishrrman]

If you have to convert to APFS for big sur, well, that's what's required.

However, I'd leave older external drives back at HFS+.

HFS+ is mature, and more importantly, disk repair utilities still work with it.
Not so much with APFS.

Platter-based hard drives in particular should be left at HFS+. APFS has a tendency to overly-fragment these drives, and performance is impacted.

 

[TwoLaneHighway]

If you have to convert to APFS for big sur, well, that's what's required.

However, I'd leave older external drives back at HFS+.

HFS+ is mature, and more importantly, disk repair utilities still work with it.
Not so much with APFS.

Platter-based hard drives in particular should be left at HFS+. APFS has a tendency to overly-fragment these drives, and performance is impacted.

In anoother thread, I am trying to make sense of how Big Sur and the newer Apple chipset changes how backups work. But it sounds like I will continue to use Carbon Copy Cloner for my clones/backups.

That being said, will it be a problem to clone my new internal APFS drive onto an external HDD (platter-based)?

And how would I format the external HDDs?

I prefer to use traditional HDDs for backups because they are cheaper, and my perception is that they are more stable from failure than SSD technology, plus to your point, it is easier to recover data off of a traditional HDD.

 

[Fishrrman]

OP wrote:
"In anoother thread, I am trying to make sense of how Big Sur and the newer Apple chipset changes how backups work. But it sounds like I will continue to use Carbon Copy Cloner for my clones/backups."

I'm a big CCC fan, and ALWAYS recommend it over time machine.
If you use it, you already know why it's great software.

"That being said, will it be a problem to clone my new internal APFS drive onto an external HDD (platter-based)?"

I don't know whether CCC will do a backup of Big Sur to an external (platter-based) HDD that is formatted HFS+. I know this WILL work with Mojave, but that's "as far as I've taken things".
I'm going to GUESS that it will.
Why don't you just try it?
If it doesn't work, reformat the drive to APFS and use it that way instead.

"And how would I format the external HDDs?"

With disk utility, same as you would format internal drives...

 

[TwoLaneHighway]

OP wrote:
"In another thread, I am trying to make sense of how Big Sur and the newer Apple chipset changes how backups work. But it sounds like I will continue to use Carbon Copy Cloner for my clones/backups."

I'm a big CCC fan, and ALWAYS recommend it over time machine.
If you use it, you already know why it's great software.

I agree. Carbon Copy Cone is one of the best pieces of software that I have ever owned!

(Not sure that I will feel thatw ay moving forward with how Apple has all but taken away the ability to clone your entire hard-drive.)

"That being said, will it be a problem to clone my new internal APFS drive onto an external HDD (platter-based)?"

I don't know whether CCC will do a backup of Big Sur to an external (platter-based) HDD that is formatted HFS+. I know this WILL work with Mojave, but that's "as far as I've taken things".

Okay.

I'm going to GUESS that it will.
Why don't you just try it?
If it doesn't work, reformat the drive to APFS and use it that way instead.

I thought I had heard that once you format a drive with APFS you cannot re-fromat it as HFS+?

I also asked, because I have dozens of HDD's formatted with the traditional HFS+, and I wanted to know if I can just plug one of those external drives into my rMBP, and use CCC to create an updated clone, or if I will have to format the drive to APFS and be stuck with that format?

For instance, I am used to clong MBP #1 and then plugging it into MBP #2 and booting things up. However, if I upgrade MBP #2 to Big Sur and APFS and then have to run CCC on an APFS-formatetd external HDD, then maybe that will prevent me from booting up that external drive on MBP #1? Or maybe my thinking is off?

Lots of changes to my old way of doing things.

 

[Fuchal]

I thought I had heard that once you format a drive with APFS you cannot re-fromat it as HFS+?

You can reformat it as HFS+, you just can't "convert" it back to HFS+ without a format.

 

[TwoLaneHighway]

You can reformat it as HFS+, you just can't "convert" it back to HFS+ without a format.

So if you had data on a drive that was APFS, you couldn't convert it to HFS+ so you could read it on an older macOS, right?

If you had data on a drive formatted as HFS+, could you convert it to APFS without losing the data? I'm guessing the answer is no.

Are there any scenarios where I might get stuck converting from pre-Big Sur/HFS+ to Big Sur/APFS?

Right now, I have two rMBP's and all of my internal and external SSD's and HDD's are formatted as HFS+. And nearly all of my external HDD's are CCC clones of my Macs.

 

[hobowankenobi]

As of right now, cloning is hard. It may get easier in the future, but we don't know yet. As of today, the safe thing to expect for all new Macs running anything on APFS is that we will NOT have a bootable external clone.

Quite a bit of coverage has been written about this, but Bombich covers it fairly succinctly here. Another nice overview here with good info on the new structure (volumes, partitions, etc.).

As for what is different about APFS vs. HFS+...it is a moving target, especially when you factor in the OS changes too....and then there are boot firmware changes for all new M1 (and beyond) Macs.

Good overview here.

And Time Machine is different now, with real improvements. I think Apple should have called it Time Machine 2 or something, as it has fundamentally changed. And we all should consider it again, even if we were not fans a decade ago.

And yeah...we all feel very comfortable with our file system utilities for the last 3 decades. But if APFS does it's job, most—if not all—of those tools will be unnecessary. iOS has been using APFS for 5+ years. How often does anybody defrag or otherwise maintain the file system on an iPhone or iPad?

 

[TwoLaneHighway]

As of right now, cloning is hard. It may get easier in the future, but we don't know yet. As of today, the safe thing to expect for all new Macs running anything on APFS is that we will NOT have a bootable external clone.

Quite a bit of coverage has been written about this, but Bombich covers it fairly succinctly here. Another nice overview here with good info on the new structure (volumes, partitions, etc.).

As for what is different about APFS vs. HFS+...it is a moving target, especially when you factor in the OS changes too....and then there are boot firmware changes for all new M1 (and beyond) Macs.

Good overview here.

And Time Machine is different now, with real improvements. I think Apple should have called it Time Machine 2 or something, as it has fundamentally changed. And we all should consider it again, even if we were not fans a decade ago.

And yeah...we all feel very comfortable with our file system utilities for the last 3 decades. But if APFS does it's job, most—if not all—of those tools will be unnecessary. iOS has been using APFS for 5+ years. How often does anybody defrag or otherwise maintain the file system on an iPhone or iPad?

Thank you for the links and comments.

I feel so frustrated right now...

My rMBP is just about full, and I want to install a 2TB SSD in my backup rMBP. I also want to update my OS, because more and more things are not compatible/failing with my current OS.

Rebuilding a machine a HUNORMOUS task and one I try to put off for at least 5 years at a time.

And data backups are crucial for me and my business. (CCC has been a dream come true until finding things out yesterday in my Big Sur thread.) :-(

I have to head out on a video shoot after Labor Day, and I am torn what to do with my Macs.

I tend to like to spend weeks or months researching things and kicking options around, and I really don't have that much time to move forward.

Maybe I can try and binge-read the links you mention and try to get a handle on what is happening with my formerly bullet-proof backup workflow.

While I am a big advocate of security, I also want 110% control of my data and my system, and it sounds like I am losing that...

 

[rpmurray]

But if APFS does it's job

That "if" needs to be in caps, bolded and otherwise emphasized. I have read some stories of users whose APFS formatted drives have become corrupted and where Disk First Aid did not repair it. Hence why I'd prefer a few big, beefy, third-party utilities that can fix that. Apple's fix has always been to restore from Time Machine or other backup, but if you don't have a recent one ...

 

[hobowankenobi]

That "if" needs to be in caps, bolded and otherwise emphasized. I have read some stories of users whose APFS formatted drives have become corrupted and where Disk First Aid did not repair it. Hence why I'd prefer a few big, beefy, third-party utilities that can fix that. Apple's fix has always been to restore from Time Machine or other backup, but if you don't have a recent one ...

We have more than 3400 Macs at my work...and I don't see any trend of APFS causing issues.

File system differences aside...SSDs are much less likely to have data recovered from them if they have a problem. Fewer problems...but the failures that do happen tend to be more catastrophic.

It is always our own fault if we don't have backups. Backups have always been essential. No sign of that going away.

 

[hobowankenobi]

Thank you for the links and comments.

I feel so frustrated right now...

My rMBP is just about full, and I want to install a 2TB SSD in my backup rMBP. I also want to update my OS, because more and more things are not compatible/failing with my current OS.

Rebuilding a machine a HUNORMOUS task and one I try to put off for at least 5 years at a time.

And data backups are crucial for me and my business. (CCC has been a dream come true until finding things out yesterday in my Big Sur thread.) :-(

I have to head out on a video shoot after Labor Day, and I am torn what to do with my Macs.

I tend to like to spend weeks or months researching things and kicking options around, and I really don't have that much time to move forward.

Maybe I can try and binge-read the links you mention and try to get a handle on what is happening with my formerly bullet-proof backup workflow.

While I am a big advocate of security, I also want 110% control of my data and my system, and it sounds like I am losing that...

It just takes a bit of changing of mindset. Bootable clones are amazing and convenient. But what really matters is our unique data, not the OS or apps or things that can be reinstalled easily. Unique user data is everything.

CCC (and many other tools, including Time Machine) do a great job of protecting our user data, even without bootable clones.

I have an M1 mini running as a local server that needs continuous backup, so I use:

• Time Machine to an attached volume for continuous backups
• Drive sync tool for instantaneous file syncing to NAS
• Chronosync to an attached volume for scheduled backups and long-term version control
• Chronosync to archive the full OS to a disk image on a remote server for full disaster recovery

All very doable on the latest OS and hardware. Would I like a bootable clone too? Sure. Can I live without it? Yep.


----


I would suggest:

• Pick one Mac that you can live without for while and make it a testbed
• Clone it to external (so you can painlessly roll back if needed)
• Make a TM backup too, to a second volume/partition
• Update to MacOS 11. I would wipe, format, and start fresh
• Use the migration tool to restore from your TM backup

Test, test, test.

An important test: Boot to internet recovery, install a fresh OS, and restore data from TM, CCC, or any other backup tool you use. It will take a bit longer than the attached clone, but good to know exactly what to expect, and how long it takes before you actually need it. Good to know that you can do it this way...even when you do a bootable clone.

If you are not satisfied with the TM restore...wipe and start again.

Consider not restoring, or restoring only specific files you need.

This process/progress has to be messy. We have a huge change: CPU, GPU, firmware, OS, security/access, file system. All changing.

We can wait, but not forever. Maybe when MacOS 12.something is here this winter, we may see some easier paths forward. Bugs squashed, features added...maybe even the return of easily bootable external clones? Eventually, we all move forward or we change paths. Can only tread water on HFS+ for so long...

P.S.: I understand SuperDuper has a beta version that may have Big Sur cloning working, but I have not tested it yet. If they did it...that means others will crack it too. And that is without Apple handing over the keys to ASR. More on the struggles and work arounds back here.

 

[hobowankenobi]

For externals (HD or SSD) I am sticking with HFS+ as the most portable/flexible.

The only exception would be for TM backups, which now support APFS. But for everything else, to the best of my knowledge, HFS+ is preferred for the foreseeable future. Setting aside bootable clones, the rest should feel unchanged.

As we are talking about an actual business and fairly large data sets, have you considered a NAS for file backups? For data (not system backups) file systems don't matter. On a Synology for example, one can use TM or their native Drive backup/sync tool, and never worry about portability for file system compatibility.

It would not answer the question of how to quickly recover from a failed device the way a bootable clone does, but OTOH, you can have backups running all the time, as long as you are on the network, without plugging in. You can also backup over the internet, as well as share files fairly easily with web links. Fairly compelling for user data protection and sharing.

 

[Fishrrman]

"Not sure that I will feel that way moving forward with how Apple has all but taken away the ability to clone your entire hard-drive."

hobo added:
"As of today, the safe thing to expect for all new Macs running anything on APFS is that we will NOT have a bootable external clone."

NO!
Not true!
As others have documented on this board, it IS possible to create a bootable clone of the latest OS's on the latest Macs using CCC. An owner of an m1 Mac posted how he did this a couple of days ago.

You just have to choose the "legacy clone" option over CCC's defaults (I think that's how it works, I don't have a new or m-series Mac with which to try it).

And when you want to boot from the clone, you can't use the "startup disk" pref pane any more. I believe you have to invoke the startup manager at boot and "do it that way". With the m1 Macs, you have to press and HOLD the power-on key to do this, I understand...

 

[jz0309]

For externals (HD or SSD) I am sticking with HFS+ as the most portable/flexible.

The only exception would be for TM backups, which now support APFS. But for everything else, to the best of my knowledge, HFS+ is preferred for the foreseeable future. Setting aside bootable clones, the rest should feel unchanged.

As we are talking about an actual business and fairly large data sets, have you considered a NAS for file backups? For data (not system backups) file systems don't matter. On a Synology for example, one can use TM or their native Drive backup/sync tool, and never worry about portability for file system compatibility.

It would not answer the question of how to quickly recover from a failed device the way a bootable clone does, but OTOH, you can have backups running all the time, as long as you are on the network, without plugging in. You can also backup over the internet, as well as share files fairly easily with web links. Fairly compelling for user data protection and sharing.

In Big Sur, TM WILL reformat to APFS case-sensitive as part of your first backup. Only exception that I am aware of is that if you use a drive that has been used previously on older OSs for TM.
I’ve had this happen to me a few times

 

[TwoLaneHighway]

I had hoped to start a new, more aptly entitled, thread about CCC and clones, but this thread is morphing quickly.

:-(

Definitely some knowledgeable people here with some great information, I just need you all to let me catch up and lead questions.

I still think a new thread would be better...

 

[KaliYoni]

I think the OP's situation is unavoidable due to Apple's development and design philosophy. Forcing rapid and unrelenting changes to hardware and software specs on users is integral to Apple's DNA. Unfortunately, anybody who wants long periods of stability and backwards compatibility probably has to go with Windows, or if one is technically inclined, an open-source OS. Using macOS, iOS, and iPadOS means having to deal with disruptive changes every couple of years, like it or not.

My advice, then, is that it might be better to move away from doing a deep dive into the specifics of APFS and Big Sur. Thinking about these areas instead might lead to a more sustainable long term tech strategy for the OP:

• Would a change away from Apple products lead to better productivity and efficiency in the medium to long term?
• Is it feasible to never connect any machine running Mojave or earlier to the Internet in order to maintain full CCC usability?
• Are there any ways to record or preserve settings in applications that take a lot of time to configure?
• Is it possible to do testing of alternative setups on a single machine?

Finally, I think for tech products in general there is a significant tradeoff between stability and upgrading. Avoiding or delaying upgrades saves time and effort in the short run but greatly increases the time and effort required when upgrades or changes are unavoidable. Which path to follow, obviously, is an individual decision.

 

[Fishrrman]

Kaliyoni wrote:
"Is it feasible to never connect any machine running Mojave or earlier to the Internet in order to maintain full CCC usability?"

This makes no sense at all.

I have a 2018 Mac Mini that came with Mojave.
It's STILL running Mojave and I have no intentions of upgrading this particular Mac.
It's connected to the internet and does just fine.
I can use CCC with it, no problems -- and do just that.

 

[TwoLaneHighway]

As of right now, cloning is hard. It may get easier in the future, but we don't know yet. As of today, the safe thing to expect for all new Macs running anything on APFS is that we will NOT have a bootable external clone.

Quite a bit of coverage has been written about this, but Bombich covers it fairly succinctly here. Another nice overview here with good info on the new structure (volumes, partitions, etc.).

As for what is different about APFS vs. HFS+...it is a moving target, especially when you factor in the OS changes too....and then there are boot firmware changes for all new M1 (and beyond) Macs.

Good overview here.

And Time Machine is different now, with real improvements. I think Apple should have called it Time Machine 2 or something, as it has fundamentally changed. And we all should consider it again, even if we were not fans a decade ago.

And yeah...we all feel very comfortable with our file system utilities for the last 3 decades. But if APFS does it's job, most—if not all—of those tools will be unnecessary. iOS has been using APFS for 5+ years. How often does anybody defrag or otherwise maintain the file system on an iPhone or iPad?

It just takes a bit of changing of mindset. Bootable clones are amazing and convenient. But what really matters is our unique data, not the OS or apps or things that can be reinstalled easily. Unique user data is everything.

CCC (and many other tools, including Time Machine) do a great job of protecting our user data, even without bootable clones.

I have an M1 mini running as a local server that needs continuous backup, so I use:

• Time Machine to an attached volume for continuous backups
• Drive sync tool for instantaneous file syncing to NAS
• Chronosync to an attached volume for scheduled backups and long-term version control
• Chronosync to archive the full OS to a disk image on a remote server for full disaster recovery

All very doable on the latest OS and hardware. Would I like a bootable clone too? Sure. Can I live without it? Yep.


----


I would suggest:

• Pick one Mac that you can live without for while and make it a testbed
• Clone it to external (so you can painlessly roll back if needed)
• Make a TM backup too, to a second volume/partition
• Update to MacOS 11. I would wipe, format, and start fresh
• Use the migration tool to restore from your TM backup

Test, test, test.

An important test: Boot to internet recovery, install a fresh OS, and restore data from TM, CCC, or any other backup tool you use. It will take a bit longer than the attached clone, but good to know exactly what to expect, and how long it takes before you actually need it. Good to know that you can do it this way...even when you do a bootable clone.

If you are not satisfied with the TM restore...wipe and start again.

Consider not restoring, or restoring only specific files you need.

This process/progress has to be messy. We have a huge change: CPU, GPU, firmware, OS, security/access, file system. All changing.

We can wait, but not forever. Maybe when MacOS 12.something is here this winter, we may see some easier paths forward. Bugs squashed, features added...maybe even the return of easily bootable external clones? Eventually, we all move forward or we change paths. Can only tread water on HFS+ for so long...

P.S.: I understand SuperDuper has a beta version that may have Big Sur cloning working, but I have not tested it yet. If they did it...that means others will crack it too. And that is without Apple handing over the keys to ASR. More on the struggles and work arounds back here.

I think the OP's situation is unavoidable due to Apple's development and design philosophy. Forcing rapid and unrelenting changes to hardware and software specs on users is integral to Apple's DNA. Unfortunately, anybody who wants long periods of stability and backwards compatibility probably has to go with Windows, or if one is technically inclined, an open-source OS. Using macOS, iOS, and iPadOS means having to deal with disruptive changes every couple of years, like it or not.

It does seem to be moving that way...

My advice, then, is that it might be better to move away from doing a deep dive into the specifics of APFS and Big Sur. Thinking about these areas instead might lead to a more sustainable long term tech strategy for the OP:

• Would a change away from Apple products lead to better productivity and efficiency in the medium to long term?

No, and I will NEVER go back to Windows. Never!!!

• Is it feasible to never connect any machine running Mojave or earlier to the Internet in order to maintain full CCC usability?

In theory, yes. Since this is on a laptop, not practically.

• Are there any ways to record or preserve settings in applications that take a lot of time to configure?

It sounds like CCC 6 will still allow me to do this, but if Apple keeps moving the bar, who knows?!

• Is it possible to do testing of alternative setups on a single machine?

Yes, that is what I am doing.

Finally, I think for tech products in general there is a significant tradeoff between stability and upgrading. Avoiding or delaying upgrades saves time and effort in the short run but greatly increases the time and effort required when upgrades or changes are unavoidable. Which path to follow, obviously, is an individual decision.

I am a "late adopter" and a "hanger-on'er", but eventually they move the line os much I have to run to catch up. That is sorta where I am in the last few months.

There is always a solution - it's just up to me (and help from MacRumors experts) to figure out what that solution is!!

 

[KaliYoni]

Kaliyoni wrote:
"Is it feasible to never connect any machine running Mojave or earlier to the Internet in order to maintain full CCC usability?"

This makes no sense at all.

I have a 2018 Mac Mini that came with Mojave.
It's STILL running Mojave and I have no intentions of upgrading this particular Mac.
It's connected to the internet and does just fine.
I can use CCC with it, no problems -- and do just that.

The context for my answer is this:

• OP is considering staying on Mojave after Monterey launches.
• Apple probably will end all updates for Mojave after Monterey launches.
• OP does not want to use unofficial OS alterations or become an OS expert user.
• OP needs to maintain at least one machine that can be used for A/V production and supports booting from a clone.

 

[Fishrrman]

My advice:
Just keep using Mojave.
Use... "what works for you".
And don't worry about the rest of it...

I will run Mojave for at least 3, 4, maybe 5 years more on this 2018 Mini...

 

[TwoLaneHighway]

My advice:
Just keep using Mojave.
Use... "what works for you".
And don't worry about the rest of it...

I will run Mojave for at least 3, 4, maybe 5 years more on this 2018 Mini...

But don't security updates end in September 2021?

 

[MarkC426]

Yes, around then.
But that changes nothing.
Safe browsing is key to keeping secure.

On a side note, you don't mention your mac version.
Considering your still on HFS+ file system indicates it may be old.
Does it support Big Sur?