Copying Time Machine Disk - one more permission problem

[toke lahti]

Question might be, that has copying TM to new disk been broken all the time, or has broken just with macos11.

macos12:

If the Time Machine backup disk for your Mac is full

If Time Machine runs out of space for your Mac backups, add another backup disk or change your backup preferences.

support.apple.com

"If you do run out of space, it’s best to connect a new backup disk.”

macos11:

https://support.apple.com/guide/mac...ine-backup-disk-is-full-mh15137/11.0/mac/11.0

"If you do run out of space, it’s best to connect a new backup disk.”

macos10.15:

If the Time Machine backup disk for your Mac is full

If Time Machine runs out of space for your Mac backups, add another backup disk or change your backup preferences.

support.apple.com

"If you connect a larger disk and wish to preserve your existing backup history, you can copy the Backups.backupdb folder from the old disk to the new one. Use Finder, and not Terminal or any other app, to copy the Backups.backupdb folder.”

macos10.14:

If the Time Machine backup disk for your Mac is full

If Time Machine runs out of space for your Mac backups, add another backup disk or change your backup preferences.

support.apple.com

"If you connect a larger disk and wish to preserve your existing backup history, you can copy the Backups.backupdb folder from the old disk to the new one. Use Finder, and not Terminal or any other app, to copy the Backups.backupdb folder.”

macos10.13:

If the Time Machine backup disk for your Mac is full

If Time Machine runs out of space for your Mac backups, add another backup disk or change your backup preferences.

support.apple.com

"If you connect a larger disk and wish to preserve your existing backup history, you can copy the Backups.backupdb folder from the old disk to the new one. Use Finder, and not Terminal or any other app, to copy the Backups.backupdb folder.”

 

[toke lahti]

If your backup is still in HFS+ format try a program call DiskWarrior. It is able to repair disk drives that may have the drive damaged. I have used it for over a decade. It does not work at present with APFS formatted drives. Maybe it is worth a try for you. It isn’t free and not cheap but it might help you.

I find it a bit hard to buy DW now, that it doesn't supoort the main filesystem in macs.
Might be that it will be EOLled.
Last version from September 2018. And $120 for that...
"Support for Big Sur coming soon..."

 

[toke lahti]

HFS has always been somewhat fragile.

According to this:

Time Machine: 1 How it works, or fails to

New series looking at how TM works, what its problems are, what tools to use to diagnose those, and how to fix it.

eclecticlight.co

there were big changes in TM with Catalina. And smaller changes before that.

I believe that TM was working just fine before they broke it.
Not counting that Sierra accident.

 

[toke lahti]

Update on this.
I had this on my screen for almost exactly 6 days (144h).

Then the connection had maybe a glitch.

Ouch!
This comment wasn't fully written, when I changed to Mojave install to test, if copying the TM disk would work over there.
Apple's support said, that since macOS11 they haven't supported copying TM disk, so they will not investigate this.

They said that for this kind of happenings you should have anothed TM disk.
I asked what to do when it also corrupts and there was no answer.

In Mojave the whole system is pretty jammed after 20 hours.

In Monterey it had counted all of 25M items by that time.
Maybe the TM copy process is broken, or it halts to the point of corruption.
One strange thing was, that even I had ticked off "Back Up Automatically" in TM preferences, it had thinned some 300GB from my old backups and halted somewhere in about 120GB region of 600GB backup.

Maybe TM does this always?
That it already had timed "one more" backup, which it will make even when automatics are off?
Or again, it's broken.

 

[toke lahti]

Sadly my Mojave install has became badly unstable. Maybe after last fw update.
Maybe because of SATSMART.
It won't stay up for a week, which is needed here.
Wondering now if a new Mojave install would behave better.
I'll try "empty"=fresh account first...

 

[toke lahti]

Note to the logbook, stardate 2459750.
Second attemp with rarely used assist admin account led to 11 copied timestamps of 49 on the source.
I was 50 hours away and when I hit the shift button (ta wake up the monitor) when I got back, there was steady "5 seconds left
of (ca.) 7.4TB copied".

I still need to search how to flag partially copied folder as having a visible content.
Inconvenient to go the folder with cmd+shift+G.

 

[toke lahti]

Checked from info windows that the fully copied 10 backups were a bit large (both megabytes and number of items) in the destination than in the source. 13.5TB vs. 15.2TB. 22.0M vs. 28.7M items.
Looks like copying TM wasn't ready for the prime time in Mojave.
Could it be better in Catalina?

Problem here obviously is, that you can't copy single backups from disk to disk.
You have to succeed with it at once.
Copying TM archive works only when you drag the Backups.backupdb from source to destination.

And partial copy leaves the destination Backups.backupdb folder hidden, so even to check what's inside, cmd+shift+G.
Sigh.
I'm moving to a new home.
Gotta pack things up and maybe continue this next month...

 

[toke lahti]

Given that loss and considering how important backups are led me to getting a third party solution that is rock solid reliable.

Care to share what this 3rd party solution is?

 

[planteater]

Care to share what this 3rd party solution is?

I was referring to Carbon Copy Cloner (CCC), an excellent product for full system backups. IMO, it's the best solution available. When I first started using it a few years ago, I did a lot of comparative research, and it was both then and now very highly recommended. I've had no issues with it in the least, unlike Time Machine where I've had corruption more than once.

I use CCC for backups to a mirrored disk set that I attach weekly. I then put that mirror in a fire vault. Monthly I rotate one of the disks offsite, and return on from offsite to the local mirror.

I also use Arq Backup to make daily encrypted online backups to Backblaze B2. Arq is setup to keep all versions of files.

 

[HDFan]

f you are going to use one drive for all backups, make sure you also do a secondary backup to a separate drive. If either the first or second backup drive fails, you won't loose everything that you have backed up.

I love TM functionality but have a hard time trusting it as a primary backup solution, secondary sure.

Absolutely. Should be only one of your backups in a 3-2-1 backup strategy.

Monterey secretely converted your TM disk from hfs+ to apfs?
Quite amazing.

Would be very surprising as it normally requires a disk reformat.

I was referring to Carbon Copy Cloner (CCC),

Due to its complexity when TM fails trying to fix things is difficult if not impossible. My fix is to wipe the disk and restart the TM backup from scratch. As above don't rely on TM as your only backup source. Use something like CCC and a backup service (Backblaze, Crashplan for Business) to make sure you don't lose data.

 

[gilby101]

When I first started using it a few years ago, I did a lot of comparative research, and it was both then and now very highly recommended. I've had no issues with it in the least, unlike Time Machine where I've had corruption more than once.

Maybe time to redo your research.
1. TM is considerably more robust since the change to APFS format for the backup disk.
2. With current versions, there is little difference between what CCC and TM are doing. Different interface, but not much else now that the concept of bootable clones in mostly dead and that both use APFS snapshots on source and destination volumes.
As I see it, TM and CCC are both highly recommended.

 

[planteater]

Maybe time to redo your research.
1. TM is considerably more robust since the change to APFS format for the backup disk.
2. With current versions, there is little difference between what CCC and TM are doing. Different interface, but not much else now that the concept of bootable clones in mostly dead and that both use APFS snapshots on source and destination volumes.
As I see it, TM and CCC are both highly recommended.

Are you saying that TM no longer uses sparsebundles? They were the source of my issues. I don't recall if I was using HFS+ or not at the time. It's been awhile, I could have been on High Sierra or Mojave but don't recall.

But, I'm happy with my current strategy and no need to change. If TM has improved reliability, then it would be an awesome backup method. I always liked the usage paradigm of it.

 

[gilby101]

Are you saying that TM no longer uses sparsebundles?

TM to a directly attached storage (DAS) has never used sparsebundles. That has only been the case (and continues to be) for network backups.

I don't recall if I was using HFS+ or not at the time. It's been awhile, I could have been on High Sierra or Mojave but don't recall.

It would have been HFS+. It was with Big Sur that Apple introduced TM to APFS. And it was then that TM became more reliable for DAS.

The destination disks for CCC and TM are very similar now. APFS disk with APFS snapshots for each backup.

Keep using CCC as you are happy with it and, I assume, confident of the restore process.

 

[HDFan]

there is little difference between what CCC and TM are doing.

TM uses pointers and CCC the file system. That makes a huge difference when you are trying to do things such as a restore. TM is glacially slow.

 

[gilby101]

TM uses pointers and CCC the file system

I was referring to current versions - both use APFS snapshots. All files systems use pointers in one form or another.

TM is glacially slow

Not for me. Have you done a recovery recently from a TM to APFS backup?

I am glad you like CCC, but I really think that the differences are now mostly cosmetic and ease of use.

 

[HDFan]

Not for me. Have you done a recovery recently from a TM to APFS backup?

Did a couple of TM restore to a new Studio SSD. Didn't remember exact time it but was well over 8 hours. You can see TM slowness (on a HD) by trying to open an older file. Takes 30 seconds to a minute. Compare that with a CCC backup where you can go to a file in seconds.

 

[gilby101]

Didn't remember exact time it but was well over 8 hours. You can see TM slowness (on a HD) by trying to open an older file. Takes 30 seconds to a minute

Your experience is very different to mine. The good thing is that we both have a backup and restore system which works well. Let's leave at that.

 

[HDFan]

Your experience is very different to mine.

How big is your TM backup? How much data are you backing up? In my case I'm backing up ~3.5 TB of data to 6 TB TM HD drives some of which are full. Will have to erase some of my TM backups to start over from scratch.

Just trying to figure out why some people have no problems with TM and others find it unusable. Looking to see what is different in your configuration.

 

[toke lahti]

I was referring to current versions - both use APFS snapshots. All files systems use pointers in one form or another.


Not for me. Have you done a recovery recently from a TM to APFS backup?

I am glad you like CCC, but I really think that the differences are now mostly cosmetic and ease of use.

Are you really saying that CCC can do incremental backups now?
I need to buy an upgrade right now...

 

[gilby101]

Are you really saying that CCC can do incremental backups now?

Absolutely. With APFS snapshots.

 

[gilby101]

How big is your TM backup? How much data are you backing up? In my case I'm backing up ~3.5 TB of data to 6 TB TM HD drives some of which are full. Will have to erase some of my TM backups to start over from scratch.

Just trying to figure out why some people have no problems with TM and others find it unusable. Looking to see what is different in your configuration.

My TM backup is about 1.2 TB.

Having had the discussion in this thread, you may be interested (or not) that I am doing another evaluation of CCC for my purposes. I last did similar 18 months ago and decided against CCC. Must admit that I am seeing some advantages in making some serious use of CCC! Not yet to replace TM entirely - but maybe!!

 

[kibepo73]

When your external drive or any storage drive shows up as “Read Only” on your mac, then you might be encountering a very common problem with the hard drive. In such a situation, the user can open and view the files on a drive but cannot copy or move files to the storage drive. The possible reasons for this issue could be:

1. External drive is NTFS formatted
2. Disk permission changed to “Read Only”
3. External drive is corrupted

However, the issue can be fixed using the below DIY methods:

1. Reformat the external drive
2. Change/Ignore permissions on an external drive
3. Repair external drive with disk utility’s first aid

Hope this will help you fix the issue!

 

[toke lahti]

My TM backup is about 1.2 TB.

Having had the discussion in this thread, you may be interested (or not) that I am doing another evaluation of CCC for my purposes. I last did similar 18 months ago and decided against CCC. Must admit that I am seeing some advantages in making some serious use of CCC! Not yet to replace TM entirely - but maybe!!

I'm looking for your experience on this.
I'm disappointed in TimeMachine.

 

[gilby101]

I'm looking for your experience on this.
I'm disappointed in TimeMachine.

I have started using CCC for some of my backups. All my disk (SSD/HDD) are formatted APFS and I am looking for backups to directly attached disks (mostly HDD). My thoughts:
1) CCC and Time Machine produce a similar results when making a single simple straight forward backup with APFS snapshots.
2) CCC has more flexibility than TM. In particular:
3) I have always found it annoying that TM has only one retention policy (1 hr for 24hrs, daily for a month, etc.). I don't need backups of my system disk beyond the last minor macOS update, but I would like other disks (with photos, documents, etc.) to have a much longer retention period. CCC enables this by allowing different backup tasks for each APFS volume. To a degree this is ameliorated by my use of ArqBackup (to OneDrive and B2) for which I currently have nearly 2 years history.
4) I make regular use of BackupLoupe to index my TM backups. CCC does not have a similar feature.
5) Without making any measurements, CCC does seem faster than TM. And there is the option to not create a spotlight index on the destination.
6) I suspect that CCC is much better optimised for performance than TM. TM avoids system load by going very slowly whilst there is other activity on the Mac. CCC just gets on with it.
7) And the CCC Support is responsive and helpful! You don't get any TM developer responding to questions or suggestions.

Until Ventura is in production, I will continue with TM for my system disk. But will probably drop TM after that. I would like to avoid too many products - TM, CCC, Chronosync and Arq seems a bit excessive.

I have not done a full restore from CCC - only test file/folder restores using drag and drop in Finder.

I am weighing up whether I replace some of my Chronosync tasks (for file synchronisation, not backup). Chronosync has more powerful selection rules but is a little slower for local disk transfers. For Mac to Mac LAN synchronisation, Chronosync is much faster than CCC because it has its own protocol optimised for synchronisation tasks. CCC relies on SMB.

 

[toke lahti]

I have started using CCC for some of my backups. All my disk (SSD/HDD) are formatted APFS and I am looking for backups to directly attached disks (mostly HDD). My thoughts:
1) CCC and Time Machine produce a similar results when making a single simple straight forward backup with APFS snapshots.
2) CCC has more flexibility than TM. In particular:
3) I have always found it annoying that TM has only one retention policy (1 hr for 24hrs, daily for a month, etc.). I don't need backups of my system disk beyond the last minor macOS update, but I would like other disks (with photos, documents, etc.) to have a much longer retention period. CCC enables this by allowing different backup tasks for each APFS volume. To a degree this is ameliorated by my use of ArqBackup (to OneDrive and B2) for which I currently have nearly 2 years history.
4) I make regular use of BackupLoupe to index my TM backups. CCC does not have a similar feature.
5) Without making any measurements, CCC does seem faster than TM. And there is the option to not create a spotlight index on the destination.
6) I suspect that CCC is much better optimised for performance than TM. TM avoids system load by going very slowly whilst there is other activity on the Mac. CCC just gets on with it.
7) And the CCC Support is responsive and helpful! You don't get any TM developer responding to questions or suggestions.

Until Ventura is in production, I will continue with TM for my system disk. But will probably drop TM after that. I would like to avoid too many products - TM, CCC, Chronosync and Arq seems a bit excessive.

I have not done a full restore from CCC - only test file/folder restores using drag and drop in Finder.

I am weighing up whether I replace some of my Chronosync tasks (for file synchronisation, not backup). Chronosync has more powerful selection rules but is a little slower for local disk transfers. For Mac to Mac LAN synchronisation, Chronosync is much faster than CCC because it has its own protocol optimised for synchronisation tasks. CCC relies on SMB.

Can BackupLoupe be used with backups made with CCC?