Warning: Arq Backup removes 'thinning'

[gilby101]

The newest version of Arq Backup (7.24.1) removes backup thinning. The release notes say:

• Removed the "thinning" function from backup plans. Too many people found backup records were permanently deleted that they weren't expecting to be deleted, and that's a really bad user experience, so we felt it was best to remove the function. As an alternative to limit the amount of storage used by your backup plan, edit the plan, click the Retention tab, check 'Limit storage used' and enter a value.

and the documentation now refers to a storage limit as the one way to control backups kept.

I don't think thinning is a feature that is mysterious to macOS users. We are all familiar with thinning of Time Machine backups.

This is a major change introduced without notice in a minor update. I am so glad I have disabled auto-update.

So don't update to 7.24.1 if you value thinning. I have contacted Arq Support and started looking at alternatives.

 

[chabig]

Good catch. I don’t think I would have noticed had you not mentioned it. That said, I don’t care. I have Time Machine already. I just want Arq to be representative of my current files in case of an “oh ****” moment occurs. For me, it‘s like fourth or fifth tier backup, after Time Machine and local CCC backup.

 

[gilby101]

The newest version of Arq Backup (7.24.1) removes backup thinning. The release notes say:

• Removed the "thinning" function from backup plans. Too many people found backup records were permanently deleted that they weren't expecting to be deleted, and that's a really bad user experience, so we felt it was best to remove the function. As an alternative to limit the amount of storage used by your backup plan, edit the plan, click the Retention tab, check 'Limit storage used' and enter a value.

and the documentation now refers to a storage limit as the one way to control backups kept.

I don't think thinning is a feature that is mysterious to macOS users. We are all familiar with thinning of Time Machine backups.

This is a major change introduced without notice in a minor update. I am so glad I have disabled auto-update.

So don't update to 7.24.1 if you value thinning. I have contacted Arq Support and started looking at alternatives.

Arq 7.24.2 reinstates thinning.

Presumably because so many users (including me) complained!

 

[chabig]

I got thinning back, but my backups are now generating hundreds of errors.

 

[Weaselboy]

I got thinning back, but my backups are now generating hundreds of errors.

Are you on Sonoma? I got a bunch of errors that looked related to the new widgets. I did a reboot and another Arq backup and have not had the errors since.

 

[chabig]

They all say "Cloud file contents not present on disk".

 

[gilby101]

They all say "Cloud file contents not present on disk".

Can you confirm that you do have: Dataset Options > "When a dataless ("cloud-only") file is encountered" set as "materialize"

 

[chabig]

Thanks!! I was not aware of that option. It was set to report error. I'll change it, materialize is what I want anyway.

 

[chabig]

That made a huge difference. I went from more than 100 errors to just one:

Error: /Users/*****/Library/Application Support/com.apple.LaunchServicesTemplateApp.dv: Failed to read file status: Operation not permitted

I might have to accept this until the new version of Onyx is released for Sonoma. Then I'll try rebuilding the Launch Services database.

Edit: I ended up excluding this folder and now my backups complete normally.

 

[Big Bad D]

Can you confirm that you do have: Dataset Options > "When a dataless ("cloud-only") file is encountered" set as "materialize"

Thanks! Had same errors, but had not identifed this setting.

 

[gilby101]

I'll change it, materialize is what I want anyway.

Had same errors, but had not identifed this setting.

It is sad that Stefan does not use his blog page (or better still a newsletter) to highlight new features like this.

The only way to catch these changes is to disable automatic updates, check once a week or so and, if an update, read the release notes first. The 'materialise' was added in version 7.21 last April with this in the release notes:

• Added a backup plan option so you can specify whether to report, ignore, or materialize dataless ("cloud-only") files such as Dropbox and Google Drive files.

I had an email discussion with Stefan back in February about the errors with Dropbox, etc. and he was quick to provide a test/beta version with the materialise change which then got into a release version in April.

 

[Weaselboy]

It is sad that Stefan does not use his blog page (or better still a newsletter) to highlight new features like this.

https://www.arqbackup.com/download/arqbackup/arq7_release_notes.html

I have the release notes page bookmarked and check there occasionally.

 

[jennyp]

Can you confirm that you do have: Dataset Options > "When a dataless ("cloud-only") file is encountered" set as "materialize"

Where is the setting "Dataset Options" ?

 

[mattjgalloway]

Has anyone else noticed that while the option is back, it doesn't actually work? I see my backup plans saying they are thinning, but then no records actually get thinned.

 

[palter]

That made a huge difference. I went from more than 100 errors to just one:

Error: /Users/*****/Library/Application Support/com.apple.LaunchServicesTemplateApp.dv: Failed to read file status: Operation not permitted

I might have to accept this until the new version of Onyx is released for Sonoma. Then I'll try rebuilding the Launch Services database.

Edit: I ended up excluding this folder and now my backups complete normally.

That folder is created when you use the "Add to Dock" feature in Safari to create an app from a web page. It's protected by rootless/SIP which means nothing can access it except the System itself. (You can't even access it using "sudo".)

Excluding it is the right thing to do. I've reported it to the Arq authors, suggesting they add it to the default set of exclusions.

 

[gilby101]

Where is the setting "Dataset Options" ?

Select a backup set. Then Edit. Then Options tab. 7th item with a drop down and choice of three.
"When a dataless ("cloud-only") file is encountered".

 

[sjinsjca]

Can you confirm that you do have: Dataset Options > "When a dataless ("cloud-only") file is encountered" set as "materialize"

What does "materialize" mean? I am not finding it described anywhere.

Context: I have a large amount of data stored in several cloud services that I don't want cluttering my disk.

 

[gilby101]

What does "materialize" mean? I am not finding it described anywhere.

Materialise means to download the file from the cloud so that Arq can back it up. Otherwise, if the file is only in the cloud, Arq will find an error or ignore the file.

 

[jennyp]

Select a backup set. Then Edit. Then Options tab. 7th item with a drop down and choice of three.
"When a dataless ("cloud-only") file is encountered".

Oh that. Yes thanks. I notice if I select "materialize" Arq warns me of possible "excessive data transfer and disk usage", but really I don't think I have many "cloud-only" files. I never had any errors with the "report an error" setting anyway.

 

[jennyp]

Has anyone else noticed that while the option is back, it doesn't actually work? I see my backup plans saying they are thinning, but then no records actually get thinned.

My "retention" is set to "keep daily backups for 30 days" but I have backups much older than that. Is that what you mean?

 

[sjinsjca]

Materialise means to download the file from the cloud so that Arq can back it up. Otherwise, if the file is only in the cloud, Arq will find an error or ignore the file.

Thank you. Does the downloaded file remain on your disk? ...Might seem a dumb question, but if that happens to all my cloud-stored files, I'll run out of disk quickly.

 

[Brian33]

Has anyone else noticed that while the option is back, it doesn't actually work? I see my backup plans saying they are thinning, but then no records actually get thinned.

My "retention" is set to "keep daily backups for 30 days" but I have backups much older than that. Is that what you mean?

Now I notice this too. Is "thinning" working?

I have retention set to "keep daily backups for 8 days" but I have 17 "backup records" (each on a different day) shown in the Arq app. Goes back to Aug 22. (For "hourly", "weekly", and "monthly" retention I have zeros.)

Shouldn't the oldest one(s) be deleted?

 

[diskrete]

Wow. I had all of these issues yesterday -- cloud file contents not present, and error backing up com.apple.LaunchServicesTemplateApp.dv.

Thanks for pointing out the "materialize" option, I will give it a try. Fortunately I have enough disk space to hold those files. I assume, once materialized, they remain?

 

[gilby101]

Now I notice this too. Is "thinning" working?

No it is not! Just as you describe. See also https://www.reddit.com/r/Arqbackup/comments/16y3rxf
There is a newer test version for which thinning does work. version 7.24.2.3

 

[gilby101]

Does the downloaded file remain on your disk? ...Might seem a dumb question, but if that happens to all my cloud-stored files, I'll run out of disk quickly.

Yes it does. So, in your case you might not want to enable "materialise".

The latest ChronoSync beta has the option to download cloud only files, and to evict them after copying. It is all becoming very complex as developers battle with backup of synchronised cloud storage.