Arq

[patent10021]

Thanks. Just signed up with that promo.

Question, Arq has some wildcard exclusion list which seems very odd. If I want everything backed up why would I allow exclusions (by default)?



I'm a dev so I want to include paths, hidden files and directories like bash files, python environments etc so that I don't have to reconfig everything again if I lose data.

I just want to click and recover. Should I remove all those exclusions?

 

[gilby101]

I just want to click and recover. Should I remove all those exclusions?

Short answer: No don't remove those exclusions unless you need any of those folders for your recovery process.

Long answer: I don't really know what you mean by 'click and recover', but here goes:

Arq is not a bare metal recovery. Recovery requires a working macOS before installing Arq and connecting to the storage provider. So there is little point in doing a backup of 'system' folders which will get recreated by the macOS install.

Look at what risks you are mitigating by using Arq and backup what is required for that.

I use a local backup (Time Machine) to cover hardware failures.

I am using Arq (to OneDrive and B2) mostly to mitigate the risks of major disasters like fire and theft (or something else that destroys all my local computer infrastructure). In that case I will be buying a new Mac (or two) and will start with a working macOS. So I need to able to recover:
1) My data (mostly documents and photo) and Arq is great for that. I also have a few difficult to get application installers I backup with Arq.
2) Passwords and software licenses and for that I use 1Password.
3) Mail - I use IMAP services so can recover from the servers (but I do keep a local copy which goes to Arq just in case).
4) .....

If I were using Arq as a replacement for Time Machine backing up to a DAS or NAS, I would definitely change the exclusion list so that I was backing up nearly everything except for the macOS system and caches. At a minimum I would then be looking at backup of at least parts of /usr and /etc and I would be going through /Library and ~/Library very carefully to decide what I needed to backup. But TM is a much better tool for local disaster recovery.

So think about why you are using Arq and what disasters you want to mitigate by using Arq.

 

[Weaselboy]

I just want to click and recover. Should I remove all those exclusions?

I looked through that list when I setup Arq and it all looked like cache and temp/swap files you would not beed to backup.

 

[Brian33]

(I'm not an Arq user at this point, but hope you don't mind me jumping in!)

Question, Arq has some wildcard exclusion list which seems very odd.

You might want to compare Arq's exclusion list to Carbon Copy Cloner's. The CCC developer has lots of experience of course and has some comments about various exclusions: (EDIT: I had originally linked an outdated page; Here's the current one...)

https://bombich.com/kb/ccc5/some-files-and-folders-are-automatically-excluded-from-backup-task

If I were using Arq as a replacement for Time Machine backing up to a DAS or NAS, I would definitely change the exclusion list so that I was backing up nearly everything except for the macOS system and caches. At a minimum I would then be looking at backup of at least parts of /usr and /etc and I would be going through /Library and ~/Library very carefully to decide what I needed to backup.

I'm quite surprised that /usr and /etc are on Arq's exclusion list -- thanks for pointing that out. My systems have various Homebrew installations in /usr/local that I would expect to be backed up if I were to use Arq for a full system local backup.

 

[bwintx]

For any of you who've gone from "standalone" Arq to Arq Premium, do you then have to uninstall your old copy of Arq (with its license code) and install a new one to use the new license code, or can you simply convert the old Arq to the new license code? I see that Arq has a "Deactivate License" setting, and wonder if that's how to do this. Any help appreciated. (I also emailed Arq Tech Support, but the reply window is 24 hours, so thought I'd see if you folks already know.)

 

[Apple_Robert]

For any of you who've gone from "standalone" Arq to Arq Premium, do you then have to uninstall your old copy of Arq (with its license code) and install a new one to use the new license code, or can you simply convert the old Arq to the new license code? I see that Arq has a "Deactivate License" setting, and wonder if that's how to do this. Any help appreciated. (I also emailed Arq Tech Support, but the reply window is 24 hours, so thought I'd see if you folks already know.)

I previously purchased the stand-alone version. And when I purchased the premium as noted in the thread, I deactivated my standalone license and then inputted the premium license.

 

[bwintx]

I previously purchased the stand-alone version. And when I purchased the premium as noted in the thread, I deactivated my standalone license and then inputted the premium license.

Thanks. That worked just fine.

 

[patent10021]

I refunded Ark. It was taking days to scan and upload. I tried multiple times. I just don't have that time/patience. I even restarted Arq, backing up the home folder minus the Applications folder to save tons of time. Same with TM. Takes way too long for me.

For now I'm only using CCC. Since I'm already paying for iCloud, for redundancy, I decided I will copy my home folder into iCloud Drive once a week or whatever since CCC is always current.

iCloud Drive also has Desktop & Documents Folders syncing which is good enough for my dev project files.

I wonder if I could symbolic link my home folder to the Documents folder and have my home folder sync to iCloud Drive as well?

 

[BigMcGuire]

Arq takes awhile the first time - but after that first index - it goes pretty fast. CCC is blazing fast.

 

[sjinsjca]

Wasabi would be the apparent logical second choice on a price/performance-comparison basis, but I certainly am open to hearing about some of the other choices available via Arq (Storage Locations (arqbackup.com).

I've been using Arq with Wasabi for a couple of years. Flawless. Can't say enough about both of 'em. Wasabi's pricing is very competitive, so I'd see no reason to look elsewhere.

 

[Apple_Robert]

I've been using Arq with Wasabi for a couple of years. Flawless. Can't say enough about both of 'em. Wasabi's pricing is very competitive, so I'd see no reason to look elsewhere.

$71.88 a year isn't bad. It would be nice if they ran specials. From what I have seen in the last several days of (first time) use, Wasabi and Arq premium are both equally fast. I think the real difference at this point, are the many features of Wasabi console.

edited to add: I think Arq should include the downloads folder by default. I also found I had to add a few exceptions to keep Arq from reading an attached storage on my Mac mini while using M1 MBA.

I would like to set up the SMB network share backup option but, have never done that before.

I have 10TB drive connected to my Mac mini I would like to use. It is on the same network as my M1 MBA which is my primary Mac.

Not sure how to set the UNC path.

 

[Weaselboy]

For those who subscribe to this particular thread who haven’t yet seen the flack about what Backblaze got caught doing...
Ben Cox on Twitter: "WTF? @backblaze 's B2 web UI seems to submit all of the names and sizes of my files in my B2 bucket to facebook. I noticed because I saw "waiting for facebook .com" at the bottom while trying to download a backup... ?!?!?!? I even opted out of their tracking widget thing! https://t.co/IkqkGNTkSi" / Twitter

Backblaze says it has pushed a fix, but this HN thread gives you an idea of the understandably unhappy reaction from Backblaze customers:
Backblaze submitting names and sizes of files in B2 buckets to Facebook | Hacker News (ycombinator.com)

While one commenter there says correctly that using B2 from Arq should be safe (especially if you’re using encryption), some of you may want to consider what this issue says about Backblaze. I’ve been a happy B2-via-Arq user since August, 2017, but am now re-evaluating the B2 part. Wasabi would be the apparent logical second choice on a price/performance-comparison basis, but I certainly am open to hearing about some of the other choices available via Arq (Storage Locations (arqbackup.com).

FYI.

Privacy Update: Third-party Tracking

Read this post for a privacy update regarding third-party tracking code on our website.

www.backblaze.com

Here is BackBlaze's response to the problem. Assuming they are telling the truth, it looks like it was unintentional on their part.

 

[Apple_Robert]

Can someone with Arq, 2 Macs, and an external drive attached to the remote Mac, try setting up a new backup plan to a network drive and see if you can get the test connection to go through? I have an email sent to Stefan to see if he can assist but, also wanted to see if anyone else is having trouble getting a connection. No matter what I try, I can't get Arq to connect to the external drive on the Mac mini. I am running Arq on the MacBook Air.

 

[gilby101]

Can someone with Arq, 2 Macs, and an external drive attached to the remote Mac, try setting up a new backup plan to a network drive and see if you can get the test connection to go through?

It does work, though there is no detail in the documentation. I have a MacBook with a test backup to a share on my iMac. This is a screenshot from the MacBook.

Beth is (Shakespeare haha) the name of my iMac. MyBook6A is HDD on my iMac with a folder called Arq. So the Storage Location is //beth.local/mybook6a/arq. Case doesn't seem to matter.

Where do you have problems?

 

[Apple_Robert]

It does work, though there is no detail in the documentation. I have a MacBook with a test backup to a share on my iMac. This is a screenshot from the MacBook.

View attachment 1754291

Beth is (Shakespeare haha) the name of my iMac. MyBook6A is HDD on my iMac with a folder called Arq. So the Storage Location is //beth.local/mybook6a/arq. Case doesn't seem to matter.

Where do you have problems?

Thanks for the reply. I have been getting a mount error.

I have Arq installed on my MacBook Air. I am trying to get Arq to connect to the external drive that is attached to my Mac mini.

The Mac mini name is
Obi-Wan-Kenobi-Mac-mini

The attached external drive is
/volume/WD easystore10TB

If I put
//obi-wan-kenobi-Mac-mini.local/volume/WD easystore10TB, I get a mount error

If I put
//obi-wan-kenobi-macbook-air.local/wd easystore10tb, I get the mount error

I have no problem connecting to said external drive via Finder.

I heard back from Stefan yesterday afternoon. He asked me to mount the WD easystore drive in Finder and then send him the Terminal > mount results, which I did. I haven't heard back from him since.

 

[gilby101]

The attached external drive is
/volume/WD easystore10TB
If I put
//obi-wan-kenobi-Mac-mini.local/volume/WD easystore10TB, I get a mount error

The 'volume' is doubly wrong.
The local path on the mini would be /Volumes/WD easystore10TB (s in Volumes).
But you should not have any volume(s) in the SMB path. It should be:
//obi-wan-kenobi-Mac-mini.local/WD easystore10TB
Here I am assuming that Arq is happy with spaces in volume names.

 

[Apple_Robert]

The 'volume' is doubly wrong.
The local path on the mini would be /Volumes/WD easystore10TB (s in Volumes).
But you should not have any volume(s) in the SMB path. It should be:
//obi-wan-kenobi-Mac-mini.local/WD easystore10TB
Here I am assuming that Arq is happy with spaces in volume names.

That worked. Thank you so much for the help.

 

[Apple_Robert]

Under Arq Retention (default ticked) > limit storage, I have Arq set to 1000GB. I just looked at my Wasabi bucket and see that my bucket is at 1.2 TB for the billing cycle. Anyone else encounter Arq going over the set limit?

edited to add: I have the same settings for Arq Premium and my bucket there is showing 1.8 TB.

 

[gilby101]

Under Arq Retention (default ticked) > limit storage, I have Arq set to 1000GB. I just looked at my Wasabi bucket and see that my bucket is at 1.2 TB for the billing cycle. Anyone else encounter Arq going over the set limit?

There are two things that may be happening here.

1. Arq's retention: If over the limit, Arq only deletes old versions of files. It keeps at least one version of everything.

2. Wasabi pricing: Every object stored in Wasabi is charged for a minimum of 90 days. If an object is created by Arq, but deleted some time later, you will be charged for 90 days.

 

[Apple_Robert]

There are two things that may be happening here.

1. Arq's retention: If over the limit, Arq only deletes old versions of files. It keeps at least one version of everything.

2. Wasabi pricing: Every object stored in Wasabi is charged for a minimum of 90 days. If an object is created by Arq, but deleted some time later, you will be charged for 90 days.

Thanks for the reply.

I forgot about Wasabi's 90 day file charge. What you posted makes sense. I will keep an eye on it and see how the following month bill goes. I just want to be mindful of not getting into paying $20 a month for Wasabi or extra Arq Premium storage.

 

[Apple_Robert]

Arq has been updated, for those who haven't gotten a notice.

Release Notes for Arq Backup Version 7.5.1​

April 19, 2021

Improvements​

• Added immutable backups feature for storage locations which support it (S3, B2 with S3-compatibility API, and soon Wasabi). If you enable immutable backups, Arq will lock your backup record data to protect against ransomware attack or accidental deletion.
• Modified the new-backup-plan flow to encourage choosing an encryption password.
• Fixed an issue which could cause not-found errors during validation.
• Added documentation for the backup data format to Arq's Help.

I like the immutable feature. Will be nice when Wasabi is officially supported by Arq.

 

[Apple_Robert]

There has been another Arq update. Glad to see you won't have to manually start immutable backups in Wasabi anymore.

This looks like another good update. Kudos to Stefan.

Release Notes for Arq Backup Version 7.5.3​

April 22, 2021

Changes​

• Added *.wasabisys.com to list of hosts that support S3 object versioning (Wasabi support starts in May 2021).
• Added support for adopting a backup set that's been copied from a non-object-lock bucket to an object-lock-enabled bucket.
• When adding an S3 storage location, if the IAM user doesn't have ListAllMyBuckets permission, prompt for the bucket name instead of reporting an 'access denied' error.

 

[Big Bad D]

Great thread, with some very helpful info. Having previously stopped using Crashplan when they cancelled their personal plans, I had more recently tried Backblaze but was not comfortable with the risk of loss of external drive data if unconnected for 30 days without paying for additional fee and also only for single computer.

I found Arq from this thread and other recommendations and have tried their Arq Premium as it seemed easy option compared to paying for Arq and also a server subscription. Shame I missed the recent discount, but still seems good value.

Only negative is that the info on Arq from their website seems not always so compete. For example no info on Arq Premium server provider (is it Wasabi?) or any comparison that could affect server location choice. Does anyone, particular those in Europe, have any experience to share for server location choice? are they all from same server provider?

 

[Weaselboy]

Does anyone, particular those in Europe, have any experience to share for server location choice? are they all from same server provider?

I can't help you with this, but the dev. is very responsive. I bet if you shoot him an email he would answer this right away for you.

 

[bwintx]

...Only negative is that the info on Arq from their website seems not always so compete. For example no info on Arq Premium server provider (is it Wasabi?)...

Last I saw, Arq Premium uses Google Cloud.