Arq Backup Scheduling Not Working, Help Needed

[JoelBC]

I am running Arq 7.26.3 and macOS 14.1.1

I am trying to get Arq to run every 12 hours, at 12:00 AM and 12:00 AM.

I have the Arq scheduling window configured as per the below image noting that the next backup is scheduled to start at 7:00 PM even thought it is now 11:40AM (i.e., I would have thought that this setup would result in the next backup being run at 12:00 PM and then every 12 hours thereafter.

If I wait until 12:00 PM to run the backup manually, teh next scheduled backup still runs every 12 hours at 7:00 AM and 7:00 PM, not the 12:00 AM and 12:00 PM that I am aiming for!

In addition, I have no idea where the 7:00 AM / 7:00 PM times are come from!

THE ASK: I would appreciate any and all help in getting this fixed!

Thank you!

 

[Apple_Robert]

Set Arc to only run at 12pm and 11:59pm not every 12hrs. Your settings show every 24hrs. Set the tick to daily at the 2 times you want and that should work.

 

[JoelBC]

Set Arc to only run at 12pm and 11:59pm not every 12hrs. Your settings show every 24hrs. Set the tick to daily at the 2 times you want and that should work.

@Apple_Robert , I appreciate the response but would appreciate a little more help:

1. How do you enter two times into the "Every day at" field as "12:00, 23:59" or "12:00 23:59" does not work. I believe that only 1 time can be entered into the field because every time I try to enter two times I get the following message:

2. What do you mean by "Your settings show every 24 hours"? Where are / do my settings show this?

Thanks.


PS. I suppose I could set two separate backups (i.e., one at 12:00 PM and on eat 12:00 AM) BUT I would prefer not to do so.

 

[Bigwaff]

I think the "Hourly .. Every" option starts when you save but the "Every" window is between when each backup starts.. literally. So, given your screenshot, if your Mac is sleeping or shutdown at 7PM, the next backup won't start until your Mac wakes or boots. If your Mac wakes or boots at 9PM, then the backup runs and the next backup is scheduled for 12 hrs later at 9AM.

I use Arq but use the "Daily" schedule. Sorry, not going to change my schedule to find out how "Hourly .. Every" actually works ;-)

 

[JoelBC]

I think the "Hourly .. Every" option starts when you save but the "Every" window is between when each backup starts.. literally. So, given your screenshot, if your Mac is sleeping or shutdown at 7PM, the next backup won't start until your Mac wakes or boots. If your Mac wakes or boots at 9PM, then the backup runs and the next backup is scheduled for 12 hrs later at 9AM.

I use Arq but use the "Daily" schedule. Sorry, not going to change my schedule to find out how "Hourly .. Every" actually works ;-)

Appreciated.

If I understand you correctly, you are saying that I should set the settings to "Every 12 hours at 0 minutes after the hour" and save it at 12:00 AM. The next backup will then start 12:00 PM! Is this the correct understanding of what ou wrote.

If yes then I will test it out and let you know.

Thanks.

 

[Bigwaff]

If I understand you correctly, you are saying that I should set the settings to "Every 12 hours at 0 minutes after the hour" and save it at 12:00 AM. The next backup will then start 12:00 PM! Is this the correct understanding of what ou wrote.

Yup.. that is my hypothesis. I think you can validate this w/ a shorter window of 2 or 3 hrs w/ some Mac sleep or power down somewhere along the way.

 

[chabig]

You could also duplicate the backup plan. Schedule one to run daily at 12:00 and the other to run daily at 00:00.

 

[JoelBC]

Yup.. that is my hypothesis. I think you can validate this w/ a shorter window of 2 or 3 hrs w/ some Mac sleep or power down somewhere along the way.

I am going to try that and will let you know! Stay tuned!!

 

[JoelBC]

You could also duplicate the backup plan. Schedule one to run daily at 12:00 and the other to run daily at 00:00.

Correct, and I noted this in one of my earlier posts! I prefer having one backup as it is easier to manage the file selection but this will certainly be my backup plan!

 

[chabig]

Correct, and I noted this in one of my earlier posts! I prefer having one backup as it is easier to manage the file selection but this will certainly be my backup plan!

Agreed, and sorry I missed that earlier. I like Arq. There have been times I’ve wished it had the ability to duplicate a backup plan. I alternate daily backups to two cloud services, and I have to keep them manually in sync.

 

[JoelBC]

Agreed, and sorry I missed that earlier. I like Arq. There have been times I’ve wished it had the ability to duplicate a backup plan. I alternate daily backups to two cloud services, and I have to keep them manually in sync.

All good, no worries.

I will try @Bigwaff's approach and see whether that works! If it does, then I will stick with it for the very reason that you note, the inability to copy backup plans and change one or two settings (i.e., the backup time) is missing.

If it does not work as proposed, then I use the sledge hammer approach and have two identical backup plans with the first staring at 12:00 AM and the second starting a 12:00 PM.

Let's see what happens!

PS. I too like Arq! It is the best overall solution that I was able to find! Just wish it was a little more flexible and teh documentation a little more robust!

 

[gilby101]

Set Arc to only run at 12pm and 11:59pm not every 12hrs. Your settings show every 24hrs. Set the tick to daily at the 2 times you want and that should work.

Sadly, there is no option to do that.

If it does not work as proposed, then I use the sledge hammer approach and have two identical backup plans with the first staring at 12:00 AM and the second starting a 12:00 PM.

Doesn't sound too good to me:
Two plans means two lots of backup storage. No de-duplication between backup plans.

I think your choices are:
1) Once a day at a time you choose.
2) Every 12 hours (or whatever you want), but not a directly controlled times.
3) Just after 12 am or 12 pm, set it to run every 12 hours at 0 minutes past the hour. That should give you starts at 12 pm and 12 am. But this may get messed up if a) a backup takes longer than 12 hours or b) your computer is sleeping or turned off at backup time.

I run all mine daily during the night.

 

[chabig]

Two plans means two lots of backup storage. No de-duplication between backup plans.

I thought that as long as they are identical and use the same destination, they would back up to the same set of backup files. But you might be right. I’ve found Arq support to be very responsive. They would give us definitive answers.

 

[JoelBC]

@gilby101 , much thanks for taking the time to respond.

Sadly, there is no option to do that.

Thank you for clarifying that. That is what I thought.

Doesn't sound too good to me:
Two plans means two lots of backup storage. No de-duplication between backup plans.

Quick follow up. Why would this be the case were I to have two backup plans that back up the exact same data set to the exact same location. Would it not be teh same as running teh same backup plan at two different times?

I think your choices are:
1) Once a day at a time you choose.
2) Every 12 hours (or whatever you want), but not a directly controlled times.
3) Just after 12 am or 12 pm, set it to run every 12 hours at 0 minutes past the hour. That should give you starts at 12 pm and 12 am. But this may get messed up if a) a backup takes longer than 12 hours or b) your computer is sleeping or turned off at backup time.

I understand how to configure 1) and 2).

I do not understand how to configure 3 as to me this would be configured exactly the same as 2) and have the same results (i.e., set it to run every 12 hours at 0 minutes after the hour and start the first backup at 12:00 AM or 12:00 PM). Please explain, what am I missing?

I run all mine daily during the night.

Probably where I will land.


PS. What is needed is the ability to specific two or more exact start times (at least to do what I want to do)!

 

[JoelBC]

I thought that as long as they are identical and use the same destination, they would back up to the same set of backup files. But you might be right. I’ve found Arq support to be very responsive. They would give us definitive answers.

I agree per my above question / response to @gilby101 !

 

[JoelBC]

An update:

I have Arq setup to run "Every 4 hour starting 0 minutes after the hour". I started the first backup at 8:00 PM and got the following results:

Backup 1: Start time = 8:01 PM, End time = 11:09 PM

Backup 2: Start time = 3:00 AM, End time = 5:38 AM
Observation --> the 4 hours is from the end time of Backup 1

Backup 3: Start time = 7:00 AM, End time = 8:00 AM
Observation --> the 4 hours is from the start time of Backup 2

Backup 4: Start time = 11:00 AM, End time = TBD
Observation --> the 4 hours is from the start time of Backup 3 and back/locked onto the 7:00 AM previously noted.

I will keep monitoring and update later in the day.

QUESTIONS:

1. How / why does it make sense the the first 4 hour period is the measured from the end of the first back set with subsequent 4 hour period measured from the start of the second and subsequent backup sets? With not knowing how long a backup would take, the would give the use no control as to when backups would occur.

2. I am begging to think this is a bug. Comments / thoughts?

Thanks and look forward to your comments / thoughts?


PS> @chabig , I have found Arq support to be mixed.

 

[gilby101]

Quick follow up. Why would this be the case were I to have two backup plans that back up the exact same data set to the exact same location. Would it not be teh same as running teh same backup plan at two different times?

Each backup plan (even if backing up the same files to the same destination) has separate storage. Evidence: If you look on your storage location there is a folder called Arq Backup Data and inside that there is a folder for each backup set named with the UUID of the backup set.

V4 (and earlier?) had de-duplication across a whole destination. That is the main thing we lost with V5 onwards. I had an email conversation with Stefan at the time: he made major changes to the backup structure to enable faster backup and recovery - de-dup change was a consequence. He was also attempting to simplify configuration.

1. How / why does it make sense the the first 4 hour period is the measured from the end of the first back set with subsequent 4 hour period measured from the start of the second and subsequent backup sets? With not knowing how long a backup would take, the would give the use no control as to when backups would occur.

2. I am begging to think this is a bug. Comments / thoughts?

I did a bit of testing. As you discovered getting the initial and subsequent backups at right time is tricky. But you are trying to manipulate the scheduling in an unintended way.

Design decision (not a bug). I suggest feedback to Stefan with a simple request for scheduling on multiple specific hours each day.

I am happy with backup once per day because I use TM and CCC to do local backups every hour and Arq (stored on GoogleDrive and OneDrive) is for recovery from "serious" disasters and for that daily is sufficient for me.

 

[JoelBC]

Each backup plan (even if backing up the same files to the same destination) has separate storage. Evidence: If you look on your storage location there is a folder called Arq Backup Data and inside that there is a folder for each backup set named with the UUID of the backup set.

V4 (and earlier?) had de-duplication across a whole destination. That is the main thing we lost with V5 onwards. I had an email conversation with Stefan at the time: he made major changes to the backup structure to enable faster backup and recovery - de-dup change was a consequence. He was also attempting to simplify configuration.

I appreciate you sharing the above. I did look at the folder called Arq Backup Data and -- as you note -- there are two distinct backup sets!

I did a bit of testing. As you discovered getting the initial and subsequent backups at right time is tricky. But you are trying to manipulate the scheduling in an unintended way.

I appreciate your confirmation of what my testing discovered!

I am not sure I am trying to manipulate the scheduling in an unintended way though admittedly in a way not designed to be done!

Design decision (not a bug). I suggest feedback to Stefan with a simple request for scheduling on multiple specific hours each day.

We are on the same page. Will be asking Stefan to:

1. Scheduling multiple specific times; and/or

2. Tweak the "Every XX hours YY minutes after the start of the hour" to consistently use the start of the first backup time.

I am happy with backup once per day because I use TM and CCC to do local backups every hour and Arq (stored on GoogleDrive and OneDrive) is for recovery from "serious" disasters and for that daily is sufficient for me.

Again, we are on the same page in that I too:

1. Have TimeMachine running continuously in teh background.

2. Have CCC run two different daily backups (though not hourly, I leave this to TimeMachine).

I will likely use Arq once per day but will still ask Stefan to make teh changes.

Thanks for sharing your experience / wisdom!

 

[JoelBC]

UPDATE: I have now completed 7 backups and note the following:

1. @gilby101 is absolutely correct in that n backup plans -> n backup sets (even for backup plans that are identical other than the time). Thank you @gilby101 !

2. If a backup plan is setup to run "Every X hour starting Y minutes after the hour" then
a) the second backup plan will start X hours after the first backup plan end time
b) the third and subsequent backup plans will start X hours after the second and subsequent backup plans start time

3. Written Arq / Stefan to modify Arq to
a) Allow scheduling at multiple specific times; and/or
b) Tweak the "Every XX hours YY minutes after the start of the hour" to consistently use the start time of the backup plan.

Thanks to everyone for their help in working through this!!

 

[JoelBC]

I had a email exchange with Stefan who explained how “Hourly / Every XX hours starting YY minutes after the start of the hour” works! And it is different than we thought!

If, for example, one schedules a backup to occur “Every 12 hours starting 0 minutes after the hour” then Arq interprets / sets backups to run at 12:00:00 UTC and 24:00:00 UTC. With my location being in EST, the backups therefore occur at 07:00 EST and 19:00 EST, which explains why I was locked in to 07:00 EST and 19:00 EST (mystery solved)!

Stefan also shared that he will not be tweaking the scheduling.

With that, I will revert to scheduling the backup at 12:00 AM every night and forgo the mid-day backup.

I hope that helps people! I know it helps me!

PS. Huge thanks to Stefan for taking the time to explain things!!

 

[chabig]

I never noticed that my scheduled start times weren't using local time, but it's true. I just compared my scheduled start time with the local time in the backup log. It's using UTC for all scheduled times.

 

[chown33]

Two product suggestions:
1. He should update the docs to say that all times are in UTC.
2. He should add a very obvious line of text to the scheduling dialog saying "All times are in UTC"

 

[JoelBC]

Two product suggestions:
1. He should update the docs to say that all times are in UTC.
2. He should add a very obvious line of text to the scheduling dialog saying "All times are in UTC"

@chown33 , I agree as it would have been very helpful and saved me a lot of time (i.e., I would not have gone down this rabbit hole). Please feel free to let him know.

On the plus side, there is now a clear understanding of how this works!

 

[svenmany]

Two product suggestions:
1. He should update the docs to say that all times are in UTC.
2. He should add a very obvious line of text to the scheduling dialog saying "All times are in UTC"

I have a Daily scheduled backup set to 22:00. My logs show the backups run at 22:00 PST. And that's consistent with other observations I've made. So, it seems that explicitly specifying a time will use the local timezone. This matches my expectations.

I don't use the Hourly Every x hours option, which appears to be centered around midnight UTC. Had I used that option, I would have seen the start time in the bottom area, just as the opening post shows. I honestly wouldn't have known what to expect for the time basis for such repetition.

I really don't think the product missed the mark with regard to clarity. But, the hourly scheduling could have been a bit more flexible, allowing you to choose the start time of the repetition.

 

[JoelBC]

I have a Daily scheduled backup set to 22:00. My logs show the backups run at 22:00 PST. And that's consistent with other observations I've made. So, it seems that explicitly specifying a time will use the local timezone. This matches my expectations.

I don't use the Hourly Every x hours option, which appears to be centered around midnight UTC. Had I used that option, I would have seen the start time in the bottom area, just as the opening post shows. I honestly wouldn't have known what to expect for the time basis for such repetition.

I really don't think the product missed the mark with regard to clarity. But, the hourly scheduling could have been a bit more flexible, allowing you to choose the start time of the repetition.

Agree with everything you wrote other than the clarity of the "Every XX hours starting YY minutes after the hour". It was not the least bit clear, at least to me, that I kept seeing 7:00 AM and 7:00 PM because it is based on UTC shifted for local time zones! This is not intuitive!