Time Machine backups painfully slow on NAS (smb)

[wkc-]

Still pounding my network with traffic FROM the NAS and not TO the Mac (solid average of > 20 Mbps). Same backup reported an hour ago now up to 216 MB of 1.79 GB (up from 57 MB) but now 9 hours remaining rather then 6.

I repeat my assertion that either TM is nuts or my setup is (or both)... I fully understand that something like TM is (most likely) designed to operate with a myriad of different setups and configurations, and thus there might not be a simple answer to my specific issue... but non of this diminishes my desire to know why the traffic flows the direcltion it does.

I will leave this thread alone unless someone addresses that question. Meanwhile I will keep looking

 

[SjoukeW]

TimeMachine usually checks the backup to see if it is correct. Usually it does that after making the backup. In order to check if all files are correct they should be read.
if the files changed are only 2gb, but the size of all the files is 40gb, then it should read 40gb to see if it is still uncorrupted.

 

[Delarado]

I am also having this issue... I have Big Sur 11.2.2 backing up via Time machine to a Synology NAS via an SMB Share. The Time machine bundle is using APFS.

tmutil status from CLI shows the same number of bytes copied for over an hour now. I put a whole bunch of new files on my machine recently and its taken over four hours now to do a 200gb update and it doesn't seem to be moving.

I don't much fancy doing all of these custom processes to get an HFS+ bundle, but I may resort to it if this continues.

Has anyone had good success from just starting a brand new backup? This one was created on the first Big Sur public release...

 

[Spike Lightfoot]

...
Has anyone had good success from just starting a brand new backup? This one was created on the first Big Sur public release...

I tried that several times and it always ended up the same way. Since my post above I have had zero problems, and my wifi backups to my Synology NAS have been faster than they ever were.

 

[ZombiePhysicist]

I am also having this issue... I have Big Sur 11.2.2 backing up via Time machine to a Synology NAS via an SMB Share. The Time machine bundle is using APFS.

tmutil status from CLI shows the same number of bytes copied for over an hour now. I put a whole bunch of new files on my machine recently and its taken over four hours now to do a 200gb update and it doesn't seem to be moving.

I don't much fancy doing all of these custom processes to get an HFS+ bundle, but I may resort to it if this continues.

Has anyone had good success from just starting a brand new backup? This one was created on the first Big Sur public release...

Yes, starting with a new APFS backup on my Synology has been crazy faster.

 

[petterihiisila]

Yes, starting with a new APFS backup on my Synology has been crazy faster.

I gave it another chance, but no. No change between 11.0.1 and 11.2.3.

The first backup to a Time Capsule using default Time Machine settings (encrypted APFS sparsebundle) was at normal speed, but now after a week, things are really starting to get slower and slower and slower. The Mac is connected to the Time Capsule with a 1 Gbit Ethernet.

What normally takes about 20 mins into a manually created, encrypted HFS+ sparsebundle, is now 3-4 hours with APFS. I hear the disk grinding, but the progress bar stays put for 5-15 minutes at a time, then nudges forward some 1-100 megabytes. Data received/sent barely moves in Activity Monitor. Data read/written, same thing. The disk itself is verified to be good. And with HFS+ it's fast too.

 

[ZombiePhysicist]

I gave it another chance, but no. No change between 11.0.1 and 11.2.3.

The first backup to a Time Capsule using default Time Machine settings (encrypted APFS sparsebundle) was at normal speed, but now after a week, things are really starting to get slower and slower and slower. The Mac is connected to the Time Capsule with a 1 Gbit Ethernet.

What normally takes about 20 mins into a manually created, encrypted HFS+ sparsebundle, is now 3-4 hours with APFS. I hear the disk grinding, but the progress bar stays put for 5-15 minutes at a time, then nudges forward some 1-100 megabytes. Data received/sent barely moves in Activity Monitor. Data read/written, same thing. The disk itself is verified to be good. And with HFS+ it's fast too.

Wow. So sorry to hear. I wonder what is causing the super big difference in experience. I take it youre on a Synology too? Im using AFS only for the connection. What are you using?

 

[petterihiisila]

Wow. So sorry to hear. I wonder what is causing the super big difference in experience. I take it youre on a Synology too? Im using AFS only for the connection. What are you using?

It's an Apple Time Capsule with an aftermarket 6 TB disk. Connected using afp, as it did by default.

 

[Chris.J]

Guys, something that worked for me that I thought I would share. I too noticed that a new APFS backup was very slow on my Synology NAS. I found a way to go back to a HFS+ backup without using terminal commands etc.

Format a spare USB hard disk as HFS+ and connect to the Synology. The NAS will mount the drive and set up a share automatically. Set this share as your Time Machine folder in the NAS settings. On your Mac, the disk will now appear as an available disk in Time Machine preferences. Run the backup as normal, creating a HFS+ sparsebundle. Once done, using File Station on the NAS, copy all files over from the USB drive to a new share on your main NAS volume. Repeat the process to set this share as the Time Machine folder and point your Time Machine to this new share again. It will say there is already a backup present, to which you accept and continue as normal.

 

[RiderX]

Don’t use HFS-sparsebundles with Big Sur. This is outdated and not recommended. There is a reason why APFS has to be used.

I use only APFS timemachine backups with my NAS. And it just works, better than Catalina and such.

 

[ZombiePhysicist]

It's an Apple Time Capsule with an aftermarket 6 TB disk. Connected using afp, as it did by default.

I wonder if this is a function of apple no longer supporting time capsules and they are not being updated to deal well with APFS.

 

[Spike Lightfoot]

Don’t use HFS-sparsebundles with Big Sur. This is outdated and not recommended. There is a reason why APFS has to be used.

I use only APFS timemachine backups with my NAS. And it just works, better than Catalina and such.

...except it doesn't work for all of us, hence all the work some of us have had to do. I'm glad APFS works for you, and I wish it worked for me. Perhaps one day we'll figure out why.

 

[petterihiisila]

I wonder if this is a function of apple no longer supporting time capsules and they are not being updated to deal well with APFS.

In the default setup that Big Sur produces, Time Capsule itself is not dealing with APFS. It's dealing with an HFS+ disk that it creates. The host disk.

Inside the host disk, there's a sparsebundle, which is indeed APFS by default in Big Sur. But the Time Capsule has no way of knowing what the internal format of the bundle is. It's just serving the bands over the wire. MacOS does all the work of converting those bits into an APFS file system, mounted in /Volumes.

That is to say, this probably has nothing to do with Time Capsule. There's no mechanism for it to make a difference.

Furthermore, the behavior is exactly the same with a Raspberry Pi as a backup target. The host disk could be formatted into anything that the Pi supports, as long as it's served over SMB/AFP. The backup target must be a sparsebundle. If that bundle is APFS, backups get slower over a week, becoming unusably slow. If that bundle is HFS+, the speed stays constant over the weeks.

If "there is a reason why APFS has to be used", it better be good, because if I use APFS on NAS boxes, it just doesn't work very long. After a few weeks, I could only produce one backup a day. But I don't think there are any reliable references to back up that statement/opinion.

The behavior is the same with MBP16 and M1 Air. As far as I can determine, "APFS sparsebundle on a NAS" is the only common factor that consistently causes slow incremental backups for me and allegedly many others, but not all.

 

[RiderX]

The file system on the NAS doesn’t matter. The NAS must serve a sparsbundle which consists of many small files and directories. Nothing fancy about that and nothing Mac specific.

The default sparsebundle Timemachine creates is an APFS filesystem. And you should stick to that. Timemachine on HFS was always a dirty hack around the limitations of HFS. That has been solved with APFS and Big Sur. Because APFS uses built in snapshots and other features missing in HFS.

A slow backup means there is something wrong with your NAS. Or there is something wrong with Macs access to the NAS. This is where you should start debugging. Not using an outdated filesystem just because it shows faster results. You don’t solve anything, you just work around some bug.

 

[petterihiisila]

The file system on the NAS doesn’t matter. The NAS must serve a sparsbundle which consists of many small files and directories. Nothing fancy about that and nothing Mac specific.

The default sparsebundle Timemachine creates is an APFS filesystem. And you should stick to that. Timemachine on HFS was always a dirty hack around the limitations of HFS. That has been solved with APFS and Big Sur. Because APFS uses built in snapshots and other features missing in HFS.

A slow backup means there is something wrong with your NAS. Or there is something wrong with Macs access to the NAS. This is where you should start debugging. Not using an outdated filesystem just because it shows faster results. You don’t solve anything, you just work around some bug.

Not all can use APFS because for some it behaves slower by an order of magnitude, over time. 20 mins vs all day. Details described earlier in this thread.

Confident snap judgements about “must be something wrong with” are borderline arrogant. Remote allegations about faulty hardware or connectivity. Semi-true but greatly exaggerated statements about HFS+ shortcomings. Are you trolling? Either way, this wasn’t a very helpful or informative contribution IMO. Sorry.

The two kinds of hardware and connections I have are verifiably fast and reliable. And I doubt everyone’s Synology is broken selectively for APFS either. But I’m glad it works for many. Maybe most. We don’t have the telemetry to make conclusions.

Anyway, let’s get constructive, please. This has been a good thread for a while.

 

[Steve121178]

I wonder if this is a function of apple no longer supporting time capsules and they are not being updated to deal well with APFS.

I thought it was my time capsule, so I tried backing up to a external HDD (APFS formatted) as a test. First initial backup is pretty fast, then the hourly ones soon become painfully slow.

My iMac is the only machine still on Catalina and time machine backups are acceptable. My M1 MBA & Intel MBP both on Big Sur are painfully slow.

M1 MacBook Air took 5 hours to backup 7.1GB earlier to time machine.

MacBook Pro took 3 hours to backup 4GB to external HDD.

Backups are horrible with Big Sur

 

[ZombiePhysicist]

I thought it was my time capsule, so I tried backing up to a external HDD (APFS formatted) as a test. First initial backup is pretty fast, then the hourly ones soon become painfully slow.

My iMac is the only machine still on Catalina and time machine backups are acceptable. My M1 MBA & Intel MBP both on Big Sur are painfully slow.

M1 MacBook Air took 5 hours to backup 7.1GB earlier to time machine.

MacBook Pro took 3 hours to backup 4GB to external HDD.

Backups are horrible with Big Sur

Really interesting. I wonder what's causing the crazy variations in experience here. So weird.

 

[Zerex71]

Greetings,

I know I'm way late to join this conversation, so please forgive anything that might seem like it was already covered or perhaps novice. (Please be patient, I promise I'm going somewhere with this long post.)

I'm currently running a single 2013 27" iMac running Catalina 10.15.7. I have a 3 TB AirPort Time Capsule (TC) and run backups exclusively through Time Machine (TM). Based on the facts that a) I'm fairly certain the wifi on it is about to go, b) all backups even of modest size are extremely slow, and c) Apple no longer makes TCs, I've been researching potential backup solutions not only for my current machine, but for a potential new iMac which I suspect will likely have to get purchased this year. I'm in a bit of a mental race to get a backup solution in place so that when it comes time to restore a new iMac from backup, I'm all set, and not unable to do so because of a failing TC.

I spent a lot of time the past couple of weeks researching an appropriate solution, and the short answer is that there seem to be two routes: Either just get a standard USB PnP external drive large enough to hold all my data, or get a NAS. Initially, I didn't want to get into the NAS world because I don't want to spend all my free time being a babysitter to a more complicated appliance/device, but now that I'm thinking about it, I'm okay with a NAS or NAS-like solution.

I was about to pull the trigger on a WD Ultra EX2 MyCloud, which is how I wound up in this thread. But the more threads from here and over on the WD forums that I read, the less convinced I am that this solution will even work for me, which is alarming. It seems like every single poster on every thread in every forum is having problems trying to use TM to do backups on their NAS, and it seems like once you upgrade to Big Sur/macOS 11 (?), the problem gets even worse. At this point, I'm thoroughly confused as to what to do. I thought I had a solution all picked out, and now I'm not sure at all, but I need to get something in place soon.

My questions are as follows:

1. Is it true that you don't or can't actually directly attach an external device such as a NAS (other than a standard USB drive) to your Mac and use it for TM backups?

2. An Apple article about doing backups suggests that, "Sure, you can do backups on a NAS, if they support SMB." I had to go look up what that was, but still really don't get it (or frankly care - I just want a backup solution). Is there any way to know if a given appliance/device supports SMB? That would obviously dictate what brand/product I would select.

3. Are all TM backups done wirelessly? Do they *have* to be done this way? It seems like, even with my current setup, I cannot force backups to be done any other way. I naively thought in years past that by plugging an Ethernet cable from the back of my Mac into my TC that it was doing backups that way. Apparently, that's not what happens. I even went out and bought a USB-to-Ethernet adapter and an Ethernet cable because of course, after seven years, my Ethernet port is bad/non-functioning, so the only remaining hardwired option to try was to go USB out of the back of the Mac to one of the Ethernet ports on the TC, but that had no effect (at least not in the time I was on the phone with the Apple support guy).

4. Is there any way to force a Mac to do a backup over a physical connection instead of wifi? Why would that limitation even be in place by Apple? It seems dumb, knowing that it's slower, more error-prone, etc.

5. Is it the general consensus of the group that things only get worse if you go to Big Sur? I'm not looking forward to that, and when I say "not looking forward to that", I mean in a flip-the-table, what-is-this-BS kind of way.

I see that I just finished a 3.74 GB backup. I know it took about three hours, because I left off the phone call with the hardware back in the original configuration, which was just that I removed my brand-new cable from both ends, went out to grab dinner, came back, read through this whole thread, and it just finished a little bit ago.

Thanks for humoring me.

Mike

 

[Spike Lightfoot]

Greetings,

I know I'm way late to join this conversation, so please forgive anything that might seem like it was already covered or perhaps novice. (Please be patient, I promise I'm going somewhere with this long post.)

I'm currently running a single 2013 27" iMac running Catalina 10.15.7. I have a 3 TB AirPort Time Capsule (TC) and run backups exclusively through Time Machine (TM). Based on the facts that a) I'm fairly certain the wifi on it is about to go, b) all backups even of modest size are extremely slow, and c) Apple no longer makes TCs, I've been researching potential backup solutions not only for my current machine, but for a potential new iMac which I suspect will likely have to get purchased this year. I'm in a bit of a mental race to get a backup solution in place so that when it comes time to restore a new iMac from backup, I'm all set, and not unable to do so because of a failing TC.

I spent a lot of time the past couple of weeks researching an appropriate solution, and the short answer is that there seem to be two routes: Either just get a standard USB PnP external drive large enough to hold all my data, or get a NAS. Initially, I didn't want to get into the NAS world because I don't want to spend all my free time being a babysitter to a more complicated appliance/device, but now that I'm thinking about it, I'm okay with a NAS or NAS-like solution.

I was about to pull the trigger on a WD Ultra EX2 MyCloud, which is how I wound up in this thread. But the more threads from here and over on the WD forums that I read, the less convinced I am that this solution will even work for me, which is alarming. It seems like every single poster on every thread in every forum is having problems trying to use TM to do backups on their NAS, and it seems like once you upgrade to Big Sur/macOS 11 (?), the problem gets even worse. At this point, I'm thoroughly confused as to what to do. I thought I had a solution all picked out, and now I'm not sure at all, but I need to get something in place soon.

My questions are as follows:

1. Is it true that you don't or can't actually directly attach an external device such as a NAS (other than a standard USB drive) to your Mac and use it for TM backups?

2. An Apple article about doing backups suggests that, "Sure, you can do backups on a NAS, if they support SMB." I had to go look up what that was, but still really don't get it (or frankly care - I just want a backup solution). Is there any way to know if a given appliance/device supports SMB? That would obviously dictate what brand/product I would select.

3. Are all TM backups done wirelessly? Do they *have* to be done this way? It seems like, even with my current setup, I cannot force backups to be done any other way. I naively thought in years past that by plugging an Ethernet cable from the back of my Mac into my TC that it was doing backups that way. Apparently, that's not what happens. I even went out and bought a USB-to-Ethernet adapter and an Ethernet cable because of course, after seven years, my Ethernet port is bad/non-functioning, so the only remaining hardwired option to try was to go USB out of the back of the Mac to one of the Ethernet ports on the TC, but that had no effect (at least not in the time I was on the phone with the Apple support guy).

4. Is there any way to force a Mac to do a backup over a physical connection instead of wifi? Why would that limitation even be in place by Apple? It seems dumb, knowing that it's slower, more error-prone, etc.

5. Is it the general consensus of the group that things only get worse if you go to Big Sur? I'm not looking forward to that, and when I say "not looking forward to that", I mean in a flip-the-table, what-is-this-BS kind of way.

I see that I just finished a 3.74 GB backup. I know it took about three hours, because I left off the phone call with the hardware back in the original configuration, which was just that I removed my brand-new cable from both ends, went out to grab dinner, came back, read through this whole thread, and it just finished a little bit ago.

Thanks for humoring me.

Mike

1. You should be able to connect a NAS, but the "N" in NAS means network, so by definition it's on ethernet or wifi and not USB. I think you can create an ad-hoc ethernet network just between the NAS and your Mac but I haven't done that in years.
2. Virtually all NAS's currently on the market support SMB.
3. You can do a NAS backup on a removable disk (e.g. USB disk connected to your router), then disconnect it and reconnect it to your Mac for subsequent backups, and vice versa.
4. If you're simultaneously connected via ethernet and wifi, the service highest in order in the Network Preference Pane is the one doing the work for the TM and other network activities.
5. It got worse for me and others on this forum after moving to Big Sur. Some on Big Sur also with Synology NAS seem not to have problems. I wish I knew why!
-Spike

 

[HDFan]

My questions are as follows:

1. Is it true that you don't or can't actually directly attach an external device such as a NAS (other than a standard USB drive) to your Mac and use it for TM backups?

2. An Apple article about doing backups suggests that, "Sure, you can do backups on a NAS, if they support SMB." I had to go look up what that was, but still really don't get it (or frankly care - I just want a backup solution). Is there any way to know if a given appliance/device supports SMB? That would obviously dictate what brand/product I would select.

3. Are all TM backups done wirelessly? Do they *have* to be done this way? It seems like, even with my current setup, I cannot force backups to be done any other way. I naively thought in years past that by plugging an Ethernet cable from the back of my Mac into my TC that it was doing backups that way. Apparently, that's not what happens. I even went out and bought a USB-to-Ethernet adapter and an Ethernet cable because of course, after seven years, my Ethernet port is bad/non-functioning, so the only remaining hardwired option to try was to go USB out of the back of the Mac to one of the Ethernet ports on the TC, but that had no effect (at least not in the time I was on the phone with the Apple support guy).

4. Is there any way to force a Mac to do a backup over a physical connection instead of wifi? Why would that limitation even be in place by Apple? It seems dumb, knowing that it's slower, more error-prone, etc.

5. Is it the general consensus of the group that things only get worse if you go to Big Sur? I'm not looking forward to that, and when I say "not looking forward to that", I mean in a flip-the-table, what-is-this-BS kind of way.

1. TM works with external devices, either via the network, ethernet, wifi, thunderbolt, USB. Performance on some devices/connections is terrible, fine on others.

2. As stated above most NAS devices support SMB. However network TM backups can be slow even on fast 10 GBe networks, may be unusable on wifi networks.

3. Wireless TM backups should be avoided if possible, depending upon your backup size and network bandwidth. I think that most are not done wirelessly, but have no data to actually confirm that suspicion. You can test the usb-ethernet bandwidth by running a testing program such as blackmagic. You need to look at the speed of your USB port. USB 1.1 is only 12 Mbps. Wifi could be faster.

4. Not sure I understand this one. Directly connected devices show in the finder and hopefully show up in the list of time machine disks. To confirm that you are directly connected turn off wifi. If the disk disappears then, obviously, it was connected via wifi. This will not happen with most disks (USB, thunderbolt). Such disks do not have wifi connectivity at all. Time capsules are the only device I can think of where this might happen.

5. I have had no problems with TM backups on Big Sur with directly connected disks. I have never found TM on NAS, either networked or via thunderbolt, to be reliable.

 

[ZombiePhysicist]

1. TM works with external devices, either via the network, ethernet, wifi, thunderbolt, USB. Performance on some devices/connections is terrible, fine on others.

2. As stated above most NAS devices support SMB. However network TM backups can be slow even on fast 10 GBe networks, may be unusable on wifi networks.

3. Wireless TM backups should be avoided if possible, depending upon your backup size and network bandwidth. I think that most are not done wirelessly, but have no data to actually confirm that suspicion. You can test the usb-ethernet bandwidth by running a testing program such as blackmagic. You need to look at the speed of your USB port. USB 1.1 is only 12 Mbps. Wifi could be faster.

4. Not sure I understand this one. Directly connected devices show in the finder and hopefully show up in the list of time machine disks. To confirm that you are directly connected turn off wifi. If the disk disappears then, obviously, it was connected via wifi. This will not happen with most disks (USB, thunderbolt). Such disks do not have wifi connectivity at all. Time capsules are the only device I can think of where this might happen.

5. I have had no problems with TM backups on Big Sur with directly connected disks. I have never found TM on NAS, either networked or via thunderbolt, to be reliable.

With regard to 5, I had the same issue. And for me I figured out the problem of bad reliability. TimeMachine NEEEDS your first backup to Time Machine to be completely uninterrupted. In the past, my backup is so big it takes over 24 actual hours to backup and often the machine would need to be rebooted etc. That would halt the initial backup and then finish it later. This ALWAYS caused problems down the line.

Then, on a weird hunch, after having TimeMachine backups on NAS corrupted for years, I tried a continuous backup with no interruptions on that first/initial backup. Ever since then, it's been working great. So that first backup, it's critical. Don't let the machine go to sleep. Dont let it get stopped or interrupted in anyway and for TimeMachine to do a full backup uninterrupted. Maybe it will make as big a difference to you as it did for me.

 

[SjoukeW]

About two years ago I was working as a software developer in a small company. When we switched to mac's we bought a nas with sufficient capacity to host all the backups for all developers.
We noticed several things... the size of the data is not the only important factor in the speed of the backup, the amount of files does really matter. Backing up literally millions of files is slow, backing up 1 large file is fast.
We also noticed that encryption on the backup can really slow it down. We played around with encryption on the nas, encryption on the backup and the nas. The speed difference between encrypted backups and unencrypted backup is huge.
In order to test this out we also tried to make the same bakcup to a external hard disk with and without encryption. Without encryption it was a couple of hours, with encryption it took almost a day.

I hope this info helps somebody.

 

[petterihiisila]

About two years ago I was working as a software developer in a small company. When we switched to mac's we bought a nas with sufficient capacity to host all the backups for all developers.
We noticed several things... the size of the data is not the only important factor in the speed of the backup, the amount of files does really matter. Backing up literally millions of files is slow, backing up 1 large file is fast.
We also noticed that encryption on the backup can really slow it down. We played around with encryption on the nas, encryption on the backup and the nas. The speed difference between encrypted backups and unencrypted backup is huge.
In order to test this out we also tried to make the same bakcup to a external hard disk with and without encryption. Without encryption it was a couple of hours, with encryption it took almost a day.

I hope this info helps somebody.

Thanks. Useful comparison. I haven’t found a viable scheme that would enable encryption at NAS level, but the backup target would appear unencrypted for Time Machine. Using a Time Capsule and a Raspberry Pi. I suppose you still have a layer of encryption somewhere, since it sounds like you’re backing up source code for a business?

I hope people don’t make this conclusion: “OK, if that‘s the case, I won’t encrypt my backups in any way then.”

A NAS box is a high-value target for an intelligent burglar. Stealing all your data history enables all sorts of extortion and hair/money loss for years. Some form of encryption is a must, IMO, whether or not the NAS stores company data. That’s why I use encryption with every disk, internal and external, even if it necessarily creates some performance overhead.

 

[Spike Lightfoot]

FWIW, despite all my woes, another Big Sur user in our household with a new M1 MBA has been backing up to the Synology NAS for days without any trouble using the default APFS. Go figure.

 

[ARMunix]

I have a different question: While TM backups from my M1 Mac to my Synology NAS work fine (speed is kinda okay-ish), I noticed that there's continuous IO happening on the NAS - even after the TM backup is finished. Looks like there's some kind of re-indexing of the existing backups happening? Anybody noticed this, too?