kernel_task running amok on network BS 11.3 (EDIT still present in 11.4)

Spindel · Apr 27, 2021

Since the 11.3 update kernel_task (KT) is running amok when it comes to network activity. I have a M1 Mini, connected to the network by wire.

I use TimeMachine (TM) to back up to my NAS. Since the 11.3 update as soon as my system has done an backup KT starts to read my NAS with high network activity. I hear the HDD in my NAS clicking away so I know the behavior is related to the NAS. And since it only starts to happen after the first TM backup after a fresh boot I assume it is connected to that.

What the hell is it doing is it indexing my TM backup and does it need to index the entire thing after each back up?
This hasn't been an issue prior 11.3.

Yesterday KT read about 150 GB over the network, so far today (2 h uptime) it has read 15 GB over the network. The speed it reads over the network varies but it is long times at 20 MB/s (which isn't the max read speed of my NAS since at least for larger files I've reached 60-65 MB/s).

This issue does not come with any excessive disk writes nor high CPU usage. Just high (local)network usage and high disk activity on my NAS.

Anyone else that have noticed this and anyone have an idea how to make it stop, or should I just let it run and hope it stops once it has read my entire volume dedicated to TM?

In fairness this doesn't pose any other problem other than a lot of reading off the NAS since I don't notice any performance penalty while using the computer (as long as I don't need something of the NAS).

EDIT:// Aaaaand about 5 min after I made this post it stopped, will se if it starts again after the next TM backup.

EDIT2:// another TM backup done and now the network activity starts again.

EDIT3:// This is insane so far today my computer has read 120 GB (6 h uptime) over the network and about 100 GB of those are probably KT!

EDIT4:// So for the time being the only fix to this for me seem to be to disable automatic TM backups. After each back up KT starts to read my TM disk (excluding it from spotlight did nothing to fix this issue), it reads the disk for about 50 minutes before stopping, then 10 minutes later it is time for the next TM back up and the cycle restarts. I'm not that worried about my HDDs in the NAS, they can probably take this constant reading for a couple of years but it seems unnecessary. So for the time being I'll have to live with manual TM back ups.

schwartzki · May 3, 2021

Also having this, over 150gb of "kernal task" network activity in the last few days.

Spindel · May 4, 2021

Problem persists after 11.3.1. I hope we get an update that sorts this.

schwartzki · May 4, 2021

I rebooted my MBP and it went down to no data for the rest of the night but might start up again next use. My iMac seems unaffected so far.

Little Snitch doesn't seem to be picking up much activity so it does seem to be local network primarily.

Spindel · May 4, 2021

schwartzki said:
I rebooted my MBP and it went down to no data for the rest of the night but might start up again next use. My iMac seems unaffected so far.

Little Snitch doesn't seem to be picking up much activity so it does seem to be local network primarily.

For me it’s only local. Seems to scan the entire back up after each back up.

Spindel · May 24, 2021

11.4 installed this issue is still present

dim_ · May 25, 2021

I have been seeing exactly the same issue since upgrading from 11.2 to 11.3, and still on 11.4. (Note that other users seem to be discussing similar phenomenons in https://forums.macrumors.com/threads/time-machine-backups-painfully-slow-on-nas-smb.2276945/, see the last page of that thread).

I even saved some logs from 11.2, where a typical Time Machine session would look like this in Console:

2021-05-24 14:46:48.993451+0200  localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Starting automatic backup
2021-05-24 14:46:49.166403+0200  localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Attempting to mount 'smb://dim@backupserver._smb._tcp.local./tmtest'
2021-05-24 14:46:53.570777+0200  localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Mounted 'smb://dim@backupserver._smb._tcp.local./tmtest' at '/Volumes/.timemachine/backupserver._smb._tcp.local./6E2D2368-48F5-4AAF-986A-7349FE3631D1/tmtest'
[... backing up takes a while ...]
2021-05-24 14:48:03.262943+0200  localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Creating APFS snapshot com.apple.TimeMachine.2021-05-24-144756.backup
2021-05-24 14:48:03.792080+0200  localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:Manifest] 2 backups: 2021-05-24-142835 to 2021-05-24-144756 (2hr) MΔ: 1,64 GB
2021-05-24 14:48:03.895249+0200  localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Completed backup: 2021-05-24-144756.backup
2021-05-24 14:48:03.895904+0200  localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Mountpoint '/Volumes/Backups of test-bigsur' is still valid
2021-05-24 14:48:04.943790+0200  localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Failed to find unused recovery set index, error: Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory" UserInfo={NSLocalizedDescription=/Backups.backupdb/.RecoverySets: No such file or directory (error 2)}
2021-05-24 14:48:04.946284+0200  localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Mountpoint '/Volumes/Backups of test-bigsur' is still valid
2021-05-24 14:48:05.402734+0200  localhost mdsync[581]: (TimeMachine) [com.apple.TimeMachine:TMSession] Finishing session for '/System/Volumes/Data/Volumes/Backups of test-bigsur' with options: 4
2021-05-24 14:48:05.402754+0200  localhost mdsync[581]: (TimeMachine) [com.apple.TimeMachine:TMSession] Finished session for '/System/Volumes/Data/Volumes/Backups of test-bigsur' with options: 4
2021-05-24 14:48:05.404370+0200  localhost mdsync[581]: (TimeMachine) [com.apple.TimeMachine:TMSession] Finishing session for '/System/Volumes/Data/Volumes/Backups of test-bigsur' with options: 4
2021-05-24 14:48:05.404385+0200  localhost mdsync[581]: (TimeMachine) [com.apple.TimeMachine:TMSession] Finished session for '/System/Volumes/Data/Volumes/Backups of test-bigsur' with options: 4

so very quickly after it says "completed backup", spotlight (aka mdsync) says "finishing session". After that message, the "Backups of test-bigsur" volume has been unmounted, and the disk ejected.

With 11.3 the beginning is mostly the same, but not the end:

2021-05-24 15:46:56.561540+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Starting automatic backup
2021-05-24 15:46:56.565981+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Attempting to mount 'smb://dim@backupserver._smb._tcp.local./tmtest'
2021-05-24 15:47:01.014611+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Mounted 'smb://dim@backupserver._smb._tcp.local./tmtest' at '/Volumes/.timemachine/backupserver._smb._tcp.local./E1A35392-8BDA-4437-887C-F8C53A2FFEF3/tmtest'
[... backing up takes a while ...]
2021-05-24 15:51:35.044025+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Creating APFS snapshot com.apple.TimeMachine.2021-05-24-154917.backup
2021-05-24 15:51:36.538540+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Manifest] 3 backups: 2021-05-24-142835 to 2021-05-24-154917 (1hr,2d) MΔ: 1,34 GB
2021-05-24 15:51:36.587306+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Completed backup: 2021-05-24-154917.backup
2021-05-24 15:51:36.587897+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Mountpoint '/Volumes/Backups of test-bigsur' is still valid
2021-05-24 15:51:37.463455+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Failed to find unused recovery set index, error: Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory" UserInfo={NSLocalizedDescription=/Backups.backupdb/.RecoverySets: No such file or directory (error 2)}
2021-05-24 15:51:37.466594+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:BackupThinning] Thinning 1 backups using age-based thinning, expected free space: 1,75 TB actual free space: 1,75 TB trigger 87,93 GB thin 146,56 GB dates: (
    "2021-05-24-144756"
)
2021-05-24 15:51:37.468698+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Mountpoint '/Volumes/Backups of test-bigsur' is still valid
2021-05-24 15:51:40.229470+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Accepted a new connection from 'Spotlight-mds' for service com.apple.backupd.session.xpc
2021-05-24 15:52:07.889492+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur'
2021-05-24 15:53:08.093613+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur'
2021-05-24 15:55:10.260623+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur'
2021-05-24 15:58:12.507667+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur'
2021-05-24 16:02:12.749660+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur'
2021-05-24 16:07:13.262866+0200  localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur'

After the "completed backup", the volume stays mounted seemingly indefinitely, and backupd says "Waiting for Spotlight to finish indexing" every few minutes.

I've even tried turning off automatic backups to let it "finish indexing" overnight, but this does not really help much. As soon as you make another backup, it starts indexing right at the beginning of the entire volume again.

What also happens is that automatic backups get *cancelled* due to Spotlight having the volume in use! This is because when Time Machine starts a backup, it tries to check the volume for runtime corruption, but to do that it has to unmount and eject it first... :/

I had some hope that 11.4 would fix this, but it behaves exactly the same, unfortunately.

Spindel · May 25, 2021

dim_ said:
I have been seeing exactly the same issue since upgrading from 11.2 to 11.3, and still on 11.4. (Note that other users seem to be discussing similar phenomenons in https://forums.macrumors.com/threads/time-machine-backups-painfully-slow-on-nas-smb.2276945/, see the last page of that thread).

I even saved some logs from 11.2, where a typical Time Machine session would look like this in Console:

2021-05-24 14:46:48.993451+0200 localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Starting automatic backup 2021-05-24 14:46:49.166403+0200 localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Attempting to mount 'smb://dim@backupserver._smb._tcp.local./tmtest' 2021-05-24 14:46:53.570777+0200 localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Mounted 'smb://dim@backupserver._smb._tcp.local./tmtest' at '/Volumes/.timemachine/backupserver._smb._tcp.local./6E2D2368-48F5-4AAF-986A-7349FE3631D1/tmtest' [... backing up takes a while ...] 2021-05-24 14:48:03.262943+0200 localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Creating APFS snapshot com.apple.TimeMachine.2021-05-24-144756.backup 2021-05-24 14:48:03.792080+0200 localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:Manifest] 2 backups: 2021-05-24-142835 to 2021-05-24-144756 (2hr) MΔ: 1,64 GB 2021-05-24 14:48:03.895249+0200 localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Completed backup: 2021-05-24-144756.backup 2021-05-24 14:48:03.895904+0200 localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Mountpoint '/Volumes/Backups of test-bigsur' is still valid 2021-05-24 14:48:04.943790+0200 localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Failed to find unused recovery set index, error: Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory" UserInfo={NSLocalizedDescription=/Backups.backupdb/.RecoverySets: No such file or directory (error 2)} 2021-05-24 14:48:04.946284+0200 localhost backupd[445]: (TimeMachine) [com.apple.TimeMachine:General] Mountpoint '/Volumes/Backups of test-bigsur' is still valid 2021-05-24 14:48:05.402734+0200 localhost mdsync[581]: (TimeMachine) [com.apple.TimeMachine:TMSession] Finishing session for '/System/Volumes/Data/Volumes/Backups of test-bigsur' with options: 4 2021-05-24 14:48:05.402754+0200 localhost mdsync[581]: (TimeMachine) [com.apple.TimeMachine:TMSession] Finished session for '/System/Volumes/Data/Volumes/Backups of test-bigsur' with options: 4 2021-05-24 14:48:05.404370+0200 localhost mdsync[581]: (TimeMachine) [com.apple.TimeMachine:TMSession] Finishing session for '/System/Volumes/Data/Volumes/Backups of test-bigsur' with options: 4 2021-05-24 14:48:05.404385+0200 localhost mdsync[581]: (TimeMachine) [com.apple.TimeMachine:TMSession] Finished session for '/System/Volumes/Data/Volumes/Backups of test-bigsur' with options: 4

so very quickly after it says "completed backup", spotlight (aka mdsync) says "finishing session". After that message, the "Backups of test-bigsur" volume has been unmounted, and the disk ejected.

With 11.3 the beginning is mostly the same, but not the end:

2021-05-24 15:46:56.561540+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Starting automatic backup 2021-05-24 15:46:56.565981+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Attempting to mount 'smb://dim@backupserver._smb._tcp.local./tmtest' 2021-05-24 15:47:01.014611+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Mounted 'smb://dim@backupserver._smb._tcp.local./tmtest' at '/Volumes/.timemachine/backupserver._smb._tcp.local./E1A35392-8BDA-4437-887C-F8C53A2FFEF3/tmtest' [... backing up takes a while ...] 2021-05-24 15:51:35.044025+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Creating APFS snapshot com.apple.TimeMachine.2021-05-24-154917.backup 2021-05-24 15:51:36.538540+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Manifest] 3 backups: 2021-05-24-142835 to 2021-05-24-154917 (1hr,2d) MΔ: 1,34 GB 2021-05-24 15:51:36.587306+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Completed backup: 2021-05-24-154917.backup 2021-05-24 15:51:36.587897+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Mountpoint '/Volumes/Backups of test-bigsur' is still valid 2021-05-24 15:51:37.463455+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Failed to find unused recovery set index, error: Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory" UserInfo={NSLocalizedDescription=/Backups.backupdb/.RecoverySets: No such file or directory (error 2)} 2021-05-24 15:51:37.466594+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:BackupThinning] Thinning 1 backups using age-based thinning, expected free space: 1,75 TB actual free space: 1,75 TB trigger 87,93 GB thin 146,56 GB dates: ( "2021-05-24-144756" ) 2021-05-24 15:51:37.468698+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Mountpoint '/Volumes/Backups of test-bigsur' is still valid 2021-05-24 15:51:40.229470+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:General] Accepted a new connection from 'Spotlight-mds' for service com.apple.backupd.session.xpc 2021-05-24 15:52:07.889492+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur' 2021-05-24 15:53:08.093613+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur' 2021-05-24 15:55:10.260623+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur' 2021-05-24 15:58:12.507667+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur' 2021-05-24 16:02:12.749660+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur' 2021-05-24 16:07:13.262866+0200 localhost backupd[463]: (TimeMachine) [com.apple.TimeMachine:Spotlight] Waiting for Spotlight to finish indexing '/Volumes/Backups of test-bigsur'

After the "completed backup", the volume stays mounted seemingly indefinitely, and backupd says "Waiting for Spotlight to finish indexing" every few minutes.

I've even tried turning off automatic backups to let it "finish indexing" overnight, but this does not really help much. As soon as you make another backup, it starts indexing right at the beginning of the entire volume again.

What also happens is that automatic backups get *cancelled* due to Spotlight having the volume in use! This is because when Time Machine starts a backup, it tries to check the volume for runtime corruption, but to do that it has to unmount and eject it first... :/

I had some hope that 11.4 would fix this, but it behaves exactly the same, unfortunately.

I’ll read the thread you linked later when I have time.

But for the record I have no problems with the back up speed (I mount with AFP so that might matter) but as I stated after back up is finished KT starts to read the TM volume and that process takes about 50 min. EVERY TIME.

dim_ · May 25, 2021

Spindel said:
I’ll read the thread you linked later when I have time.

But for the record I have no problems with the back up speed (I mount with AFP so that might matter) but as I stated after back up is finished KT starts to read the TM volume and that process takes about 50 min. EVERY TIME.

Yes, I forgot to mention that of course, but for me it's approximately the same, apologies!

Because Spotlight re-indexes all previous backups every time, it continuously uses 40-50% CPU in the background. Also, because my backup disks are spinning rust, you can actually hear them grinding most of the time...

As for AFP or SMB, I don't think it makes too much difference in regards to the backup speed (it is sometimes VERY slow regardless), or this continuous Spotlight re-indexing problem. The former has always been the case, the latter is new since 11.3.

Spindel · May 25, 2021

dim_ said:
Yes, I forgot to mention that of course, but for me it's approximately the same, apologies!

Because Spotlight re-indexes all previous backups every time, it continuously uses 40-50% CPU in the background. Also, because my backup disks are spinning rust, you can actually hear them grinding most of the time...

As for AFP or SMB, I don't think it makes too much difference in regards to the backup speed (it is sometimes VERY slow regardless), or this continuous Spotlight re-indexing problem. The former has always been the case, the latter is new since 11.3.

Actually my CPU usage isn’t that high (might be because I’m on a M1 machine) so I really don’t notice this other than the HDDs in my NAS clicking away. And I run MenuBar Stats so I see KT going crazy with network activity.

EDIT:// If I would have had my NAS in another room and Not been running MenuBar Stats I probably never would have noticed this thing.

Spindel · Jun 17, 2021

Sooooo...

I'm bumping this thread just to see if anyone figured out how to stop this irritating behavior?

sddawson · Jul 1, 2021

Just replying to add to the voices. I’ve noticed this behaviour too, and I'm kind of relieved it's not just me!

EDIT: In my case, I'm backing up to a Mac Mini share that's set up to be a Time Machine backup destination in its advanced settings.

dim_ · Jul 26, 2021

I reported this to Apple via their Feedback Assistant, but there has been no reply at all, and it still has not been fixed in 11.5.

Spindel said:
Sooooo...

I'm bumping this thread just to see if anyone figured out how to stop this irritating behavior?

The only thing that helps is to manually unmount all the backup snapshots under /Volumes/.timemachine, using diskutil unmount, then ejecting the backup disk (usually mounted as /Volumes/Backups of <your computername>), using diskutil eject /dev/diskX (where X is the disk number).

For my own machine I wrote a shell script that does this, but it's rather tricky to make that work for all possible users and environments... :/

sddawson · Jul 26, 2021

dim_ said:
I reported this to Apple via their Feedback Assistant, but there has been no reply at all, and it still has not been fixed in 11.5.

I’ve reported it too. I guess the more people that do, the more it might be taken notice of. Disappointing there’s no change in 11.5.

I’m not sure unmounting the drives while it’s doing this is the best idea. We do know that Time Machine relies on Spotlight to do its job, so I’d worry that the backups aren’t properly usable. But who knows? Personally, I’ve turned off automatic backups and go round and do a manual one every week on all our Macs. That’s a real pain though. I do run iDrive backups everywhere, so at least that catches snapshots of all the important user files.

Search

Search

kernel_task running amok on network BS 11.3 (EDIT still present in 11.4)

Spindel

macrumors 6502a

schwartzki

macrumors member

Spindel

macrumors 6502a

schwartzki

macrumors member

Spindel

macrumors 6502a

Spindel

macrumors 6502a

dim_

macrumors newbie

Spindel

macrumors 6502a

dim_

macrumors newbie

Spindel

macrumors 6502a

Spindel

macrumors 6502a

sddawson

macrumors member

dim_

macrumors newbie

sddawson

macrumors member

Our Staff