ssd swap - high usage of Terabytes Written

[panjandrum]

30.6 GB written in 15 minutes

It's a long thread, so I'm sure someone has already mentioned this; but just in case - you can use Activity Monitor to see what app/process is writing all the data. Maybe that will help narrow this down, so if you haven't taken that step yet definitely do so.

 

[VitoBotta]

It's a long thread, so I'm sure someone has already mentioned this; but just in case - you can use Activity Monitor to see what app/process is writing all the data. Maybe that will help narrow this down, so if you haven't taken that step yet definitely do so.

The only process that is writing a ton is kernel_task. Who knows what's doing.. It does a lot of stuff

 

[BacioiuC]

Quick Update: After 30 minutes, I checked up on my M1 MBPro and it hadn't written anything while sleeping. I restarted it, plugged it in, check the data to make sure nothing changed (about 400 mb written by kernal_d on startup, launching stuff and what not). Took a screenshot and let it to sleep for 15 minutes while plugged in.

3 GB written.

NOTE that through out the day, not sleeping on battery I had maybe 7 GB in total written the entire day so 3 GB in 15 minutes is a lot.

I took another screenshot, started some apps and put it to sleep again. Be back in 30-40 minutes to see how things are going buut maybe we have our first indication? Sleep while plugged in/charging could upset something (like some task checking if the battery is charged or something [same process could be running on m1 battery-less macs like the mini]).

See you guys soon.

 

[Quackers]

The only process that is writing a ton is kernel_task. Who knows what's doing.. It does a lot of stuff

Wasn't launchd doing the same for someone?

 

[VitoBotta]

Quick Update: After 30 minutes, I checked up on my M1 MBPro and it hadn't written anything while sleeping. I restarted it, plugged it in, check the data to make sure nothing changed (about 400 mb written by kernal_d on startup, launching stuff and what not). Took a screenshot and let it to sleep for 15 minutes while plugged in.

3 GB written.

NOTE that through out the day, not sleeping on battery I had maybe 7 GB in total written the entire day so 3 GB in 15 minutes is a lot.

I took another screenshot, started some apps and put it to sleep again. Be back in 30-40 minutes to see how things are going buut maybe we have our first indication? Sleep while plugged in/charging could upset something (like some task checking if the battery is charged or something [same process could be running on m1 battery-less macs like the mini]).

See you guys soon.

Dunno... my gut feeling tells me that it's not related to sleep somehow

 

[VitoBotta]

I have just disabled Spotlight completely rather than just adding the disk to the Privacy tab in the settings. This is to ensure that the mds process is not running at all, because it might still write to the index on the main disk while indexing the external drives.

Can anyone else try the same and report in one day to see if it makes any difference?

To disable Spotlight completely, reboot in Recovery Mode (shutdown, then keep pressed the power button until the screen shows a "Load options" menu or something like that, then click on Options) and run "csrutil disable" in the terminal. Double check that SIP is disabled (that's a requirement for this) with "csrutil status".

Then restart, and run this in the terminal:
"sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist"

When/if you want to enable Spotlight again, run
"sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist"

Then reboot in the Recovery Mode and run "csrutil enable".

I would appreciate if someone could also try this.

 

[VitoBotta]

This is interesting.... before, EACH time I started all of my usual apps including Docker with my usual containers for work, the swap would quickly climb in a matter of minutes. I have opened absolutely all the stuff that I use and the swap is still at 2.64 GB only. The mds process is not running as expected (and the external drives are so quiet now!) and kernel_task has written 6.13 GB total so far.

I am really curious if disabling Spotlight completely is making a difference.

 

[BacioiuC]

I'm back and I THINK I nailed the behavior. It seems to be sleep related (due to how sleep functions) but also due to many apps loaded in memory.

Describing the situation now, but remember my 425 GB written screenshot from 1.5 hours ago? THis is the current situation:

Describing the issue in the next post.

 

[VitoBotta]

BTW just to compare numbers, what is an acceptable amount of data written per hour or per day? For example for people who are not having this issue. Thanks

 

[BacioiuC]

The way I understand sleep works is that it saves the status of your loaded apps into memory so when you wake up, they are there and prepared.

Now, if your apps take more than the amount of memory you have they "memory" gets compressed and written to the SSD.

When you go into sleep, this happens - it saves the state of all the apps, some to memory, some compressed to the drive. Now if something wakes up from sleep and then goes back down the cycle repeats? My guess is there's a process running in the background that periodically wakes up my mac and seeing that this is the first time it happened and the first time I put it to sleep while plugged in - my guess is that it's related to a energy monitor/battery monitor or some other daemon running.

@VitoBotta: I had 425 gb written over the course of the week and about 380-400 of those gigabytes I could account for (I did some math and showed what I had installed on page 26). Up till the sleep experiment, through out the entire day I had about 7 gb written in total.

30 gb in half an hour while sleeping for me it's not normal.

Edit: forgot to mention, before the last sleep experiment with the mac plugged in - I loaded up safari with tabs, opened up my unity project and all the apps in my dashboard to load up the amount of RAM occupied.

 

[flopticalcube]

BTW just to compare numbers, what is an acceptable amount of data written per hour or per day? For example for people who are not having this issue. Thanks

Anything less than half your drive capacity per day, ideally 1/10th or less.

 

[VitoBotta]

The way I understand sleep works is that it saves the status of your loaded apps into memory so when you wake up, they are there and prepared.

Now, if your apps take more than the amount of memory you have they "memory" gets compressed and written to the SSD.

When you go into sleep, this happens - it saves the state of all the apps, some to memory, some compressed to the drive. Now if something wakes up from sleep and then goes back down the cycle repeats? My guess is there's a process running in the background that periodically wakes up my mac and seeing that this is the first time it happened and the first time I put it to sleep while plugged in - my guess is that it's related to a energy monitor/battery monitor or some other daemon running.

@VitoBotta: I had 425 gb written over the course of the week and about 380-400 of those gigabytes I could account for (I did some math and showed what I had installed on page 26). Up till the sleep experiment, through out the entire day I had about 7 gb written in total.

30 gb in half an hour while sleeping for me it's not normal.

30 GB during sleep? That's nuts.

I am seeing some difference with Spotlight disabled too.... in 23 minutes only 10 GB have been written total! Before It was 130-150 GB per hour or even worse.

 

[VitoBotta]

Anything less than half your drive capacity per day, ideally 1/10th or less.

Thanks, that number helps.

 

[eyetic]

So we should disable "sleep" for good?

 

[BacioiuC]

Not sure but it was the only thing different (outside of the apps I was running) between me not having the problem and the people that reported the problem.

I can see how it's more common on 8Gb max since what ever is not loaded into memory due to not enough ram, get's written to the SSD. A go to sleep, load into ram or store into ram and waking up in the background and repeating the process again could trigger it.

My initial sleep tests didn't have much effect (3gb written max) until I saw that VitoBotta has a lot of apps opened up (he posted the list of running apps on page 26). That's what got me to load a lot of apps, take a screenshot and let it sleep while plugged in.

And lo-and-behold it happened to me 2.

Edit: Normally, my ram usage is around 10-14 GB out of 16 and I don't have tooooo many apps open. Unity Hub + Unity + Visual Studio Code + Discord + whatsapp (rarely) + limeChat for IRC + safari with 1-3 tabs max. I had to load a lot of Unity instances and projects to go over 16 GB and into the swap before i put it to sleep.

 

[k-hawinkler]

So we should disable "sleep" for good?

Well I always disabled sleep because my mini couldn’t wake up the LG display.

 

[inversed]

Absolutely no difference before and after sleep for mine. No sleep problem with my screen either (Eizo CG276). Again, i don't have TimeMachine or iCloud activated.

 

[BacioiuC]

@inversed: the amount of apps opened up also matters. A lot of them => lot of ram, it gotta save their "state" to ram and if it runs out of ram, to the SSD.

That's the problem. Waking from sleep, read, go to sleep, write repeatedly. Again, until now I had not had this behavior because 1) I stay under ram limit (not many apps running at the same time) and almost never get into swap, 2) I never let it sleep.

Can you try maxing out the ram usage and putting it to sleep?

 

[VitoBotta]

The first hour with Spotlight completely disabled 25.39 GB have been written to the main disk and the swap is still just 5.94 GB, so there seems to be a big improvement. However if that's the average written per hour it would be still over 600 GB per day while my drive is 256 GB...

 

[VitoBotta]

@inversed: the amount of apps opened up also matters. A lot of them => lot of ram, it gotta save their "state" to ram and if it runs out of ram, to the SSD.

That's the problem. Waking from sleep, read, go to sleep, write repeatedly. Again, until now I had not had this behavior because 1) I stay under ram limit (not many apps running at the same time) and almost never get into swap, 2) I never let it sleep.

Can you try maxing out the ram usage and putting it to sleep?

If there is a problem where the OS writes something to disk also when the computer goes just to sleep and not hibernate (which sounds weird to me), that would likely be one single biggish write when it goes to sleep, not some constant writing I think.

 

[BacioiuC]

If there is a problem where the OS writes something to disk also when the computer goes just to sleep and not hibernate (which sounds weird to me), that would likely be one single biggish write when it goes to sleep, not some constant writing I think.

One big-ish write unless the memory delta is low. Then doing subsequent writes every time it's woken up. Let's say there are about 17 GB of memory in usage from apps that need to be saved when going to sleep. it writes 1 gb to the SSD at a time each time it's waken up in the background (16 gb can be stored in memory). That happens (numbers out of my ars) 5 times in 40 minutes during sleep, that's 5 GB written. It's an unaccurate example due to compression and what-not but it's a good way of putting it.

Have you tried only having safari open in your case instead of all the apps on that list and see if the behavior still occurs?

Anyone else not having problems with this, how many apps do you have open? I'm sure my behavior is not 100% correct but it's somewhere in the ballpark of the issue.

 

[Dockland]

If there is a problem where the OS writes something to disk also when the computer goes just to sleep and not hibernate (which sounds weird to me), that would likely be one single biggish write when it goes to sleep, not some constant writing I think.

If there is a problem where the OS writes something to disk also when the computer goes just to sleep and not hibernate (which sounds weird to me), that would likely be one single biggish write when it goes to sleep, not some constant writing I think.

Remembering the old swap days in Gentoo Linux when, and if, configuring hibernation, the rule, so to speak, was to have double the amount of RAM as swap. Or perhaps it was 1,5 the amount, don’t really remember. But for sleep I’ve never heard of that being mandatory.

 

[VitoBotta]

One big-ish write unless the memory delta is low. Then doing subsequent writes every time it's woken up. Let's say there are about 17 GB of memory in usage from apps that need to be saved when going to sleep. it writes 1 gb to the SSD at a time each time it's waken up in the background (16 gb can be stored in memory). That happens (numbers out of my ars) 5 times in 40 minutes during sleep, that's 5 GB written. It's an unaccurate example due to compression and what-not but it's a good way of putting it.

Have you tried only having safari open in your case instead of all the apps on that list and see if the behavior still occurs?

Anyone else not having problems with this, how many apps do you have open? I'm sure my behavior is not 100% correct but it's somewhere in the ballpark of the issue.

It's a good theory. We'll see. For now I am testing with Spotlight disabled so I don't want to try multiple things at the same time. So please keep us posted on your tests

Remembering the old swap days in Gentoo Linux when and if configuration hibernation, the rule so to speak, was to have double the amount of RAM as swap. Or perhaps it was 1,5 the amount, don’t really remember. But for sleep I’ve never heard of that being mandatory.

Oh Gentoo. Yeah good old days. I loved and hated it Yeah sleep AFAIK sleep shouldn't dump memory stuff to disk. The standby keeps stuff in memory as it is, both physical and virtual memory, that's why I'm not 100% convinced yet.

 

[Fomalhaut]

Maybe it's Siri?

How so? What does Siri write to disk? I've never used it on a desktop Mac.

 

[VitoBotta]

I have Siri disabled too.