ssd swap - high usage of Terabytes Written

[zkap]

I never had a big problem with this like some of you, but I did notice a considerable drop in writing. It is now, by my probably inaccurate estimates, down to about a fifth of the writes I had before 11.3. I think Apple has or will solve this with updates.

 

[Baff]

@Baff No, as I said previously, I am running my machine on the Developer Seed from day one. So it has run all beta release published since the launch of M1 Macs. Currently running 11.4 B1.

Sorry, I missed that. That could very well be why you aren't seeing high writes. Others have noted significant decreases in writes with the lastest versions of MacOS.

 

[Baff]

No, you are wrong. All kinds of users have this issue (or not). The use case does not seem to matter. That's why nobody figured out the cause yet.

I would say that the use case absolutely matters, which is why we were able to find a work-around which immediately reduced the writes for most people.

 

[Pmaddocks86]

Thank you for the great post. I am still very frustrated by huge SSD writes when using Lightroom Classic on M1s. Especially the Develop module. And unfortunately there is not much help over in the Adobe forums. Can anyone here provide some tips as to how to better manage usage in Lightroom Classic until Adobe finally gives us an M1 version that does a better job managing swapping?

If you think Adobe will have an M1 version of Lightroom classic any time soon, you will be sorely disappointed.

They have let LRC get to such a bloated state that making it M1 native will take them at least the rest of this year. Meanwhile competitors like Capture One are launching their native version in May...

 

[ambient_light]

No, you are wrong. All kinds of users have this issue (or not). The use case does not seem to matter. That's why nobody figured out the cause yet.

It's not that the cause is not known here ... Quite a lot you can understand from the existing diagnostics (Activity Monitor, vm_stat, smartmontools...).

Specifically, Big Sure 11.0.x-11.2.x exposed irrational swapping behavior right after the reboot, when kernel_task was writing ca. 1TB / day or so even when there were 3-4Gb RAM free and memory pressure level around 25%. The only possibility here is that OS swapped out memory pages to disk when it's not needed, and likely with a 5-10 multiplier, i.e. swapping algorithm didn't work properly.

11.3 introduced more improvements: fixed swap use right after the startup (it's 0 for a much longer time), but the issue resurface after the swap use reach ca. 5Gb., so I was getting few hundreds Gbs written per day.

11.4beta1 works best for me so far. Apple tweaked the algorithm to suppress the swap activity, and average memory pressure is around 50% instead of 25% before, with moderate swapping activity (few tenth Gb/day).

Use case matters in a sense that certain activity (browsing with a lot of tabs) may provoke VM thrashing and random excessive swapping faster than other less memory-intensive workloads. But OS issue is always there, and processes are not culprits.

 

[CMMChris]

Sorry, I missed that. That could very well be why you aren't seeing high writes. Others have noted significant decreases in writes with the lastest versions of MacOS.

No, I never had significant writes. Again, this machine is my daily driver since November 2020 and I was having a very close look at my writes ever since this issue has been brought up.

I would say that the use case absolutely matters, which is why we were able to find a work-around which immediately reduced the writes for most people.

It might contribute in some cases but there have been reports from people who only did light browsing with their machine and have >50TBW on their drives, some cases only after a couple weeks of use. And then, again, there are the heavy users who don't have any issues at all like me.

We are going in circles with that discussion 😀

 

[rob984]

If you think Adobe will have an M1 version of Lightroom classic any time soon, you will be sorely disappointed.

They have let LRC get to such a bloated state that making it M1 native will take them at least the rest of this year. Meanwhile competitors like Capture One are launching their native version in May...

C1 Pro released beta with M1 support, just saw it today.
As for Adobe products, I have best results with Photoshop, it's using very small amount of swap or writing to disk.
Maybe it's because of the tweaks I did in settings (Basic GPU, and reduces amount of RAM available to PS), but it's running quite good with 24MP files from D750.

 

[Sherif Iskander]

So. I think I have found a discrepancy. I have an Intel MacBook Pro 15# 2018 and an M1 MacBook Pro. Both were just updated today to 11.3. So basically identical OS.

On the Intel based MBP the page size for VM is 4KB. On the M1 it is 16KB. So anything that pages small amounts would essentially be paging 4X on the M1. Could that be the culprit??

./sherif

 

[ambient_light]

So. I think I have found a discrepancy. I have an Intel MacBook Pro 15# 2018 and an M1 MacBook Pro. Both were just updated today to 11.3. So basically identical OS.

On the Intel based MBP the page size for VM is 4KB. On the M1 it is 16KB. So anything that pages small amounts would essentially be paging 4X on the M1. Could that be the culprit??

./sherif

Had the same thought, but that would be too easy Application processes don't influence paging directly, it's all up to OS kernel to decide, and certain optimizations were implemented for fragmented memory cases.

However, CPU arch differences might contribute to virtual memory management issues, including frequent but unnecessary paging / swapping out etc.

 

[IceStormNG]

So. I think I have found a discrepancy. I have an Intel MacBook Pro 15# 2018 and an M1 MacBook Pro. Both were just updated today to 11.3. So basically identical OS.

On the Intel based MBP the page size for VM is 4KB. On the M1 it is 16KB. So anything that pages small amounts would essentially be paging 4X on the M1. Could that be the culprit??

./sherif

View attachment 1765871 View attachment 1765872

That's simple: Apple Siliocon has a page size of 16kiB and x86_64 a size of 4kiB. So the swap page size is obviously of the same size as RAM page size of the respective system.

Large pages cause more page faults, waste more memory because of fragmentation. But they're easier on the TLB as there are less pages in total. So there are less TLB misses.

Both has pro and cons. Most systems use 4kiB of page size these days and some systems (like Linux) have the option to use large pages (4MiB). Very large pages are only useful for certain applications. For most situations, smaller (but not too small) sizes are optimal.

 

[vcsyc]

2 days ago I updated to 11.4. I have uptime: up 2 day, 10:44 hours. 95 GB written, including update 6,5GB.
so, before this I never left MacBook on charger when sleeping because it was writing like crazy. yesterday I left Mac sleeping at night, on charger and I think they fixed the bug when Mac was writing a lot to the ssd when sleeping on AC.

 

[wirtandi]

Could someone please explain this:

I am currently tracking my data written in activity monitor. Looking at smartmontools/drivedx, I have 319GB on the 26th April. Then on the 27th, I downloaded the software update 11.3 and I noticed I wrote lots of GB (maybe 20GB or so) to my "data written' in activity monitor. Logically, my TBW should be about 339GB.

However, when I check smartmontools/drivedx on the 28th, I have about 321GB, so only an increase of 2GB. How? Clearly my "data written" was about 20GB on the 27th of april. It should have been 319GB plus 20GB, right?

Or is it not that simple? I am confused :S

 

[stigman]

I can also report, I am now on 11.3. I have my usual 65+ Tabs Open. For the first time since I have owned this machine, Swap is at 0!!!

Could you take take some screenshot/video while your browsing and having 65+ tabs open?

 

[osplo]

Could someone please explain this:

I am currently tracking my data written in activity monitor. Looking at smartmontools/drivedx, I have 319GB on the 26th April. Then on the 27th, I downloaded the software update 11.3 and I noticed I wrote lots of GB (maybe 20GB or so) to my "data written' in activity monitor. Logically, my TBW should be about 339GB.

However, when I check smartmontools/drivedx on the 28th, I have about 321GB, so only an increase of 2GB. How? Clearly my "data written" was about 20GB on the 27th of april. It should have been 319GB plus 20GB, right?

Or is it not that simple? I am confused :S

I assume you did not have an external disk connected to the machine. If so, the 20 GB are shared between the internal SSD and every connected external disk, so the discrepancy can be caused by that.

 

[Kyros_G]

And remember you don't need to buy DriveDX just to check writes.

Some generous soul (nickname jdb8167) wrote this utility that does exactly that and works like a charm:

The release is here: https://github.com/jamesdbailey/smartTBW/releases/download/v0.1/smartTBW

(Thanks again James!)

Hello,

I tried to install/run this app but I'm getting a message that

"
“smartTBW” can’t be opened because the identity of the developer cannot be confirmed.
"

Any idea of how to overcome this?

 

[Ningj]

Hello,

I tried to install/run this app but I'm getting a message that

"
“smartTBW” can’t be opened because the identity of the developer cannot be confirmed.
"

Any idea of how to overcome this?

You need to open System Preferences > Security & Privacy > General and allow the app. Should be listed in the bottom half of the screen - open anyway or something to that affect

 

[jdb8167]

Hello,

I tried to install/run this app but I'm getting a message that

"
“smartTBW” can’t be opened because the identity of the developer cannot be confirmed.
"

Any idea of how to overcome this?

Download the latest installer package here and let me know if the problem persists. Without a second M1 Mac to test on and no available Big Sur VMs it is very difficult to confirm that my notarization is working.

 

[jdb8167]

You need to open System Preferences > Security & Privacy > General and allow the app. Should be listed in the bottom half of the screen - open anyway or something to that affect

You shouldn’t have to do that. I have now added a notarized installer package to avoid that problem.

 

[Maximara]

Hello,

I tried to install/run this app but I'm getting a message that

"
“smartTBW” can’t be opened because the identity of the developer cannot be confirmed.
"

Any idea of how to overcome this?

Yes. You left click on the app and a menu box should pop up. You select "Open" from that. The dialogue will now have an open option.

 

[wirtandi]

I assume you did not have an external disk connected to the machine. If so, the 20 GB are shared between the internal SSD and every connected external disk, so the discrepancy can be caused by that.

No I did not have any extrenal disk connected to the macbook.

Still looking for clarification on how my numbers add up

 

[k-hawinkler]

BS 11.3 has definitely reduced my TBW numbers even further.
They were excellent already before.
I think I can stop reading this thread.
Although that has been very helpful.

 

[leons]

BS 11.3 has definitely reduced my TBW numbers even further.
They were excellent already before.
I think I can stop reading this thread.
Although that has been very helpful.

Agreed, I am hanging in here just to help others now. What is your average GB written per day now?

 

[anna_fun]

Until now, I have been a shy "lurker".
Now, 11.3 has fixed it for me t0o. Thanks to all you guys for the wonderful help! 😍

 

[anna_fun]

P.S. I'm down to about 55GB/Day!

 

[leons]

This thread has now grown to 100 pages, and I feel we are on the cusp of the problem becoming an issue only for those few who may be using certain disk-intensive programs that may not yet be optimized for M1.
Many thanks to all who have contributed to this effort over time!. Special thanks to @TheSynchronizer!!!
I and others will be "hanging in" here to help new users and/or help deal with outliers/new issues.

I hereby "bless" the 100th page of this thread!
(This thread has more posts and views than any other in this forum, with the exception of "order status").