ssd swap - high usage of Terabytes Written

[Ningj]

While looking into an app issue and poking around in "Console > Mac Analytics Data" I stumbled on Apple sourced SMART statistics confirming data reported by other utils (most of us knew this already). Interestingly there are two percentage used stats. My drive is still on 0% and aren't sure if these report the same values for systems with significant writes (Big Sur 11.4).

If you are interested, launch Console, click on "Mac Analytics Data" and filter on com.apple.massstorage.systemcounters_v2_1

There are 8 different SSD statistics messages ...v2_1 - ...v2_8

Look for the Smart block in the statistic's summary:

...
com.apple.message.smart_available_spare: 100
com.apple.message.smart_available_spare_threshold: 99
com.apple.message.smart_critical_warnings: 0
com.apple.message.smart_data_units_read: 16505470
com.apple.message.smart_data_units_written: 12577264
com.apple.message.smart_host_read_commands: 321740324
com.apple.message.smart_host_write_commands: 305557069
com.apple.message.smart_media_errors: 0
com.apple.message.smart_percentage_used: 0
com.apple.message.smart_percentage_used_normalized: 0
com.apple.message.smart_power_cycles: 338
com.apple.message.smart_power_on_hours: 207
com.apple.message.smart_temperature: 298
com.apple.message.smart_unsafe_shutdowns: 24
...

 

[leons]

While looking into an app issue and poking around in "Console > Mac Analytics Data" I stumbled on Apple sourced SMART statistics confirming data reported by other utils (most of us knew this already). Interestingly there are two percentage used stats. My drive is still on 0% and aren't sure if these report the same values for systems with significant writes (Big Sur 11.4).

If you are interested, launch Console, click on "Mac Analytics Data" and filter on com.apple.massstorage.systemcounters_v2_1

There are 8 different SSD statistics messages ...v2_1 - ...v2_8

Look for the Smart block in the statistic's summary:

...
com.apple.message.smart_available_spare: 100
com.apple.message.smart_available_spare_threshold: 99
com.apple.message.smart_critical_warnings: 0
com.apple.message.smart_data_units_read: 16505470
com.apple.message.smart_data_units_written: 12577264
com.apple.message.smart_host_read_commands: 321740324
com.apple.message.smart_host_write_commands: 305557069
com.apple.message.smart_media_errors: 0
com.apple.message.smart_percentage_used: 0
com.apple.message.smart_percentage_used_normalized: 0
com.apple.message.smart_power_cycles: 338
com.apple.message.smart_power_on_hours: 207
com.apple.message.smart_temperature: 298
com.apple.message.smart_unsafe_shutdowns: 24
...

Good find! This verifies (what others have doubted) that Apple validates the data that we have found elsewhere. My "percentage used" stats are 1% for both. As you, I would be interested to see if anyone with a high percentage used sees any diddernce in the 2 numbers.

 

[sneeks]

I checked the console and this is the data I have. Terminal shows data units written as 840gb.

com.apple.message.smart_available_spare: 100
com.apple.message.smart_available_spare_threshold: 99
com.apple.message.smart_critical_warnings: 0
com.apple.message.smart_data_units_read: 2004376
com.apple.message.smart_data_units_written: 1545967
com.apple.message.smart_host_read_commands: 25922064
com.apple.message.smart_host_write_commands: 13369811
com.apple.message.smart_media_errors: 0
com.apple.message.smart_percentage_used: 0
com.apple.message.smart_percentage_used_normalized: 0
com.apple.message.smart_power_cycles: 141
com.apple.message.smart_power_on_hours: 11
com.apple.message.smart_temperature: 296
com.apple.message.smart_unsafe_shutdowns: 12

 

[jdb8167]

While looking into an app issue and poking around in "Console > Mac Analytics Data" I stumbled on Apple sourced SMART statistics confirming data reported by other utils (most of us knew this already). Interestingly there are two percentage used stats. My drive is still on 0% and aren't sure if these report the same values for systems with significant writes (Big Sur 11.4).

If you are interested, launch Console, click on "Mac Analytics Data" and filter on com.apple.massstorage.systemcounters_v2_1

There are 8 different SSD statistics messages ...v2_1 - ...v2_8

Look for the Smart block in the statistic's summary:

...
com.apple.message.smart_available_spare: 100
com.apple.message.smart_available_spare_threshold: 99
com.apple.message.smart_critical_warnings: 0
com.apple.message.smart_data_units_read: 16505470
com.apple.message.smart_data_units_written: 12577264
com.apple.message.smart_host_read_commands: 321740324
com.apple.message.smart_host_write_commands: 305557069
com.apple.message.smart_media_errors: 0
com.apple.message.smart_percentage_used: 0
com.apple.message.smart_percentage_used_normalized: 0
com.apple.message.smart_power_cycles: 338
com.apple.message.smart_power_on_hours: 207
com.apple.message.smart_temperature: 298
com.apple.message.smart_unsafe_shutdowns: 24
...

What an awesome find. Thanks for posting this. We finally have an Apple endorsed reading of the SMART data. Not that there was any doubt in my mind anyway but it is great to have ultimate confirmation.

 

[helloapple1]

What an awesome find. Thanks for posting this. We finally have an Apple endorsed reading of the SMART data. Not that there was any doubt in my mind anyway but it is great to have ultimate confirmation.

In layman’s terms what does it mean?

 

[jdb8167]

In layman’s terms what does it mean?

It is direct confirmation from Apple developed software that the third-party SMART monitoring tools are producing accurate results. You can look at the Apple log and compare Data Units Written and Percentage Used to see that they match the third-party tools.

 

[TheSynchronizer]

So we can now with pretty much full certainty say that these 256GB SSDs are rated for 1200-1600 TBW by taking the official Apple percentage used value.

I think it’s safe to say the majority of people who never even noticed the SSD write issue will not have their mac’s SSD die in them during the lifetime of their mac.

And now with 11.4 out, the issue largely resolved, I think it’s pretty much safe to say the issue is no more.

 

[k-hawinkler]

So we can now with pretty much full certainty say that these 256GB SSDs are rated for 1200-1600 TBW by taking the official Apple percentage used value.

I think it’s safe to say the majority of people who never even noticed the SSD write issue will not have their mac’s SSD die in them during the lifetime of their mac.

And now with 11.4 out, the issue largely resolved, I think it’s pretty much safe to say the issue is no more.

So if I take for a 256 GB (I.e. ¼ TB) SSD your above value of 1200 TBW, then that implies 4 * 1200 = 4800 writes per cell. And your above value of 1600 TBW would imply 4 * 1600 = 6400 writes per SSD storage cell. YEAH!

Previously I quoted the Toshiba number of 3000 writes per SSD storage cell.
Well, this is even much better.

Problem truly laid to rest for the vast majority of M1 users. Amen.

 

[Fomalhaut]

I wonder if Apple has taken the swap usage optimization too far in the other direction....

After installing MacOS 11.4, and running the same workloads, my memory usage now looks like this:

I have never seen my memory pressure this high. Previously it was always green, and "swap used" would increase to 8-10GB for typical moderately heavy usage.

This is probably a more accurate representation of actual memory usage, after all. I have used almost the RAM for apps and have the rest is used for cache, so this looks correct.

Disk writes are noticeably lower. After a full work day, it looks like this:

Previously I would expect about double this for similar usage.

 

[telo123]

I wonder if Apple has taken the swap usage optimization too far in the other direction....

After installing MacOS 11.4, and running the same workloads, my memory usage now looks like this:

View attachment 1784398
I have never seen my memory pressure this high. Previously it was always green, and "swap used" would increase to 8-10GB for typical moderately heavy usage.

This is probably a more accurate representation of actual memory usage, after all. I have used almost the RAM for apps and have the rest is used for cache, so this looks correct.

Disk writes are noticeably lower. After a full work day, it looks like this:

View attachment 1784395

Previously I would expect about double this for similar usage.

As long as it’s not red for a long period of time and you don’t notice any performance hiccup, I think it’s fine.

 

[sneeks]

I think that looks ok and will be the most efficient way for things to run, better it makes use of RAM before any other method.

 

[osplo]

I think that looks ok and will be the most efficient way for things to run, better it makes use of RAM before any other method.

Agree. If the system has available RAM it should use it. Nothing to worry, on the contrary.

I am happy to see that we can declare this whole thing solved.

 

[IceStormNG]

I wonder if Apple has taken the swap usage optimization too far in the other direction....

After installing MacOS 11.4, and running the same workloads, my memory usage now looks like this:

View attachment 1784398
I have never seen my memory pressure this high. Previously it was always green, and "swap used" would increase to 8-10GB for typical moderately heavy usage.

This is probably a more accurate representation of actual memory usage, after all. I have used almost the RAM for apps and have the rest is used for cache, so this looks correct.

Disk writes are noticeably lower. After a full work day, it looks like this:

View attachment 1784395

Previously I would expect about double this for similar usage.

What you see there is simply your mac running low on memory. This would also happen on any other machine if you run too many tasks for the amount of RAM you have.

The highly aggressive swap was probably an idea to overcome the RAM limitation of the M1, but turned out to cause other issues. Or it was simply just a bug nobody found during their "testing".

Either way, your mac is running low on memory. Half of your RAM is already compressed. 16GB is still just 16GB. M1 cannot do magic.
You either need more RAM (which is not possible, in this case) or close some applications when you work with a few very memory heavy applications.

 

[Mike Boreham]

While looking into an app issue and poking around in "Console > Mac Analytics Data" I stumbled on Apple sourced SMART statistics confirming data reported by other utils (most of us knew this already). Interestingly there are two percentage used stats. My drive is still on 0% and aren't sure if these report the same values for systems with significant writes (Big Sur 11.4).

If you are interested, launch Console, click on "Mac Analytics Data" and filter on com.apple.massstorage.systemcounters_v2_1

There are 8 different SSD statistics messages ...v2_1 - ...v2_8

Look for the Smart block in the statistic's summary:

...
com.apple.message.smart_available_spare: 100
com.apple.message.smart_available_spare_threshold: 99
com.apple.message.smart_critical_warnings: 0
com.apple.message.smart_data_units_read: 16505470
com.apple.message.smart_data_units_written: 12577264
com.apple.message.smart_host_read_commands: 321740324
com.apple.message.smart_host_write_commands: 305557069
com.apple.message.smart_media_errors: 0
com.apple.message.smart_percentage_used: 0
com.apple.message.smart_percentage_used_normalized: 0
com.apple.message.smart_power_cycles: 338
com.apple.message.smart_power_on_hours: 207
com.apple.message.smart_temperature: 298
com.apple.message.smart_unsafe_shutdowns: 24
...

Thanks! I need help using Console please!

When I open Console > Mac Analytics Data there are 1024 messages.
When I cut and paste com.apple.massstorage.systemcounters_v2_1 into the search box they all disappear.
I have tried searching on "smart" "percentage" "systemcounters" "massstorage" and a few other combos with same result.

Thanks for any help.

 

[Asclepias]

Try applessdstatistics

 

[Mike Boreham]

Try applessdstatistics

Thanks but same problem.

 

[TheSynchronizer]

The highly aggressive swap was probably an idea to overcome the RAM limitation of the M1, but turned out to cause other issues. Or it was simply just a bug nobody found during their "testing".

In my opinion it was most definitely a bug - some of it being a deep rooted kernel bug as noticed by that person on twitter porting linux to M1. I’ve found performance to be completed unchanged on my M1 with 11.4 despite the memory being in yellow memory pressure more often, and in my situation and workflow 8GB on this M1 still certainly performs better than 8GB did on the intel macbook I had in the past.

But I agree that it’s not magic and 8GB on M1 is not 16GB. It entirely depends on what one does on the machine and there are simply many more situations that would’ve required more than 8GB on an intel mac to do without freezing and spinning beach balls, which work great on the 8GB M1. But likewise there are other workflows that remain like RAW photo / intensive video editing where the physical space of memory that the 8GB M1 provides is simply not enough, just as it wouldn’t be enough on an intel.

 

[Mike Boreham]

Thanks but same problem.

Just tried again. This time Mac Analytics Data had 32,782 messages (just 1024 last time) and searching with com.apple.massstorage.systemcounters_v2_1 found the smart block in the applessdstatistics info pane:

Thanks for the help.

 

[Mike Boreham]

Good find! This verifies (what others have doubted) that Apple validates the data that we have found elsewhere. My "percentage used" stats are 1% for both. As you, I would be interested to see if anyone with a high percentage used sees any diddernce in the 2 numbers.

As you say it would be very interesting to know what "percentage used" values show for some of the really high users who have posted here.

What is your TBW that gives you the 1% number?

My usage numbers are 0% with 21.73 TBW to a 512GB internal new in November 2020.

 

[mikeboss]

https://twitter.com/i/web/status/1399283505016754181

 

[LowAffinity]

This is a stupid question but is it normal for kernel_task to write to the SSD while no swap file is in use, and the system is just sitting idle?

 

[leons]

This is a stupid question but is it normal for kernel_task to write to the SSD while no swap file is in use, and the system is just sitting idle?

NOT a stupid question. Yes, there are many system processes that write in the background, even when idle. Not a concern unless excessive. Welcome to the forum!

 

[telo123]

I truly wonder what the reactions of any Apple engineer were if and when they read this thread lol

 

[LowAffinity]

NOT a stupid question. Yes, there are many system processes that write in the background, even when idle. Not a concern unless excessive. Welcome to the forum!

Thank you for clearing that up😅

Edit: I read somewhere that using an external monitor with the M1 air/pro results in higher SSD writes. Has this been fixed, or was it never a thing in the first place?

 

[leons]

Thank you for clearing that up😅

Edit: I read somewhere that using an external monitor with the M1 air/pro results in higher SSD writes. Has this been fixed, or was it never a thing in the first place?

Never a significant issue.