ssd swap - high usage of Terabytes Written

[IceStormNG]

Oh gosh, I surely hope Apple is silently investigating this. It is unfair that us customers have to do all the investigations when they are the one who should be informing us of the issue

Sorry to tell you: This has alway been the case. Companies tend to blame the customers unless they can't do it anymore. Apple is no different here. Look how long it took for them to acknowledge issues with the butterfly keyboards and how long they still implemented these.

Now we have people reporting it may be Big Sur related, not M1, because their Intel based machines are affected.

Yeah. My one also does. And it's not because I don't have enough RAM, but because macOS decides to just swap out gigabytes of memory and fill the other part with caches instead of just leave everything in RAM as there's tons of free memory available.

I've now disabled swap via vm.compressor. But the machine still swapped 20GB at some point for no reason. I also had to unload the dynamic_pager to fully disable swap.
Preemptive swap is fine to some extend. But the current implementation is a mess. I already noticed that behavior on Catalina. I didn't thought to much about it, because Catalina was a buggy mess anyway. Looks like Big Sur didn't changed that.

What I would really like to see is a way to disable preemptive swapping but still leave swapping on. So it only starts to swap if really needed and not because the OS is bored and Apple wants it to destroy the SSD.

 

[telo123]

For those who use any of the Chromium browsers that have VP9 hardware decoding enabled through #videotoolbox-vp9-decoding as an experimental flag enabled OR by default on Chromium 90 (Edge Dev, Chrome Beta, etc), notice that when 4k video is playing on YouTube, Utility Network Service writes to the SSD relatively quickly on M1 Macs?

I'm not overly concerned, but it is what I've noticed. I don't have an Intel Mac with me right now, so I don't know if it is exclusive to M1 or not.

Does anyone with technical knowledge know why it is doing so? Is it because of the VP9 decoding? I suspect VP9 decoding because it seems that when VP9 is blocked through h264ify, it writes much slower.

I've tested it on Brave and Chrome as an experimental feature and Edge Dev (enabled by default), all doing the same behaviour.

I attached a screenshot highlighting the video watching 3 minutes of the following video (4K HDR resolution) on Chrome:

 

[IceStormNG]

VP9 is just much larger than H.264. Mostly because H.264 caps out at 1080p on YouTube. Anything beyond that and anything in HDR uses VP9.

When streaming video, the Webbrowser actually downloads the video and plays it from there. The larger the Video file, the more it has to write.

 

[telo123]

VP9 is just much larger than H.264. Mostly because H.264 caps out at 1080p on YouTube. Anything beyond that and anything in HDR uses VP9.

When streaming video, the Webbrowser actually downloads the video and plays it from there. The larger the Video file, the more it has to write.

Ah, I understand now. When playing 4k on YouTube on Safari, though, it doesn't seem to write as much (correct me if I'm wrong). I hear that Safari forces YouTube to use H.264 (again, correct me if I'm wrong), which explains that then

 

[qap]

MBA 8/256 bought bought 22 Dec 2020

Today

Home@Giulio-MacBook-Air ~ % smartctl -a /dev/disk0 | egrep "Units|Hours"
Data Units Read:                    6,754,369 [3.45 TB]
Data Units Written:                 4,870,618 [2.49 TB]

5 March

Home@Giulio-MacBook-Air ~ % smartctl -a /dev/disk0 | egrep "Units|Hours"
Data Units Read:                    6,028,042 [3.08 TB]
Data Units Written:                 4,443,651 [2.27 TB]

It writes a lot (8/10GB per day) also if you leave the Mac in sleep mode with Safari or other apps opened, if you close all the apps before put it to sleep (or you shut down it), you will save some GBs.

Because the ARM Macs are not really in sleep, I see that it send packets to my router almost every 20/30secs.

PS: I'm lazy to do more research, someone found the command to disable the "auto power ON" when the Mac is OFF and you open the display?

Thanks!

 

[qap]

macOS has to write the whole RAM to disk when it hibernates. If you have 16GB of RAM, that's 16GB it has to write.

Okay that does make sense and accounts for atleast some of those writes, although I do have 8GB of ram and not 16.

As long as my writesGB / uptime < 3GB its a non issue anyway, as I expect the SSD to survive atleast 20 years at those rates, but its definitely an observation ill look into to see if it would spiral out of control if I did something else.

WTF?! No it doesn't work in this way... it's not an "hibernate" command, sleep mode keeps the RAM alive with few mW of power and it doesn't write the RAM to disk.

 

[Mike Boreham]

The SSD Endurance Experiment: They’re all dead puts this in prospective.

Your link above doesn't work for me, but I found it easily and reposting here as it is a good (and reassuring) read:

The SSD Experiment: They're all dead

 

[qap]

Your link above doesn't work for me, but I found it easily and reposting here as it is a good (and reassuring) read:

The SSD Experiment: They're all dead

10:22 AM, MARCH 12, 2015

 

[Jouls]

Thanks, I already tried the "easy way", but for some reason I can't get it to work after installation (terminal doesn't recognise launch commands)

I also had the problem, that the command was not found. Someone pointed out I should use the full path instead:

/usr/local/sbin/smartctl --all /dev/disk0

That worked for me.

 

[Mike Boreham]

10:22 AM, MARCH 12, 2015

Yes I realise it is quite old but I (maybe naively) assume the situation is not worse with newer SSDs.

Would be interested in links to more recent.

 

[Spudlicious]

macOS has to write the whole RAM to disk when it hibernates. If you have 16GB of RAM, that's 16GB it has to write.

If so, then that rather goes against the prevailing 'bigger RAM is better' wisdom.

Edit: A later post shoots down IceStormNG's point and my comment on it.

 

[osplo]

The SSD Experiment: They're all dead

WOW! Up to 2 petabytes written gracefully in 256GB SSDs that used five years old technology.

Latest Apple SSDs with today's tech should last even longer. And they issue all kind of warnings before total failure.

Thanks for the link. It seems that SSDs are tougher than we thought.

And in the worst case, if the internal SSD fails after a number of years, we can always boot from a (very small and light) external SSD with practically the same performance and probably much greater capacity by that time. That would keep our machines chugging along for a long, long time.

 

[IceStormNG]

WTF?! No it doesn't work in this way... it's not an "hibernate" command, sleep mode keeps the RAM alive with few mW of power and it doesn't write the RAM to disk.

It does. When you close the lid, your Machine goes to sleep, where RAM stays powered. It also writes the ram image to disk for the case when the battery runs out. That’s called safe sleep (hibernatemode 3) and is the default for Apple laptops since 2005.

 

[Maximara]

10:22 AM, MARCH 12, 2015

Yes, The SSD Endurance Experiment: They’re all dead article is that old but SSDs have improved since then. Also remember I used the 200 TB number for the calculations. So I am way low balling the data and still getting 4+ years with the numbers given.

 

[Maximara]

WOW! Up to 2 petabytes written gracefully in 256GB SSDs that used five years old technology.

Latest Apple SSDs with today's tech should last even longer. And they issue all kind of warnings before total failure.

Thanks for the link. It seems that SSDs are tougher than we thought.

And in the worst case, if the internal SSD fails after a number of years, we can always boot from a (very small and light) external SSD with practically the same performance and probably much greater capacity by that time. That would keep our machines chugging along for a long, long time.

It should be pointed out that some of the drive TBWs that can be calculated based on the TB and percentage are totally off the wall nuts.

Fomalhaut reported 170TB written with smartctl reporting 2% used. If we go with basic math where 2% x 50 = 100% then we get 8500 TBW (170TB x 50) for a 1TB drive which is insane. Even if we go with the idea that is really 2.499... one still gets a totally off the wall impossible 6827 (170*100/2.49) TBW for his 1TB drive.

Apple is likely doing something similar and looking at these numbers and logically deducing the tools being used are garbage. Doesn't mean they are ignoring the issue but they have a lot to sort through.

 

[qap]

It does. When you close the lid, your Machine goes to sleep, where RAM stays powered. It also writes the ram image to disk for the case when the battery runs out. That’s called safe sleep (hibernatemode 3) and is the default for Apple laptops since 2005.

hibernatemode 3 doesn't write all the ram to disk as you said

macOS has to write the whole RAM to disk when it hibernates. If you have 16GB of RAM, that's 16GB it has to write.

Only the real memory used, ie: if you have 16GB of physical memory, 4GB used and xGB of cached or compressed (and swapped obviously), it writes only the 4GB, not "the whole RAM".

And anyway on ARM Mac I'm not sure it works in this way, infact "grepping" the hibernatemode mode doesn't return anything on the MBA.

Home@Giulio-iMac ~ % pmset -g | grep hibernatemode
hibernatemode        3

Home@Giulio-MacBook-Air ~ % pmset -g | grep hibernatemode
Home@Giulio-MacBook-Air ~ % pmset | grep hibernatemode

 

[MacModMachine]

nov 28th - now

 

[Moka Akashiya]

And in the worst case, if the internal SSD fails after a number of years, we can always boot from a (very small and light) external SSD

But you cannot because of Apple's security chip, doesn't?

 

[Mike Boreham]

But you cannot because of Apple's security chip, doesn't?

No you can boot from an external, without reducing security. Many threads about this. Thunderbolt externals seems to work most consistently, USB externals work for some people some of the time.

Security chip will prevent booting from drives created on other computers. In other words someone can't steal your computer, plug a bootable external into and boot up from it and steal your data.

 

[Maximara]

nov 28th - now

View attachment 1744032

Nov 28 to Mar 15 = 107 days with 20.8 TB (1%). ~0.269 TB/day with a projected drive TBW of 2080 at the end of 10,700 days. That is 29.3 years. Explain to me again why everyone is freaking out?

 

[IceStormNG]

But you cannot because of Apple's security chip, doesn't?

You have to make sure to disable boot security before that happens. From my experience, you can only do that when the internal disk is working because you have to authorize the change using a macOS user account.

On a factory new machine, you cannot disable it unless you run the macOS setup and created a user account.

After that you can boot from external SSD.

 

[IceStormNG]

hibernatemode 3 doesn't write all the ram to disk as you said


Only the real memory used, ie: if you have 16GB of physical memory, 4GB used and xGB of cached or compressed (and swapped obviously), it writes only the 4GB, not "the whole RAM".

Yeah I know. I just pointed out the worst case. Maybe I should've written "up to".

BTW: I still found something out. When this happens (the machine goes to deep sleep), macOS also swaps out as much RAM as possible. Although I have swapping disabled, which works under regular use, the machine still heavily swaps during sleep and still creates swap files. I'm not talking about the sleepimage for the RAM backup.

My machine swapped 4GiB over night while in sleep. As I opened the lid today, there's still 2GiB swapped out, the other 2GiB got swapped in pretty quickly.

Could it be that macOS tries to reduce RAM usage before hibernate as much as possible?

Even though it might be an issue for the lifetime, but I really hate this lack of control.

 

[Mike Boreham]

You have to make sure to disable boot security before that happens. From my experience, you can only do that when the internal disk is working because you have to authorize the change using a macOS user account.

On a factory new machine, you cannot disable it unless you run the macOS setup and created a user account.

After that you can boot from external SSD.

Not true. see my post #1270 a few above. You do not need to change Startup Security to boot from an external. I have bootable Thunderbolt and USB externals and full security.

 

[Moka Akashiya]

You have to make sure to disable boot security before failure happens

Do you know how this can be disabled? Not sure this can be checked, but would be nice to know.
Article for reference, that has reverse statement:

M1 Macs radically change boot and recovery

Unlike Intel Macs, M1 models always start booting from internal SSD, and can only offer full Recovery from there too. This has benefits, and some drawbacks too.

eclecticlight.co

 

[Mike Boreham]

Do you know how this can be disabled? Not sure this can be checked, but would be nice to know.
Article for reference, that has reverse statement:

M1 Macs radically change boot and recovery

Unlike Intel Macs, M1 models always start booting from internal SSD, and can only offer full Recovery from there too. This has benefits, and some drawbacks too.

eclecticlight.co

Trust me you do not need to do this!

https://forums.macrumors.com/thread...red-to-install-and-boot-usb-external.2279421/

I can't see wherein that article Howard says need to reduce security.

I have done several installs on to externals with standard security.