ssd swap - high usage of Terabytes Written

[Maximara]

His screen shows 23% of life left, meaning that 77% is used, right? 77% because 100%(of maximum) - 23%(life left).
For this 77% used, he also consumer 1.3PB. Which means that the math is:
1.3PB = 77%
xPB = 100%.
77%*x=1.3*100
x=1.69PB
1.69 PB for 256GB SSD is ok as you said, cause you said projected 3.4 PB of my 512GB ssd is something huge and not reachable.
It looks like he is reaching the max level and need to sell the laptop or trade-in asap. Numbers don't lie here.

Sure the numbers can "lie". In fact, very early in this whole thing it came out that the tool being used was using an old reference file which was causing it to cough up nonsensical data.

Also you missed the point I was making: "The lifetime number is supposed to be what the SSD is warranted for not its total lifespan and there there is no 256GB SSD on the flipping planet what by the numbers has a 5.2 PB warranty."

"Attribute 202: Percentage Lifetime Used

This attribute is exactly as its name implies. It is a measure of how much of the drive’s projected lifetime has been used at any point in time. When the SSD is brand new, Attribute 202 will report “0”, and when its specified lifetime has been reached, it will show “100,” reporting that 100 percent of the lifetime has been used.

However, it’s important to realize what it means to reach 100 percent of projected lifetime – it does not mean that the drive is going to fail when that counter rolls over to 101 percent, only that your SSD may need to be replaced soon."

Take the 256GB Samsung 850 PRO used in the 2015 test for example.
Samsung 840 PRO was warranted for 5 years or 73 TB TBW depending on what came first and yet it lasted into the 2 PB range. A SMART tool should base the number on the 73 TB not on the 2 PB ie at 73 TBW it would show 100% used and 2,739.7% when it finally died. That is why the numbers are nonsense - no one in the freaking world with a brain in their head would set Attribute 202 to 5.2 PB

 

[Maximara]

Just ran SmartMonTool and my 16" M1Pro, 16GB/1TB has written 1.94TB for around a month of use.

How much does streaming Apple Music and videos write to the SSD?

None in my case because I moved both iTunes and AppleTV storage to my external platter drive.

 

[Maximara]

If you have Apple Care or in your country consumer laws establish 2 or 3 years mandatory warranty (here in Spain, three) then continue using the machine until it breaks, then get a new disk from Apple, then sell it

This thread has been very quiet lately so I assume that the problem is mostly gone for everybody, at least since the latest Monterey version (12.3). It was never a problem for me, BTW.

Having said that your "normal business software" seem not to be so normal. Any suspicious app? I would be inclined to think that one of these is forcing the machine to swap like crazy or it is just plain writing to the disk directly like there is no tomorrow.

I agree as this is really bizarre at it is the first report (I know of) of somebody reaching 1PB and if the program is working correctly that just should not happen unless it need more than what is left with the System doing its thing. Unless one is saving insanely big files that shouldn't move the number much either.

 

[jdb8167]

Sure the numbers can "lie". In fact, very early in this whole thing it came out that the tool being used was using an old reference file which was causing it to cough up nonsensical data.

That itself is a lie. You know it’s not true. No SMART tools are using a “reference file”, whatever that is.

 

[Maximara]

That itself is a lie. You know it’s not true. No SMART tools are using a “reference file”, whatever that is.

Not a "lie". From Mac Hard Drive Testing Tools and SMART Analysis:

"SMART testing is of limited value in many cases, and cannot often predict failure of a hard drive. This is because different hard drive manufacturers implement different parts of the spec and (as you mentioned) different software and OS manufacturers look at different pieces of the spec in interpreting the SMART status."

"SMART analysis is a REPORTING technology, not a testing technology. Tools like Scannerz, TechTool Pro, etc. are TESTING tools. There are some implementations of SMART that can perform testing on a drive, but this is vendor specific. SMART implementation also varies from vendor to vendor and the implementation isn't consistent. (...) The problem that I see with SMART testing is that it can report a problem which isn't lethal, but some SMART tools will portray the test results in a manner that, for lack of better words, "scares" a user into trashing what might be a drive with correctable problems."

Elsewhere there is this:

"Samsung isn't quoting any specific TB written values for how long it expects the EVO to last, although the drive comes with a 3 year warranty. Samsung doesn't explicitly expose total NAND writes in its SMART details but we do get a wear level indicator (SMART attribute 177). The wear level indicator starts at 100 and decreases linearly down to 1 from what I can tell. At 1 the drive will have exceeded all of its rated p/e cycles, but in reality the drive's total endurance can significantly exceed that value." - Samsung SSD 840 EVO Review: 120GB, 250GB, 500GB, 750GB & 1TB Models Tested

So Samsung uses attribute 177 rather than attribute 202 (Percentage Lifetime Used) ie a different "reference". So if your SMART tool is looking at the wrong "reference" (ie Attribute) without making the proper calculations its going to produce nonsense.

 

[jdb8167]

Not a "lie". From Mac Hard Drive Testing Tools and SMART Analysis:

"SMART testing is of limited value in many cases, and cannot often predict failure of a hard drive. This is because different hard drive manufacturers implement different parts of the spec and (as you mentioned) different software and OS manufacturers look at different pieces of the spec in interpreting the SMART status."

"SMART analysis is a REPORTING technology, not a testing technology. Tools like Scannerz, TechTool Pro, etc. are TESTING tools. There are some implementations of SMART that can perform testing on a drive, but this is vendor specific. SMART implementation also varies from vendor to vendor and the implementation isn't consistent. (...) The problem that I see with SMART testing is that it can report a problem which isn't lethal, but some SMART tools will portray the test results in a manner that, for lack of better words, "scares" a user into trashing what might be a drive with correctable problems."

Elsewhere there is this:

"Samsung isn't quoting any specific TB written values for how long it expects the EVO to last, although the drive comes with a 3 year warranty. Samsung doesn't explicitly expose total NAND writes in its SMART details but we do get a wear level indicator (SMART attribute 177). The wear level indicator starts at 100 and decreases linearly down to 1 from what I can tell. At 1 the drive will have exceeded all of its rated p/e cycles, but in reality the drive's total endurance can significantly exceed that value." - Samsung SSD 840 EVO Review: 120GB, 250GB, 500GB, 750GB & 1TB Models Tested

So Samsung uses attribute 177 rather than attribute 202 (Percentage Lifetime Used) ie a different "reference". So if your SMART tool is looking at the wrong "reference" (ie Attribute) without making the proper calculations its going to produce nonsense.

That “article” is over 9 years old. I thought we were talking about the SSD in Apple silicon Macs. None of the points you highlight are remotely relevant.

 

[Maximara]

That “article” is over 9 years old. I thought we were talking about the SSD in Apple silicon Macs. None of the points you highlight are remotely relevant.

The 2015 test which showed us actual TBW is way beyond the warranted value is 7 years old and Samsung still sales the 840 Pro drive used in that test. Your point?

 

[jdb8167]

The 2015 test which showed us actual TBW is way beyond the warranted value is 7 years old and Samsung still sales the 840 Pro drive used in that test. Your point?

My point is that you keep repeating the same tired arguments after they have been shown to be complete nonsense in regards to Apple silicon Macs. There was never any issue with the various tools used by people troubleshooting the issues that this thread is discussing.

 

[Fishrrman]

A very general question (if anyone cares to answer).

What "tools" do I want to use to obtain total amount of data written to the SSD (as listed in previous postings in this thread)?

I've had a 2021 MBP 14" for a couple of months now, and want to see what "the stats" are...

 

[jdb8167]

A very general question (if anyone cares to answer).

What "tools" do I want to use to obtain total amount of data written to the SSD (as listed in previous postings in this thread)?

I've had a 2021 MBP 14" for a couple of months now, and want to see what "the stats" are...

Take a look at the first post in this thread.

 

[Maximara]

My point is that you keep repeating the same tired arguments after they have been shown to be complete nonsense in regards to Apple silicon Macs. There was never any issue with the various tools used by people troubleshooting the issues that this thread is discussing.

Yes there was an issue with the tools - it was one of the first things posted out (by jdb8167 on Feb 24, 2021):

"How has it been proven? I haven't seen anything definitive. I've looked at the smartmontools code and even their code documentation says they are guessing and they only have two specific drives in their database neither of which matches the NVMe SSD in the M1 Macs."

The numbers just don't make sense.

 

[harshw]

Sure the numbers can "lie". In fact, very early in this whole thing it came out that the tool being used was using an old reference file which was causing it to cough up nonsensical data.

Honestly - I don't know whether you are trolling, trying to elicit a reaction or you have little or no understanding of modern storage systems. I am assuming it is the latter.

That is why the numbers are nonsense - no one in the freaking world with a brain in their head would set Attribute 202 to 5.2 PB

Those numbers are reported by the Apple NVME controller. The NAND Flash can be from various companies but the NVME controller, firmware etc are 100% Apple. Ergo, the SMART info returned is directly from Apple. Are you saying Apple has no brain in their heads? Or Apple puts nonsense numbers into the controller? Or are you saying that Apple is deliberately lying and changing SMART to not return the correct info?

 

[jdb8167]

Yes there was an issue with the tools - it was one of the first things posted out. The numbers just don't make sense.

From someone with way more knowledge about M1 internals than either of us.

https://twitter.com/i/web/status/1396450298244132864

There never was an issue with smartmontools.

 

[Maximara]

Honestly - I don't know whether you are trolling, trying to elicit a reaction or you have little or no understanding of modern storage systems. I am assuming it is the latter.

Took me a while but here is the relevant post where my statements are coming from (Post is from Mar 9, 2021):

Is this accurate? (Skip to 6:55)

He says that SMART % is an assumed number based on typical drives with much lower ratings, so the % is artificially high.

Is this also correct? Toshiba Nand Flash drives used in Macs are rated for complete drive write per day. That would mean if you have a 256 GB M1 Mac, you do:

256 x 365 = 93.4 TB x 8 years (I'm not quite sure where he got 8 from in the video, though) = ~750 TB as your TBW.

iCaveDave makes the statement at 8:48.

What I found when I checked my M1 MacBook Air SSD similarly states "Don't pay much attention to SSD lifetime specs. They're set to reduce vendor warranty cost risk, not estimate actual SSD life."

Some other material that came out a while ago:

SSDs and SMART Data goes into the details of how things are supposed to work. 11 Myths About SMART Monitoring and SSD Data Protection shows a serious issue with this whole thing:

2. Standard SMART utilities like Smartmontools and CrystalDiskInfo adequately manage SSDs.
While those utilities may be adequate for the casual user, most such utilities only extract attribute data and don’t sufficiently parse or interpret the data. In other words, they don’t necessarily provide immediately usable data.

Joelist back then stated: "There is logic here. This anomaly has been seen on only a small number of units and interestingly in both M1 and Intel Macs. Maybe what is going on is since both use Apple custom designed controllers the SMART attribute numbering is not what smartmon is expecting? The Crucial article is clear that there is no uniformity among manufacturers about attribute codes."

M1 Mac SSD Swap Issues Explained: Should you be WORRIED? (7:11) points out that some people back then were pointing to a bug in the reporting tools. This is what appears on screen when he brings this up:

* The reporting software - based on SMART monitoring from hard drives - is mischaracterizing the SSD's activity. SMART was a good idea haphazardly implemented across many but not all drive's firmware.

* SSD controller firmware is misreporting write activity. SSD controller firmware is complex due to the limitations of NAND flash, Mitigating write amplification requires careful juggling of DRAM butfers in the controller to ensure that only necessary data gets written to flash.

"Each health indicator (S.M.A.R.T. attribute) has a raw value aka raw data. Raw measured values (provided by a sensor or a counter) are stored in this field. Sometimes different parts (high word, low word, etc) of raw value contain different kinds of information.

The drive manufacturer defines the meaning of this value (but often corresponds to counts or a physical unit). The exact meaning of raw value often considered a trade secret. These values may significantly vary between different manufacturers and drive models and should NOT be compared with other devices or other vendors."

This is from DriveDX. Take from it what you will.

That is what I am basing my comments on.

 

[jdb8167]

@Joelist back then stated: "There is logic here. This anomaly has been seen on only a small number of units and interestingly in both M1 and Intel Macs. Maybe what is going on is since both use Apple custom designed controllers the SMART attribute numbering is not what smartmon is expecting?

Conclusively proven to not be the case. smartmontools uses the same buffer layout for the data structure as Apple’s own API does. Using just Apple’s API and comparing the results between both tools shows that they produce the same data.

You steadfastly refuse to believe that all this reporting data is coming directly from Apple’s controllers and software. If you had the ability to read the open source code you would have no more doubt. As you clearly don’t have that skill, you’ll have to take others word for it. Or you can continue to spread misinformation. Your choice.

 

[harshw]

2. Standard SMART utilities like Smartmontools and CrystalDiskInfo adequately manage SSDs.
While those utilities may be adequate for the casual user, most such utilities only extract attribute data and don’t sufficiently parse or interpret the data. In other words, they don’t necessarily provide immediately usable data.

From your previous posts I am of the opinion that you're not a developer hence you are relying on (obsolete) articles and cherry picking with confirmation bias. I am a developer and I have compiled @jdb8167's SmartTBW package and verified that the numbers line up with using smartctl. There are many people on the forum and in this thread who have written and are using their own utilities to interface directly with Apple's IOKit - no smartmontools involved whatsoever

If you are not trolling and interested in the truth - and you know anyone with knowledge of Objective-C and MacOS programming - you can ask them to take a look at the nvme.h file used. The SmartTBW author is not interpreting anything. They are only calling the SMARTReadData function supplied by Apple itself. (ref: https://developer.apple.com/documen...interface/3521172-smartreaddata?language=objc)

The percentage used field? It comes from Apple itself

So when you're claiming (without proof) that smartmontools and 'other utilities' are changing and interpreting results - it is you who is mistaken. There's source code for those utilities so it's open for everyone to see

EDIT: corrected SmartTBW's author

 

[jdb8167]

I am a developer and I have compiled @Forti 's SmartTBW package and verified that the numbers line up with using smartctl.

Just a slight correction, the package is mine. I posted on the first message of this thread because it is a wiki thread and anyone can update it.

But you are the first person who has stated that they looked at the code and built it. So thanks for that. 👍

 

[harshw]

Just a slight correction, the package is mine. I posted on the first message of this thread because it is a wiki thread and anyone can update it.

I corrected my post to have the proper username

 

[robfoll]

A very general question (if anyone cares to answer).

What "tools" do I want to use to obtain total amount of data written to the SSD (as listed in previous postings in this thread)?

I've had a 2021 MBP 14" for a couple of months now, and want to see what "the stats" are...

DriveDX.app

 

[Maximara]

DriveDX.app

DriveDX is effectively the open source smartmontools with a more friendly user interface. Stats is other open source tool which can help find what may be the problem though it ties into the activity monitor and so doesn't show activity between reboots.

 

[Spudlicious]

The issue here is not the data: this software pulls out original data of Apple.
Issue is that Apple will replace your battery if you are under 80% of health. But Apple never stated anything similar about SSDs. So here is the mercy of the store manager, otherwise there is no rules of Apple stating that such heavy write is something faulty. For example, ssd drive failing after so many PB and if the laptop is still under warranty or AC+, then they will replace it. If the product is alive then there is nothing much to talk about.

Just so, if the product is alive then there is nothing much to talk about. Similarly, I would rather have a car that's running well with 230,000 miles on the clock than a factory-fresh car that's dead at the roadside. It is possible to find the devil in irrelevant detail.

 

[leman]

How about 1.3PB! written in just over a year. SSD on 23% of lifetime left. MBP M1 8GB memory / 256GB HD. Clean machine and normal business software. Applestore Doncaster Melbourne refuses to take seriously "3rd Party Software" giving this info. We don't do preventative maintenance.

This doesn't make any sense. That kind of usage in a year means that you are writing 120GB per hour every single day and night 24/7, without ever stopping. That 3TB per day. I can assure you that the computer is not overwriting it's full SSD ten times over every single day for an entire year. Unless someone started an infinite write benchmark and forgot to turn it off. For a year.

And besides, as others already said, that kind of endurance is not possible with current SSD technology. Apple SSDs are very good in this regard — much better than the usual consumer drives — but they don't give you PB-levels of endurance for a 256GB SSD.

 

[Maximara]

This doesn't make any sense. That kind of usage in a year means that you are writing 120GB per hour every single day and night 24/7, without ever stopping. That 3TB per day. I can assure you that the computer is not overwriting it's full SSD ten times over every single day for an entire year. Unless someone started an infinite write benchmark and forgot to turn it off. For a year.

And besides, as others already said, that kind of endurance is not possible with current SSD technology. Apple SSDs are very good in this regard — much better than the usual consumer drives — but they don't give you PB-levels of endurance for a 256GB SSD.

"that kind of endurance is not possible with current SSD technology" is what I have been saying from the point I could back figure using the percentages. The numbers Smartmontools (and related stuff) is putting out just doesn't make any degree of sense as the numbers supposed to be based on the warranted value. What sane company would give any 256 GB SSD a warranty in the PB range?!

M1 Mac SSD Swap Issues Explained: Should you be WORRIED? (7:11)'s SSD controller firmware is misreporting write activity idea fits Occam's Razor as we have a version Smartmontools that uses Apple's own hooks and its producing nonsense. Or as someone in another forum suggested the hooks in the OS are buggy (which given the the "quality" of the recent Safari/OS updates seems likely)

floatingatoll on Feb 23, 2021 in Hacker News stated something along the same thing:

"It seems like there are a lot of assumptions in play in this "headline news":
(1) assuming that the reported numbers are valid for this measurement at all
(2) that they are not a bug in the IOKit implementation used by smartctl
(3) that they are not not a bug in smartctl itself
(4) that they are directly comparable to non-M1 hardware without further processing
(5) that they do not increment when Time Machine or other filesystem snapshots occur
(6) that they do not increment when APFS copy-on-writes occur
(7) that they do not include the total byte size of sparsely-modified files
I don't see anyone checking these assumptions yet, but if y'all do, supporting links on those points would improve this HN post considerably. There are other assumptions that could be tested too! Outrage is cool, but science is productive."

Around the same time wccftech had this to say: "It’s not confirmed if there’s an error in the software reporting the total amount of bytes written, or if macOS is unintentionally causing this issue."

A post from 2021 had this: "Percentage Used: Contains a vendor specific estimate of the percentage of NVM subsystem life used based on the actual usage and the manufacturer’s prediction of NVM life." Apple doesn't manufacture the SSDs (I can't recall if anybody figured out who did). Again what sane company would predict a life in the PB range for a 256 GB SSD?!

Even if the numbers are correct in some way there is clearly something way wrong but given the number other have presented - likely some third party software that if it is doing this must have been written while on a a gallon of Jolt cola.

'The speed at which a program is completed is directly proportional to the amount of Jolt cola consumed. Sadly the bugs in said program is also directly proportional to the amount of Jolt cola consumed.'

 

[Mike Boreham]

That 3TB per day.

My M1 MBA apparently managed 28 TB in 31 hours in this post, from Drive DX supported by iStat disk write trace. This is more than 3TB per day, but 'only' lasted 31 hours.

My complete history on this machine shows that this was (so far) a one off event, quite untypical of before or after behaviour:

 

[leman]

My M1 MBA apparently managed 28 TB in 31 hours in this post, from Drive DX supported by iStat disk write trace. This is more than 3TB per day, but 'only' lasted 31 hours.

My complete history on this machine shows that this was (so far) a one off event, quite untypical of before or after behaviour:

View attachment 1987255

That also doesn't make any sense. Almost a TB writes per hour? How would that work?