I have lost data many times due to power loss while IDE/SATA HDDs were being written back in Windows XP days. It’s why I always use a UPS now.Basically don't lose power on your desktop Mac if losing the last few seconds of writes to your internal SSD is crucial.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
SSD/Flash storage in Macs: Premium prices for garbage?!
- Thread starter k27
- Start date
- Sort by reaction score
There really is no excuse for lack of end-to-end integrity protection of files, or end-to-end ECC for memory, other than to be cheap.
Let’s be a bit realistic here. Where do you even find a system with the characteristics you describe? I would say Apple does better in this area than most.
It seems that Hector will keep studying the issue.
Fun fact: at one of the Virtual WWDC labs last year, I spoke to someone on the file system team. One of my questions was "Now that the root volume is sealed, using checksums, can this ability be extended to user data?" His response was "modern ssds don't suffer data corruption in any form". While this thread doesn't directly contradict that, I still feel checksums would help and would certainly provide some peace of mind. I do hope Apple allow checksums as an option going forward.
A lot of people in here jumping to conclusions...
Do other filesystems support checksums? Which ones?
I have read that Btrfs supports checksums, but it has some issues. What are the differences between Btrfs and APFS?
They do have checksums, just at the lower level. For them it is a reasonable design since they avoid additional overhead and complexity in the filesystem. This does make APFS an objectively worse choice for external non-Apple drives though. Which is a shame as it’s a great system.
Interestingly enough, they do checksum the metadata, but I can imagine that this is not so much because of potential SSD errors but rather to catch bugs in the FS itself. Corrupted metadata = entire drive lost, potentially.
All modern ”next-gen” file systems (Btrfs/ZFS) offer checksums. And all of them have problems, depending on what blog authors like more. I’ve read multiple “why I use btrfs instead of ZFS” and “why I use ZFS and btrfs sucks” pieces and I think most of it is a load of nonsense. Some points being argued are quite ridiculous. I’ve been running btrfs on my Synology, and it works fine.
An ECC RAM equipped PC running Linux with ZFS? There are not many laptops with ECC, but they do exist and you can run Linux on them.
Checksums within the SSD do not prevent errors in transmission, especially on external drives.
Why do you think ECC offers more in terms of end-to-end file integrity than APFS on Apple hardware? I would say they are pretty much comparable in this regard, only that Apple gives you consistent flush behaviour while with other hardware you are at the mercy of the hardware vendor.
Regarding Apple and ECC... we know very little about how their memory actually works. But I think if they had ECC they would have mentioned it.
I though the basic data transmission protocols (PCI-e/USB) already include transmission error detection and sometimes correction?
As said, ZFS and Btrfs would be the main two. NetApp's WAFL I think does too. The issues around Btrfs are to do with certain raid levels. I believe parity raid (5/6) are spicy on btrfs. That is to say they can and may eat your data. Other than that it's fine. NAS vendor's get around the raid problem by using traditional mdraid on Linux and putting a btrfs volume on top. It works fine. I prefer ZFS personally although some complain about memory use.
The differences between btrfs and apfs are numerous and difficult to explain due to Apple's secrecy on this. The main one that jumps out to me is apfs was designed with ssds in mind from the outset. I feel both zfs and btrfs came out before ssds became as ubiquitous. No built in raid for apfs. No checksums for user data. That could potentially change though. I do feel that given the huge number of apfs installs out there, Apple did a very good job with apfs. File systems are notoriously difficult and can take an eternity to be production ready. It's a huge topic, and one that I am most interested in out of all computing.
Secrecy? What secrecy? ?
I’m aware of that.
Better than nothing but with zfs and btrfs you can see the source. they haven’t even had a fs session at WWDC in a couple of years. Maybe this year!
The way how they tested it was nothing sort of genius. I am sure you know the story, but in case other forum posters are interested: before officially release APFS to the public Apple included a hidden beta version of it in the production iOS. The main file system would still be HFS+, but every operation was mirrored in the APFS backend and they stored two sets of metadata: HFS+ and APFS. The first was the "real" filesystem, the second was the shadow filesystem used to test drive APFS. This gave them access to millions of test platforms and tons of real, not artificial data to get comprehensive coverage and problem identification. Extremely impressive stuff.
Yeah, it was so so clever. It one of the reasons I wish they were a little more open. Fascinating stuff.
There are loads of ways that APFS is better than HFS+; example modern data recovery, optimizations for SSDs, etc. Would be best to do a quick search and find one of the many articles explaining the differences and benefits of APFS for modern systems (specifically, devices that have SSDs as their standard storage configuration).
HFS+ dates back to MacOS 8.1, and itself is built on top of HFS. So it's not really built for modern computing as it hasn't really changed much since 1998.
The big things in my mind are:
* Snapshots
* Copy on Write
* Containers instead of Partitions
A lot of this benefits from SSDs. Containers are a form of thin provisioning of volumes. So I don't have to cut a 2TB drive into two 1TB partitions, but rather I can let them share the free space on the drive and each container only uses what it needs. But this means a heavily fragmented drive in the end. Containers make the sealed system volume possible without overhead of resizing partitions/etc.
Copy on write will create more fragmented files, but means less wear or need for deduplication at the SSD controller level on the NAND flash, and makes more space on the drive available for the user. Copy on write also makes journaling easier. So while you can lose data that hasn't been fully written yet, odds are good that the metadata (and the volume) will remain in a good state so long as it's the last thing you do in any operation.
Snapshots are probably the least useful day to day, but back the sealed system volume functionality, since you are booting snapshots that are signed. A malicious actor wanting to modify the system volume has to recreate the seal which is extra work. If I recall correctly, Time Machine has benefited from snapshots as well. But it'd be a good way to snapshot the state you want to backup and not require exclusive access to files you are trying to backup.
My AMD based Linux OS which has ECC memory and a ZFS file system...