Risks associated with having used exFAT for valuable data in the past?

[harrisonjr98]

Just to clarify from the jump - I am no longer using exFAT.

I am novice "hoarder" - hardly deserve that title compared to many here. I actually had most of my collection in the cloud up until the past few years as I became increasingly aware that I was losing control of my data.

When Google Play Music shut down, that was the trigger - I took my 50k or so uploaded songs via Takeout and also grabbed my entire Google Photos library. Put it on a drive for more control down the road.

The thing is, I formatted the drive as exFAT because I switched between mac and PC regularly at the time. I am now firmly planted on an M1 Mac, and have come to read some of the many horror stories about exFAT blowing itself up.

I stored some of my most prized info on an exFAT formatted drive for a little over a year. I painstakingly went through music and photos and categorized them into folders with a very slow and barely responsive navigation - that I thought was due to having a mechanical drive, but actually was due to exFAT. After switching to HFS+, navigation and transfer within a drive is much, much faster. This has me slightly worried about using exFAT for so long.

I haven't noticed any ill effects - I don't have a fancy network storage solution or ZFS checksums or what have you, just an external hard drive backing up to an identical second one - so my "testing" to make sure my data is intact amounts to literally going into random photo and music folders and making sure things are still openable.

My question is this - I'm on a more stable file system now, but is there any chance I could've ****ed up some of my data by using exFAT for a while? Is there anything I can do to verify the integrity of my data on my newly minted HFS setup or am I probably in the clear? Much of this stuff is no longer stored in the cloud, so the drive copy is the *only* copy. It's nothing that I would die without, but it's years and years of music collecting and phone photos.

Thanks in advance for any advice.

 

[Honza1]

Most likely you are OK. Testing is difficult - you can try to open each file and see if it works. That can be pain. If files copy from exFAT to other media, they are very likely OK. Now, there can be some edge cases where things look OK but later will fail, so this is not 100% sure thing, but likely.
Either way, if you have only one copy of the data, you are on borrowed time. You need at least two different copies on two different media and stored in different places. Ideally three... I throw away 5-10 disks per year (work related) and ~20% of these disks are due to failure, others due to age/small size. Every disk will die, it is just question when.

 

[harrisonjr98]

Most likely you are OK. Testing is difficult - you can try to open each file and see if it works. That can be pain. If files copy from exFAT to other media, they are very likely OK. Now, there can be some edge cases where things look OK but later will fail, so this is not 100% sure thing, but likely.
Either way, if you have only one copy of the data, you are on borrowed time. You need at least two different copies on two different media and stored in different places. Ideally three... I throw away 5-10 disks per year (work related) and ~20% of these disks are due to failure, others due to age/small size. Every disk will die, it is just question when.

Definitely aware of the importance of backups, my setup isn't very complicated though and so switching format of a drive required "committing" to the state of the data at a certain point in time.

I had an exFAT HDD and it was backing up with CCC to an identical exFAT HDD. I moved the "master" copy to an SSD that I had enough space on - formatted HFS+, and then moved it back to the newly-HFS "master" HDD. To switch the backup drive as well I just had to start over and back up again. So you can see how I ended up committing to the "master" HDD as the version of the data I'd be starting over with.

 

[Wando64]

is there any chance I could've ****ed up some of my data by using exFAT for a while?

I have no idea what you have been reading, but I am not aware of any evidence of exFAT (by itself) compromising the integrity of individual files.

Where did you get this idea from?

 

[frou]

The concept is generally called Bit-Rot and more primitive filesystems that don't use checksumming etc fail to mitigate it.

Funnily enough, IIRC APFS is not that hot in this area either.

 

[razmanugget]

I've read that zipping files (or any other type of compression) can be helpful to decreasing the frequency of bit-rot. With photos, Apple Photos combines them into a single file (.photoslibrary). I'm not sure if this is compressed though.

With music/videos, I don't think Apple Music performs any type of consolidation or compression, so you are more susceptible to bit rot here.

At least, this is what I was lead to believe years ago when I did some research on it too. Since then, I have come to expect that some of my music will/has gone bad and this is definitely the case when I check my music library. If it's super important to you to save your files, you could back up to CDs/DVDs. A major pain, but I don't think they suffer from bit rot if you get high quality ones (gold was best long ago).

 

[Wando64]

The concept is generally called Bit-Rot and more primitive filesystems that don't use checksumming etc fail to mitigate it.

Funnily enough, IIRC APFS is not that hot in this area either.

But Bit-Rot is not a specific problem of exFAT.
As for checksumming, I thought mostly it is done at low level by the disk controller.
It doesn’t matter which file system you use, if you leave a disk (or worse, an SSD) unused for long enough, it will degrade.
Unless you decide to use file systems based on ZFS or ReFS, you are vulnerable.