Solution: Reclaim storage back from "System"

[Mike Boreham]

The latest version Carbon Copy Cloner (5.1) can also create and delete snapshots, as well as showing space used by them.

https://bombich.com/blog/2018/03/30...ups-carbon-copy-cloner?utm_source=xlr8yourmac

https://bombich.com/kb/ccc5/leveraging-snapshots-on-apfs-volumes

 

[theFalcon]

After upgrading to High Sierra the "System" storage was 260GB (About This Mac -> Storage). I also used DaisyDisk to confirm that there were "200GB of hidden system files" that can't be shown or deleted.

Turned to our good friend Google and I found that Time Machine local backups were the reason and 'sudo tmutil disablelocal' command was supposed to help, if only "disablelocal" verb had not been removed from High Sierra. So back to square one.

Did some digging a.k.a. opened the manual for tmutil. I found that there were two useful verbs "listlocalsnapshots" and "deletelocalsnapshots". Used the first one to get the exact date stamps required for the second one and deleted all local snapshots manually.

Result: "System" went from 260GB to 60GB.

Step by step I went as following:

sudo tmutil listlocalsnapshots /

This resulted:

com.apple.TimeMachine.2017-09-27-005259
com.apple.TimeMachine.2017-09-27-104645
com.apple.TimeMachine.2017-09-27-114218
com.apple.TimeMachine.2017-09-27-124220

I took these four date stamps and followed the next command with each as following:

tmutil deletelocalsnapshots 2017-09-27-005259

So in the end if i double checked with

sudo tmutil listlocalsnapshots /

there were no snapshots and after checking "About This Mac -> Storage" I was happy

Hope this helps!

This worked a treat. I had to reset my mac for it to take effect but it removed about 250gb of space I could not recover. Thank you very much.

 

[MacDawg]

The latest version Carbon Copy Cloner (5.1) can also create and delete snapshots, as well as showing space used by them.

https://bombich.com/blog/2018/03/30...ups-carbon-copy-cloner?utm_source=xlr8yourmac

https://bombich.com/kb/ccc5/leveraging-snapshots-on-apfs-volumes

This was my issue... the Carbon Copy Cloner snapshots
I had read through this and a few other threads here on MR searching for answers
None of the suggestions were working for me
My available space was falling gradually and I ignored it until it started getting critical and dropping at an alarming rate
I ended up with 5GB available on a 1TB drive before I was able to reverse it

I found the snapshots in CCC, listing both the Time Machine snapshots and the CCC snapshots
However I could not delete them as suggested in the CCC documentation
It would tell me it was unable to delete (even selected individually) and to restart the system and try again
Apparently I didn't have enough free space to even delete them

I ended up setting the retention policies to 0 and running a clone and that deleted them
Then I turned snapshots off

I reclaimed 490 GB of space

 

[Goldmonkey]

After upgrading to High Sierra the "System" storage was 260GB (About This Mac -> Storage). I also used DaisyDisk to confirm that there were "200GB of hidden system files" that can't be shown or deleted.

Turned to our good friend Google and I found that Time Machine local backups were the reason and 'sudo tmutil disablelocal' command was supposed to help, if only "disablelocal" verb had not been removed from High Sierra. So back to square one.

Did some digging a.k.a. opened the manual for tmutil. I found that there were two useful verbs "listlocalsnapshots" and "deletelocalsnapshots". Used the first one to get the exact date stamps required for the second one and deleted all local snapshots manually.

Result: "System" went from 260GB to 60GB.

Step by step I went as following:

sudo tmutil listlocalsnapshots /

This resulted:

com.apple.TimeMachine.2017-09-27-005259
com.apple.TimeMachine.2017-09-27-104645
com.apple.TimeMachine.2017-09-27-114218
com.apple.TimeMachine.2017-09-27-124220

I took these four date stamps and followed the next command with each as following:

tmutil deletelocalsnapshots 2017-09-27-005259

So in the end if i double checked with

sudo tmutil listlocalsnapshots /

there were no snapshots and after checking "About This Mac -> Storage" I was happy

Hope this helps!

[doublepost=1529523848][/doublepost]Thank chirst, I had the same issue. This worked but after being in terminal I also had a bunch of snapshots left over from CCC and those weren't getting deleted using the command in terminal but you can access these in CCC itself on the left had column. Just select the disk and you can see them straight away, also you can disable CCC from creating them in the first place. Thanks for this thread though , it's the only one that seemed to offer a proper solution.

 

[manneh]

You have some app that is crashing and dumping the troubleshooting data in that folder. Other than just trial and error testing by watching the folder when you launch apps, I don't know a good way to tell what app is doing this.

Hi there I have been having the same issue recently, my system storage is at 87GB, and I'm currently on Sierra.

 

[Weaselboy]

Hi there I have been having the same issue recently, my system storage is at 87GB, and I'm currently on Sierra.

Run this command in Terminal and give it a couple minutes to complete, then tell us the result. This will list all the base folders along with their size in GB, including all system and hidden folders.

sudo du -d 1 -x -c -g /

 

[ignatius345]

Time Machine has bailed me out more than once, and I don't find it all that invasive.

CCC is perhaps a better program but the simplicity of Time Machine makes it well worthy.

Totally agree. I use both, for different purposes, as they each have their strengths.

Time Machine runs twice a day (thanks to TimeMachineEditor, I don't need to hear my loud external drive spin up every 60 minutes) so that I have local, updated snapshots of everything without my having to do anything. It's saved me on many occasions as well, and the recovery process is very simple should I need to retrieve an old version of something. I think I've got a year's worth of backups right now.

I also use Carbon Copy Cloner once a month to do full clones of my internal drive and my external media drives. Those backups (encrypted, naturally) are kept offsite at my office.

As a supplement, a lot of my stuff is in iCloud Drive or Dropbox at this point as well, so in a disaster any big gaps since my last CCC backup would be largely covered.

 

[manneh]

Run this command in Terminal and give it a couple minutes to complete, then tell us the result. This will list all the base folders along with their size in GB, including all system and hidden folders.

sudo du -d 1 -x -c -g /

/.fseventsd

du: /private/var/db/ConfigurationProfiles/Store: Operation not permitted

du: /private/var/folders/zz/zyxvpxvq6csfxvn_n00000y800007k/0/com.apple.nsurlsessiond: Operation not permitted

du: /private/var/folders/8d/d1pr55hj407bnb4f7kg3cwwh0000gn/0/SafariFamily: Operation not permitted

du: /private/var/folders/8d/d1pr55hj407bnb4f7kg3cwwh0000gn/0/com.apple.LaunchServices.dv: Operation not permitted

du: /private/var/folders/8d/d1pr55hj407bnb4f7kg3cwwh0000gn/0/com.apple.nsurlsessiond: Operation not permitted

du: /private/var/folders/8d/d1pr55hj407bnb4f7kg3cwwh0000gn/0/com.apple.routined: Operation not permitted

32 /private

1 /.DocumentRevisions-V100

0 /.vol

7 /Users

11 /Applications

1 /opt

1 /dev

0 /Volumes

0 /cores

66 /

66 total




This is what I got.

 

[Weaselboy]

32 /private

Something odd going on here. Run this to drill down a little more.

sudo du -d 1 -x -c -g /private

 

[manneh]

Something odd going on here. Run this to drill down a little more.

sudo du -d 1 -x -c -g /private

And I got this.



1 /private/etc

du: /private/var/db/ConfigurationProfiles/Store: Operation not permitted

du: /private/var/folders/zz/zyxvpxvq6csfxvn_n00000y800007k/0/com.apple.nsurlsessiond: Operation not permitted

du: /private/var/folders/8d/d1pr55hj407bnb4f7kg3cwwh0000gn/0/SafariFamily: Operation not permitted

du: /private/var/folders/8d/d1pr55hj407bnb4f7kg3cwwh0000gn/0/com.apple.LaunchServices.dv: Operation not permitted

du: /private/var/folders/8d/d1pr55hj407bnb4f7kg3cwwh0000gn/0/com.apple.nsurlsessiond: Operation not permitted

du: /private/var/folders/8d/d1pr55hj407bnb4f7kg3cwwh0000gn/0/com.apple.routined: Operation not permitted

32 /private/var

1 /private/tmp

0 /private/tftpboot

32 /private

32 total

 

[Weaselboy]

32 /private/var

Keep on drilling down. It is normal to have a few GB in there as that is where some of the system caches are, but 32GB is a bit high. How much system memory do you have?

sudo du -d 1 -x -c -g /private/var

 

[manneh]

Keep on drilling down. It is normal to have a few GB in there as that is where some of the system caches are, but 32GB is a bit high. How much system memory do you have?

sudo du -d 1 -x -c -g /private/var

I have 121GB. I have tried everything for the past 3 days and I couldn't figure it out, I downloaded OmniDiskSweeper and I deleted a lot of files but it still won't solve the issue.

 

[Weaselboy]

I have 121GB

Not drive space, but system RAM.

 

[manneh]

Not drive space, but system RAM.

8GB

 

[RobbieTT]

Thanks guys, it brought my 2018 MBP system size down to less than 8GB - very happy with the crash diet!

Take that Windows!

 

[KostisLag]

After upgrading to High Sierra the "System" storage was 260GB (About This Mac -> Storage). I also used DaisyDisk to confirm that there were "200GB of hidden system files" that can't be shown or deleted.

Turned to our good friend Google and I found that Time Machine local backups were the reason and 'sudo tmutil disablelocal' command was supposed to help, if only "disablelocal" verb had not been removed from High Sierra. So back to square one.

Did some digging a.k.a. opened the manual for tmutil. I found that there were two useful verbs "listlocalsnapshots" and "deletelocalsnapshots". Used the first one to get the exact date stamps required for the second one and deleted all local snapshots manually.

Result: "System" went from 260GB to 60GB.

Step by step I went as following:

sudo tmutil listlocalsnapshots /

This resulted:

com.apple.TimeMachine.2017-09-27-005259
com.apple.TimeMachine.2017-09-27-104645
com.apple.TimeMachine.2017-09-27-114218
com.apple.TimeMachine.2017-09-27-124220

I took these four date stamps and followed the next command with each as following:

tmutil deletelocalsnapshots 2017-09-27-005259

So in the end if i double checked with

sudo tmutil listlocalsnapshots /

there were no snapshots and after checking "About This Mac -> Storage" I was happy

Hope this helps!

My friend you saved me!!!! Thank you so much!!!

 

[pdmcmahon]

Thanks for the research and your post. After reading your comment, I dug around some more and read the man pages.

Edit: I also found this command that will remove them all.

tmutil  listlocalsnapshotdates / |grep 20|while read f; do tmutil deletelocalsnapshots $f; done

tmutil listlocalsnapshotdates / |grep 20|while read f; do tmutil deletelocalsnapshots $f; done

This just freed up about 65 GB on my 500 GB MacBook Pro SSD. I went from about 85 GB free to over 151 GB free.

THANK YOU!

 

[turbineseaplane]

Phenomenal thread!

THANK YOU all!
Just went from 80gb free back up to 350gb

 

[Merilyn Fairskye]

Thanks for the research and your post. After reading your comment, I dug around some more and read the man pages.

I found this post by @oatman13 with the command below to thin those local snapshots.

sudo tmutil thinLocalSnapshots / 10000000000 4

That 10000000000 there is how much you want to thin in bytes (so about 9GB). I tested the command and the
listlocalsnapshots as you suggested before and after and this completely removed all the snapshots from my MacBook. I suppose if you had more than ~9GB you could stick another zero on that command to make it ~90GB.

At any rate the thinLocalSnapshots command removes them all with one command.

Edit: I also found this command that will remove them all.

tmutil  listlocalsnapshotdates / |grep 20|while read f; do tmutil deletelocalsnapshots $f; done

Thank you!!! Tried all of the above, was getting desperate, , then your second solution did the trick.

 

[dai-leung]

From this thread, snapshots seem to have given some expert computer users headache and obviously a hopeless situation to resolve for a non computer person like me. Why wouldn’t apple make it easy for people like me? For example, on the time machine window/panel list how much space snapshots is occupying, allow user to set an allowable space in GB for snapshots, and finally allow users to purge snapshots easily.

Trying to understand how snapshots works led me to this thread. The discussions here are above my head and I have the following elementary questions.

Q1) Does turning off Time Machine delete ALL snapshots stored in the startup disk? (Obviously not, based on the elaborate steps used by expert users on this thread.) Members of this thread use terminal and codes to delete snapshots. However, Apple suggests:

“If you want to delete local snapshots manually, turn off Time Machine temporarily:

1. Open Time Machine preferences from the Time Machine menu in the menu bar. Or choose Apple menu  > System Preferences, then click Time Machine.
2. Deselect ”Back Up Automatically” or click the Off/On switch, depending on what you see in Time Machine preferences.
3. Wait a few minutes to allow the local snapshots to be deleted. Then turn on Time Machine again. It remembers your backup disks.”

Q2) how do snapshots take up hundreds of GB of space(and without the user knowing it)?
I understand that after the initial TM backup (which consumes about the same space as what is on the startup disk). For subsequent backups, TM backups only changes made since the last backup. Thus initially if an external drive is used for the initial first ever backup which is then disconnected from the Mac, subsequently all the snapshots created should contain only changes made. These changes could be very small. So I don’t understand how they could consume hundred of GBs.

Q3) Apple says that only snapshots of the last 24 hours are stored on the startup disk. Is this true? If yes, are these 24 snapshots consuming hundreds of GBs? Or there are some other hidden snapshots which accumulate over time?

Q4) is there an easy way to find out how much space the snapshots are occupying (I have Carbon Copy Cloner)?

 

[Weaselboy]

Q3) Apple says that only snapshots of the last 24 hours are stored on the startup disk. Is this true? If yes, are these 24 snapshots consuming hundreds of GBs? Or there are some other hidden snapshots which accumulate over time?

Q4) is there an easy way to find out how much space the snapshots are occupying (I have Carbon Copy Cloner)?

Yes... they are saved hourly going back 24 hours, then purged by the OS. You should be able to just ignore them and the OS will deal with it.

If you open CCC and select your boot drive you will see the snapshots listed like in my screenshot.

 

[dai-leung]

Yes... they are saved hourly going back 24 hours, then purged by the OS. You should be able to just ignore them and the OS will deal with it.

If you open CCC and select your boot drive you will see the snapshots listed like in my screenshot.

Thanks for answering my questions, but I am still confused.

I found the following Apple document:
https://support.apple.com/en-us/HT201250 Back up your Mac with Time Machine dated 2/15/2019

On the time machine window, it says
”Time Machine keeps:
-Local snapshots as space permits”

”As space permits”, Does it mean that ‘disk is not 80% full” and “there is more than 5GBs free space left”. If this interpretation is correct, then it explains why there are many experienced users ran into the problem that hundreds of GBs of their startup disk was used by hidden snapshots.

So does each snapshot will be saved for only 24 hrs or until disk is 80% full or there is only 5GB left?

 

[Honza1]

This has been discussed many times elsewhere, but keep in mind, that this information varies a bit:

1. local snapshots are deleted after 24 hours. This is hard limit. (except when they are not as some supposedly experienced, but that seems some kind of failure of TM).
2. If users start "running out of space" (not sure what that exactly means for Apple) oldest snapshots are deleted to make space for user files.

This means, that your local snapshots may be deleted before 24 hours, when needed. But they are for sure deleted after 24 hours, even if you have plenty of space.

This also means, that - if the space is used by the snapshots - checking the space all the time and getting worried about not having XY% of free space all the time is not helpful. Snapshots are managed by OS. I have not seen anywhere the limits OS is imposing. "as space permits" is not very informative.

One thing to keep in mind is, that making snapshot on APFS is basically free now - in time (takes little to no time) and space (does not increase used space when created). APFS is smart and just records/links used files/their parts in snapshot and that is it. More space is needed ONLY when user writes something new which is not on the drive already. And space is not released from deleted file space when user deletes the file, but only after the last snapshot linking to this file is removed. At which point the file is marked deleted and space is made available.

Also, with SSD disks I am not sure if the old "you need to have 20% of free space" applies anymore. SSDs are different beasts that old spinners were.

In conclusion: old rules may be obsolete by now, we should stop obsessing about the snapshots.

That said, seems that there are cases when all of this smart stuff fails - there were few who reported real issues, not just OCD with space checking.

 

[dai-leung]

...local snapshots are deleted after 24 hours. This is hard limit.

... checking the space all the time and getting worried about not having XY% of free space all the time is not helpful.

One thing to keep in mind is, that making snapshot on APFS is basically free now - in time (takes little to no time) and space (does not increase used space when created). APFS is smart and just records/links used files/their parts in snapshot and that is it. More space is needed ONLY when user writes something new which is not on the drive already. And space is not released from deleted file space when user deletes the file, but only after the last snapshot linking to this file is removed. At which point the file is marked deleted and space is made available.

Also, with SSD disks I am not sure if the old "you need to have 20% of free space" applies anymore. SSDs are different beasts that old spinners were.

I have been reading ur posts in other threads to learn more about snapshots but have never expected to hear from u. Thank you for your clear and definitive answer! When experienced users were reporting missing several hundreds of GBs seemingly consumed by snapshots and asking for help, it is a scary situation to a non computer person. Going forward, I won’t be worrying about space consumed by snapshots anymore. I wish Apple would use ur posts in their write up to explain snapshots.

 

[Weaselboy]

https://support.apple.com/en-us/HT204015

@dai-leung I think what may also be confusing you is before High Sierra and APFS, the old HFS+ version of Time Machine local snapshots worked differently. Under HFS+ they were not deleted after 24 hours and would just pile up until the disk got close to 80% full. It also seemed very buggy if you go by forum posts here with people that had trouble with it. Now with High Sierra and Mojave using APFS, local snapshots only stay 24 hours so will normally not grow and grow like they did with HFS+. There seem to be far fewer reports of this being a problem now with the newer APFS method.

I'm guessing maybe you are seeing forum posts about the old HFS+ method and that is confusing you.