Optimized Storage reports?

[digitalfx]

If you allow the operating system to purge that data, then you can get it back only if you have a backup.

Or if, for example, you originally used iTunes to download the data: you may re-use iTunes.

Do you, or did you, have any iTunes film or TV programme?



That may be normal, whilst the data is identified.

I have all optimize features disabled, so not sure why Apple should decide to purge anything at all. This is pretty scary. The fact is Apple has erased 50GB of data without my consent.

Another question:
Why does "Documents" and "Desktop" no longer show up as local files? They are on my drive, but I can only see them as "in iCloud"

 

[RoboWarriorSr]

That may be normal, whilst the data is identified.

The only issue I have is that the memory continually increases until I force quit the process, got over 5 GB of swap which doesn't seem normal.

 

[blipmusic]

With a clean installation I found the optimisation enabled by default after allowing iCloud Drive Desktop and Documents folders. Maybe relevant: it was not a golden master.

Hmm, maybe that's a necessary evil. It could be to prevent iCloud sync from "force-filling" the hd of the target computer (source computer might have 500GB saved on desktop, whereas the target might only have a 256GB hd etc...). Hopefully incoming iCloud data will be prioritized for optimization over existing data though...

The more I read about storage optimization the less I want it on any of the computers I use.

Anyway, since I probably won't enable either of these features I should be safe, at least on paper. I always clean-install major versions though, so I'll have fresh backups at hand anyway.

 

[digitalfx]

Try a safe boot followed by a normal boot.

Thanks!!
Doing this restored most of the "purgeable" " data back to my drive. Now I'm back to 88GB free.
So the big question remains...why doesn't deselecting "optimize" restore the files back to the HD?

 

[chercm]

Thanks!!
Doing this restored most of the "purgeable" " data back to my drive. Now I'm back to 88GB free.
So the big question remains...why doesn't deselecting "optimize" restore the files back to the HD?

you mean after doing the safe boot and normal boot , you can see the documents folder in the local hDD ?

 

[navaira]

2. Is optimized storage enabled by default? (I hope not)

YES.

After reading this: https://9to5mac.com/2016/10/27/opinion-sierra-storage-management-system-nightmare/ I thought, wait, I definitely disabled this when installing. So I went on to find the setting. It took me a while to find it – System Preferences>iCloud>iCloud Drive (options). And it was on, both the laptop and desktop. I did NOT enable it myself.

I have no idea how to assess the damage. My MBP had 10 GB more free space for some reason, but my iCloud sub is 50 GB, out of which 40 is taken by photos. As for the desktop I don't remember how much space there was previously vs now. I installed 10.12.1 on the desktop and laptop Saturday. Laptop still has the sleep issue where it switches the screen off and it's impossible to switch back on without reboot. Photos is still analysing a few hundred pics every restart, then stopping. People still don't sync between devices.

Be right back, I'm off to find some aspects of iCloud that DO work as expected. I'll be hitting Time Machine hard in the meantime to restore El Capitan.

 

[blipmusic]

YES.

After reading this: https://9to5mac.com/2016/10/27/opinion-sierra-storage-management-system-nightmare/ I thought, wait, I definitely disabled this when installing. So I went on to find the setting. It took me a while to find it – System Preferences>iCloud>iCloud Drive (options). And it was on, both the laptop and desktop. I did NOT enable it myself.

I have no idea how to assess the damage. My MBP had 10 GB more free space for some reason, but my iCloud sub is 50 GB, out of which 40 is taken by photos. As for the desktop I don't remember how much space there was previously vs now. I installed 10.12.1 on the desktop and laptop Saturday. Laptop still has the sleep issue where it switches the screen off and it's impossible to switch back on without reboot. Photos is still analysing a few hundred pics every restart, then stopping. People still don't sync between devices.

Be right back, I'm off to find some aspects of iCloud that DO work as expected. I'll be hitting Time Machine hard in the meantime to restore El Capitan.

Ouch. In terms of possible data loss in your case, shouldn't the files at the very least be available online or are you saying they are lost forever?

I did a re-install and made sure nothing related to space optimisation was turned on (it wasn't IIRC) before re-installing apps and copying data back. My iCloud->iCloud-drive settings in sys-settings have Desktop and Documents unchecked so it seems they do obey the overall setting for optimisation. I actually couldn't find anything turned on by default.

I think Apple did a bad job of conveying the current settings, considering the risks. The optimisation settings almost feel hidden in the "About this Mac..." option in the Apple-menu. For such an important choice there's no overview telling me explicitly what the current settings are other than somewhat ambiguous "switch on..." buttons. On top of that there's a simple check box for Desktop and Documents in the iCloud/iCloud-drive-settings.

In short, my data is the king and queen the rest of the system should kneel to. My computer and its software exists for the sole purpose of handling my data, whether that is an e-mail I just received, files describing a working theory for cold fusion or a photo of a kitten. The only entity who can reliably determine (relatively speaking) whether a piece of data is important to ME is me, myself and I. It's not about how or when these files have been used, it's what's inside of them. Content. Maybe there is a file I only open once a year but is extremely important? Maybe it's huge in size? Maybe my hd space is lacking? Maybe I absolutely need that file in an offline environment?

Spreadsheet editors are already permuting my data if I don't turn off the "clever" options quickly enough, I don't want my computer to be that kind of clever (I realise it already is in many ways).

Besides possible security risks, "small" data in constant flux such as contacts, e-mail, bookmarks are all great to have synced online. The rest of my data? Online as a backup: yes. Online as the "main" storage? No.

I never upgrade between e.g. 10.11->10.12 versions. It's always a complete re-install and I do manual backups (I do even when I use time machine). I've only upgraded my personal laptop so far, the work Mac is still on 10.11.

 

[navaira]

Ouch. In terms of possible data loss in your case, shouldn't the files at the very least be available online or are you saying they are lost forever?

There is nothing I could have possibly lost forever – unless my Time Machine dies, my yearly copy of everything that I keep separately upstairs dies AND CrashPlan dies. But the thing is I don't know what I can be missing. I restored the laptop completely from 10.11.6 so it has everything. But with the desktop I assume I am just going to bump into "file not found" at some random point in time. My 4TB Time Machine should retain no matter what that file would be, but if not, I'll get it back from CrashPlan.

This is the single biggest FAIL I have encountered from Apple – personally. I couldn't switch on iCloud Music Library when it launched, because I had too many songs, and when I read how that ended for many people I felt so lucky. Dodged the... was it Lion update that deleted user accounts? Can't remember. Anyway, I didn't have that either. iTunes deleted some of my songs without permission, but I had them backed up. Not nice, iTunes. But undo-able easily.

But Sierra doesn't ask during config process whether you want "optimised storage". It asks you if you want to store documents and desktop in iCloud. I said no. And then only because I happened to read a link to a link to something posted here I find out that Optimised Storage is something else and that it switched on by default on my computers, but judging by 9to5mac comments – not on everyone's! Which leads again to the "you are lying/you're holding it wrong/where's a screenshot" comment fight.

 

[blipmusic]

There is nothing I could have possibly lost forever – unless my Time Machine dies, my yearly copy of everything that I keep separately upstairs dies AND CrashPlan dies. But the thing is I don't know what I can be missing. I restored the laptop completely from 10.11.6 so it has everything. But with the desktop I assume I am just going to bump into "file not found" at some random point in time. My 4TB Time Machine should retain no matter what that file would be, but if not, I'll get it back from CrashPlan.

This is the single biggest FAIL I have encountered from Apple – personally. I couldn't switch on iCloud Music Library when it launched, because I had too many songs, and when I read how that ended for many people I felt so lucky. Dodged the... was it Lion update that deleted user accounts? Can't remember. Anyway, I didn't have that either. iTunes deleted some of my songs without permission, but I had them backed up. Not nice, iTunes. But undo-able easily.

But Sierra doesn't ask during config process whether you want "optimised storage". It asks you if you want to store documents and desktop in iCloud. I said no. And then only because I happened to read a link to a link to something posted here I find out that Optimised Storage is something else and that it switched on by default on my computers, but judging by 9to5mac comments – not on everyone's! Which leads again to the "you are lying/you're holding it wrong/where's a screenshot" comment fight.

In my case all the optimisation options were switched off after a fresh install IIRC (again, complete re-install not on top of 10.11). I did uncheck the "store in iCloud" option during install. It's really odd (and bad) that users experience such inconsistency with a critical option in terms of being default or not.

As soon as the system starts tampering with my data behind the curtain I take issue. Data I manually put there for a reason. I have colleagues who travel to the middle of the jungle to do research (i.e. no internet whatsoever). What if the hd was almost full and the system decided to silently move critical data to the cloud just before heading out?

First time off was iTunes (many years ago) that re-organized my music - I'm fine with this but it was a surprise - to the optimisation we have now that decides whether or not the user needs a file available for offline use based on usage and available disk space (as far as I know - please correct me if I'm wrong).

By the way, the "Benjamin Button" MPB review in your sig is a thing of beauty.

 

[navaira]

As soon as the system starts tampering with my data behind the curtain I take issue. Data I manually put there for a reason. I have colleagues who travel to the middle of the jungle to do research (i.e. no internet whatsoever). What if the hd was almost full and the system decided to silently move critical data to the cloud just before heading out?

Exactly – I don't really travel to the middle of the jungle much but that doesn't mean I am willing to part with my files at Apple's whim, find myself offline and discover I lost half of my documents...

From the article I linked to a few posts ago:

First, it deleted almost all my documents; it left me with just a handful. Not the most recently-opened ones, as you’d expect, but completely random ones. [...] Yes, one of the files Sierra had deleted was my mailbox profile (I use Postbox rather than Apple Mail). So I couldn’t even open the app! I tried some other apps. Some opened, others couldn’t, either because Sierra had deleted needed files or because there was not enough drive space to create the temporary files needed to open the app.

Who came up with this, why and how – and how the hell does Cue still have a job?

 

[grahamperrin]

… Who came up with this, …

When it was announced, probably at WWDC, my first guess was:

• some of what Apple uses for optimised storage with iCloud Drive might relate to the company's Location independent files patent.

If that guess is correct, then it may help to know that one of the three inventors, Deric Horn (Apple's Core Storage guy), left the company in December 2013.