Shorter encryption time for empty volumes?

[Thor774]

Setting up my external 5TB USB3.0 Time Machine disk I decided to try 2 approaches:

1. First I enabled TM on the empty drive that was formatted with HSF+ (non encrypted). I configured TM to encrypt my backup drive on the go. The total size of my TM backup is more than 3TB and this process was taking an eternity. After 2 days the diskutil cs list command was showing that the drive conversion progress was just 20% and TM was still running its first backup work.

I decided to cancel that process (was not thinking on waiting a whole week for the conversion to take place. I then took route number 2:

2. I used Disk Utility to erase my TM backup disk and formatted with HSF+ but this time with encryption. I provided the password and after this the drive showed up on Finder. I then enabled TM backups on this drive and this time encryption was selected by default. The whole backup process took this time something like 22 hours.

I have also tested with other external disks and this has shown me that it appears to always be much more faster to format a volume with HSF+ encrypted and then put files on it instead of encrypting the disk with all the files already on it.

At the beginning I was thinking that full disk encryption would take the same amount of time using erase+format with encryption within Disk Utility and using just right click + Encrypt disk from Finder with files already present on the disk. I had understood that full disk encryption always wrote over the whole disk (files and empty disk space) so it didn't matter if the disk had 10MB or 2TB of data on it, what mattered was the size of the volume, but after these experiments I have found that it is definitely much more faster (almost instantaneous) to just erase and format the disk with encryption from Disk Manager and then copy the files into it.

Have you seen this behavior too? You can check the encryption progress by using diskutil cs list on Terminal.

Thanks for your feedback.

 

[Weaselboy]

Have you seen this behavior too?

Yep... I have done both and it works exactly like you described. The difference is in the first case the system has to go through and encrypt all that data on the drive already, where with the reformat it is not really encrypting any data at all... it just erases the drive and formats it in the encrypted format.

 

[BLUEDOG314]

Ive experienced this as well and it really makes me wonder what is actually going on. Like you said even empty space, or zeros, has to or at least should now look random. From how I see it, it SHOULD take the same amount of time. I feel that the encrypted format for erasing just makes the volume core storage because we know you cant write 5TB of random data instantly.

 

[Thor774]

Then we all agree on this, something that rises a new concern in my eyes:

We have 2 approaches to the volume encrypting process:

1. Converting an already existing volume with or without data to an encrypted volume using the Finder -> Right Click on the volume -> Choose Encrypt Volume, means that every block in the volume (used and free) is visited and rewritten with the encryption key (a process that takes a very long time). You can reboot the Mac and use the volume without any problems as all the data that is written to the volume from this point is going to be encrypted, the volume encryption process continue in the background.

2. Using Disk Utility for erasing a volume and encrypting it from scratch: Right Click on the disk / volume -> Erase -> choose filesystem Mac OS Extended (Journaled, Encrypted) -> Assign a password -> Click Erase. This process is almost instantaneous. You can use diskutil cs list on Terminal to confirm that the conversion process is finished in no time. After this you can write data to the volume and it will be encrypted. This process is faster as there is no background encryption taking place, just the new data is been encrypted.

As far as I can see there is no rewriting of the volume unallocated space using the 2nd. method, against the 1st. method that indeed rewrites every sector on the disk with the encryption key. If this indeed works like this it is very concerning. All the unallocated space on the newly encrypted volume using method 2 will be non rewritten and available to recuperation by forensic methods or maybe even by a good disaster recovery tool.

What do you think of this? What am I missing here?

 

[Alrescha]

What do you think of this? What am I missing here?

Formatting a volume (encrypted or not) does not do anything regarding removing existing data. If you want to encrypt you use method 1. If you do not care about the old data (which was not encrypted in the first place) you use method 2. No surprises anywhere.

A.

 

[Thor774]

Formatting a volume (encrypted or not) does not do anything regarding removing existing data. If you want to encrypt you use method 1. If you do not care about the old data (which was not encrypted in the first place) you use method 2. No surprises anywhere.

A.

But there is a big difference: You cannot get anything back (even forensically) from a drive that has been encrypted using method 1 if you do not have the encryption key, as all the sectors (used and free) in that drive have been rewritten with the encryption key.
With method 2 I see that it could be possible to extract old data present on the unallocated space without needing the encryption key using forensic methods!

I guarantee that lots of people think that if the disk shows as full encrypted (diskutil cs list on Terminal shows Conversion Status: Complete instead of Converting) then nothing can be extracted from it if if the encryption key is not available, if this process happens indeed as I have described. That is why I ask what it is that I am missing here? I really hope that I am missing something, because it cannot be this easy to pull data out from a drive that apparently is encrypted.

 

[Alrescha]

The only mystery for me is why you would expect data that you *did not encrypt* to be protected - regardless of whatever diskutil says.

If you are knowledgeable enough to use Disk Utility and diskutil list in Terminal, you should be familiar with how Format works. It is all very consistent.

A.

addendum:

I am sorry if I seem unsympathetic, but all the big buttons in the UI for the naive user say "Encrypt" on them and do the right thing. You have found the relatively hidden and obscure cool tools and appear to have the objection that the naive user will be surprised. I think that is unreasonable.

 

[Thor774]

You are right in that, format is format, even if you are using encryption, but I was been confused by the encryption part, and I guess many can be tricked in thinking the same way, that if you use encryption when formating it will mean full security on the whole disk and that no files can be pulled out from it if the encryption key is unknown.

It looks like using method 2 format plus security wipe has to be the way then, or even a faster and better way would be to use Secure Erase with something like hdparm or Parted Magic and then erase and encrypt the disk with Disk Utility.
Thanks for a constructive thread.
[doublepost=1490400508][/doublepost]

addendum:
I am sorry if I seem unsympathetic, but all the big buttons in the UI for the naive user say "Encrypt" on them and do the right thing. You have found the relatively hidden and obscure cool tools and appear to have the objection that the naive user will be surprised. I think that is unreasonable.

The only "hidden and obscure cool tool" I was using that is not present in the common and accessible GUI was diskutil cs list and this was only for showing the status of the encryption. It doesn't change anything on the fact that it is not difficult to get confused by this when encrypting a disk with Disk Utility. Read some of the previous posts by other members and you will see what I am talking about.

I understand that for experienced users this could be trivial, but many novices and others (like myself) that have not fiddled much with encryption before can be surprised to find that erasing and encrypting a disk with DU does not clean up previous information if you do not use secure wipe or the other methods explained before.

Anyway, it is excellent to finally understand this topic and I thank you again for your contribution.

 

[Alrescha]

The only "hidden and obscure cool tool" I was using that is not present in the common and accessible GUI was diskutil cs list and this was only for showing the status of the encryption.

I did say "relatively". I suggest that the majority of users never know of, nor touch Disk Utility.

On the bright side, you only have to encrypt (or zero-out and format/encrypted) the first time. After you have done that all your future work can be accomplished with a quick Format/Encrypted.

Best,

A.