APFS: restore to new volume not working

[chucker23n1]

So, my situation is (still) that post-High Sierra upgrade, I ended up with three APFS containers, with one volume each. There's still (as of 10.13.4 beta) no way to merge containers, but I've found there is a way to pull in a volume from one container to another, given enough space: the Restore button in Disk Utility, with the special Restore to new volume checkbox. Cool!

Except it doesn't work

Validating target...
Validating source...
Validating sizes...
Restoring
Verifying
Inverting target volume...
Couldn't unmount all volumes prior to inversion - Resource busy
The operation couldn’t be completed. (OSStatus error 16.)
Operation failed…

It gets busy for a while, and then near what appears to be the finish line, it fails to "invert the target volume", whatever that means.

So,

• what is "inverting target volume"?
• what are the equivalent commands in diskutil/asr/etc. so I can try this step-by-step in Terminal?
• what resource is busy? Surely it's only talking about the volumes within the same container, but those are all unmounted.

 

[BLUEDOG314]

Could you do a diskutil apfs list in terminal and paste results please? Also please let me know what your end goal is. I have received the inversion message before, but to my knowledge asr with APFS can only work on entire containers, and requires in image that has been "imagescanned" as well.

 

[chucker23n1]

Could you do a diskutil apfs list in terminal and paste results please?

Sure.

APFS Containers (5 found)
|
+-- Container disk1 F1100A99-B7DD-4112-A8FD-1066924DDA92
|   ====================================================
|   APFS Container Reference:     disk1
|   Size (Capacity Ceiling):      29062860800 B (29.1 GB)
|   Minimum Size:                 28052054016 B (28.1 GB)
|   Capacity In Use By Volumes:   11300425728 B (11.3 GB) (38.9% used)
|   Capacity Not Allocated:       17762435072 B (17.8 GB) (61.1% free)
|   |
|   +-< Physical Store disk0s3 D4752035-5A6E-4868-AAE6-CC423D277FFC
|   |   -----------------------------------------------------------
|   |   APFS Physical Store Disk:   disk0s3
|   |   Size:                       29062860800 B (29.1 GB)
|   |
|   +-> Volume disk1s1 213000D4-263A-3DD7-89CC-24853BAFF15A
|       ---------------------------------------------------
|       APFS Volume Disk (Role):   disk1s1 (No specific role)
|       Name:                      Détresse (Case-insensitive)
|       Mount Point:               Not Mounted
|       Capacity Consumed:         11230281728 B (11.2 GB)
|       FileVault:                 No
|
+-- Container disk3 C679DF15-10AF-4675-B007-315669F6E160
|   ====================================================
|   APFS Container Reference:     disk3
|   Size (Capacity Ceiling):      100000002048 B (100.0 GB)
|   Minimum Size:                 100247486464 B (100.2 GB)
|   Capacity In Use By Volumes:   83856551936 B (83.9 GB) (83.9% used)
|   Capacity Not Allocated:       16143450112 B (16.1 GB) (16.1% free)
|   |
|   +-< Physical Store disk0s4 0370D37C-6B6D-4A03-830B-061EE0083ABD
|   |   -----------------------------------------------------------
|   |   APFS Physical Store Disk:   disk0s4
|   |   Size:                       100000002048 B (100.0 GB)
|   |
|   +-> Volume disk3s1 6A0C75D4-508C-3057-A0AF-4E1DE2D449D5
|   |   ---------------------------------------------------
|   |   APFS Volume Disk (Role):   disk3s1 (No specific role)
|   |   Name:                      Renaissance (Case-insensitive)
|   |   Mount Point:               /
|   |   Capacity Consumed:         78896635904 B (78.9 GB)
|   |   FileVault:                 No
|   |
|   +-> Volume disk3s2 309ECAED-D48A-4173-B1C1-4A633AF2883B
|   |   ---------------------------------------------------
|   |   APFS Volume Disk (Role):   disk3s2 (Preboot)
|   |   Name:                      Preboot (Case-insensitive)
|   |   Mount Point:               Not Mounted
|   |   Capacity Consumed:         22265856 B (22.3 MB)
|   |   FileVault:                 No
|   |
|   +-> Volume disk3s3 7D983E87-0C2E-4425-8E84-5D1F168A1721
|   |   ---------------------------------------------------
|   |   APFS Volume Disk (Role):   disk3s3 (Recovery)
|   |   Name:                      Recovery (Case-insensitive)
|   |   Mount Point:               Not Mounted
|   |   Capacity Consumed:         518266880 B (518.3 MB)
|   |   FileVault:                 No
|   |
|   +-> Volume disk3s4 9993BAEA-A837-475C-B454-FBEC2C836374
|       ---------------------------------------------------
|       APFS Volume Disk (Role):   disk3s4 (VM)
|       Name:                      VM (Case-insensitive)
|       Mount Point:               /private/var/vm
|       Capacity Consumed:         4295397376 B (4.3 GB)
|       FileVault:                 No
|
+-- Container disk2 349D36D5-7188-4B1A-A5AA-12A7BD0C3A78
|   ====================================================
|   APFS Container Reference:     disk2
|   Size (Capacity Ceiling):      371005173760 B (371.0 GB)
|   Minimum Size:                 356178317312 B (356.2 GB)
|   Capacity In Use By Volumes:   271784476672 B (271.8 GB) (73.3% used)
|   Capacity Not Allocated:       99220697088 B (99.2 GB) (26.7% free)
|   |
|   +-< Physical Store disk0s2 B3946DEB-341F-4C16-B601-9698BEB3EA00
|   |   -----------------------------------------------------------
|   |   APFS Physical Store Disk:   disk0s2
|   |   Size:                       371005173760 B (371.0 GB)
|   |
|   +-> Volume disk2s1 F9B87FA7-D7E6-4C6D-B9F1-C6BE0B6332A1
|       ---------------------------------------------------
|       APFS Volume Disk (Role):   disk2s1 (No specific role)
|       Name:                      Naviguer (Case-insensitive)
|       Mount Point:               /Volumes/Naviguer
|       Capacity Consumed:         271652421632 B (271.7 GB)
|       FileVault:                 No
|
+-- Container disk5 F1100A99-B7DD-4112-A8FD-1066924DDA92
|   ====================================================
|   APFS Container Reference:     disk5
|   Size (Capacity Ceiling):      29062860800 B (29.1 GB)
|   Capacity In Use By Volumes:   11300425728 B (11.3 GB) (38.9% used)
|   Capacity Not Allocated:       17762435072 B (17.8 GB) (61.1% free)
|   |
|   +-< Physical Store ERROR -69629
|   |   ---------------------------
|   |   APFS Physical Store Disk:   disk4
|   |   Size:                       29062860800 B (29.1 GB)
|   |
|   +-> Volume disk5s1 213000D4-263A-3DD7-89CC-24853BAFF15A
|       ---------------------------------------------------
|       APFS Volume Disk (Role):   disk5s1 (No specific role)
|       Name:                      Détresse (Case-insensitive)
|       Mount Point:               Not Mounted
|       Capacity Consumed:         11230281728 B (11.2 GB)
|       FileVault:                 No
|
+-- Container disk8 F1100A99-B7DD-4112-A8FD-1066924DDA92
    ====================================================
    APFS Container Reference:     disk8
    Size (Capacity Ceiling):      29062860800 B (29.1 GB)
    Capacity In Use By Volumes:   11300425728 B (11.3 GB) (38.9% used)
    Capacity Not Allocated:       17762435072 B (17.8 GB) (61.1% free)
    |
    +-< Physical Store ERROR -69629
    |   ---------------------------
    |   APFS Physical Store Disk:   disk7
    |   Size:                       29062860800 B (29.1 GB)
    |
    +-> Volume disk8s1 213000D4-263A-3DD7-89CC-24853BAFF15A
        ---------------------------------------------------
        APFS Volume Disk (Role):   disk8s1 (No specific role)
        Name:                      Détresse (Case-insensitive)
        Mount Point:               Not Mounted
        Capacity Consumed:         11230281728 B (11.2 GB)
        FileVault:                 No

Renaissance is the current boot volume.
Détresse is also a bootable system, but not currently running.
Naviguer is purely data.

They're all on their own container, which wasn't intended.

Also please let me know what your end goal is.

There's three containers with one volume each. They're all on the same physical disk. I want to turn them into one container with three volumes, to gain the space sharing feature.

I understand if this isn't possible with the boot volume, but I can't even get the two non-boot volumes to merge.

I have received the inversion message before, but to my knowledge asr with APFS can only work on entire containers, and requires in image that has been "imagescanned" as well.

The UI in Disk Utility seems to suggest you can take an APFS volume from container X and add it as a new volume to container Y. (Presumably, to accomplish this, it really does multiple internal steps, such as maybe creating a temporary disk image?)

The asr manpage, then, suggests you can similarly take a single APFS volume from a container disk image and add that to an existing container:

     Volume Restore with Creation
          If the target is a synthesized APFS whole disk or Apple_APFS partition, and the --erase option is
          not present, then asr will create a new volume in the given container, after which it will do a
          volume restore to that new volume, as with the previous section.  All other volumes in the con-
          tainer are preserved.

And:

     Creating a new APFS volume on the fly
          sudo asr restore -s <APFS image> --sourcevolumename SourceVolume -t /dev/disk2
    Here we get the same effect as the last example, except that asr will create a new volume on the target
    APFS container disk, given by /dev/disk2, and use that newly created volume as the target.  Any volumes
    which already existed in the container will still be there after the restore.

 

[BLUEDOG314]

I have a spare machine that I am going to set up similar to yours in terms of the partitions and see if i can achieve what you want. If it works I will post the entire input to terminal here.

EDIT: Actually can you tell me exactly what volumes, in terms of the disk identifiers from what you posted, you want in what containers ideally? It looks like multiple volumes have the same name.

EDIT 2: Ok looking at this one more time what you pasted shows 5 APFS containers so I am a little confused unless this is an error. I am assuming the two at the end showing the error are sort of made up that got their by some failed attempt to fix the issue. I'll post back, I'm setting up a machine like yours now.

EDIT Final: So I was playing around and attempted using asr with the --restorenewvolume thing and it outright didnt work saying file copy is no longer supported. Now clearly I didnt fill the partitions up as much as you, which brings up the other problem that you barely have enough free space to move things around even if it did work. Then you would have two more issues, one being that you would have to move everything into a container that is not the first container on the disk, and the second being that you would very likely have boot issues if you try to move around the main startup volume and ancillary preboot, recovery, and vm.

The only other way I can thing of doing this would be to attempt making images of all three containers, imagescanning them and let them reside on an external drive. Then wipe the drive to only one apfs container, first restore the container with the main boot volume, then try using asr and the --restorenewvolume thing again since you now are doing it with an image and not as a "file copy."

Essentially I do not think you are getting around reformatting without extensive work and image creation which will likely take longer than just backing up contents of data volume and reformatting anyway. If its any consolation, you arent even supposed to be able to have multiple containers on the same physical disk, it straight up blocks creation through terminal. To get it I had to make two hfs+ partitions and convert them afterwards.

 

[chucker23n1]

EDIT: Actually can you tell me exactly what volumes, in terms of the disk identifiers from what you posted, you want in what containers ideally? It looks like multiple volumes have the same name.

EDIT 2: Ok looking at this one more time what you pasted shows 5 APFS containers so I am a little confused unless this is an error. I am assuming the two at the end showing the error are sort of made up that got their by some failed attempt to fix the issue.

Oh wow, you're right. Disk Utility never showed those in the GUI. After a reboot, they disappeared again. Here's a listing that makes more sense to me:

APFS Containers (3 found)
|
+-- Container disk1 F1100A99-B7DD-4112-A8FD-1066924DDA92
|   ====================================================
|   APFS Container Reference:     disk1
|   Size (Capacity Ceiling):      29062860800 B (29.1 GB)
|   Minimum Size:                 28052054016 B (28.1 GB)
|   Capacity In Use By Volumes:   11308638208 B (11.3 GB) (38.9% used)
|   Capacity Not Allocated:       17754222592 B (17.8 GB) (61.1% free)
|   |
|   +-< Physical Store disk0s3 D4752035-5A6E-4868-AAE6-CC423D277FFC
|   |   -----------------------------------------------------------
|   |   APFS Physical Store Disk:   disk0s3
|   |   Size:                       29062860800 B (29.1 GB)
|   |
|   +-> Volume disk1s1 213000D4-263A-3DD7-89CC-24853BAFF15A
|       ---------------------------------------------------
|       APFS Volume Disk (Role):   disk1s1 (No specific role)
|       Name:                      Détresse (Case-insensitive)
|       Mount Point:               /Volumes/Détresse
|       Capacity Consumed:         11238498304 B (11.2 GB)
|       FileVault:                 No
|
+-- Container disk2 349D36D5-7188-4B1A-A5AA-12A7BD0C3A78
|   ====================================================
|   APFS Container Reference:     disk2
|   Size (Capacity Ceiling):      371005173760 B (371.0 GB)
|   Minimum Size:                 371275542528 B (371.3 GB)
|   Capacity In Use By Volumes:   252938465280 B (252.9 GB) (68.2% used)
|   Capacity Not Allocated:       118066708480 B (118.1 GB) (31.8% free)
|   |
|   +-< Physical Store disk0s2 B3946DEB-341F-4C16-B601-9698BEB3EA00
|   |   -----------------------------------------------------------
|   |   APFS Physical Store Disk:   disk0s2
|   |   Size:                       371005173760 B (371.0 GB)
|   |
|   +-> Volume disk2s1 F9B87FA7-D7E6-4C6D-B9F1-C6BE0B6332A1
|       ---------------------------------------------------
|       APFS Volume Disk (Role):   disk2s1 (No specific role)
|       Name:                      Naviguer (Case-insensitive)
|       Mount Point:               /Volumes/Naviguer
|       Capacity Consumed:         252789538816 B (252.8 GB)
|       FileVault:                 No
|
+-- Container disk3 C679DF15-10AF-4675-B007-315669F6E160
    ====================================================
    APFS Container Reference:     disk3
    Size (Capacity Ceiling):      100000002048 B (100.0 GB)
    Secondary Minimum Size:       78881599488 B (disk0s4) (78.9 GB)
    Capacity In Use By Volumes:   76009504768 B (76.0 GB) (76.0% used)
    Capacity Not Allocated:       23990497280 B (24.0 GB) (24.0% free)
    |
    +-< Physical Store disk0s4 0370D37C-6B6D-4A03-830B-061EE0083ABD
    |   -----------------------------------------------------------
    |   APFS Physical Store Disk:   disk0s4
    |   Size:                       100000002048 B (100.0 GB)
    |
    +-> Volume disk3s1 6A0C75D4-508C-3057-A0AF-4E1DE2D449D5
    |   ---------------------------------------------------
    |   APFS Volume Disk (Role):   disk3s1 (No specific role)
    |   Name:                      Renaissance (Case-insensitive)
    |   Mount Point:               /
    |   Capacity Consumed:         73207623680 B (73.2 GB)
    |   FileVault:                 No
    |
    +-> Volume disk3s2 309ECAED-D48A-4173-B1C1-4A633AF2883B
    |   ---------------------------------------------------
    |   APFS Volume Disk (Role):   disk3s2 (Preboot)
    |   Name:                      Preboot (Case-insensitive)
    |   Mount Point:               Not Mounted
    |   Capacity Consumed:         22261760 B (22.3 MB)
    |   FileVault:                 No
    |
    +-> Volume disk3s3 7D983E87-0C2E-4425-8E84-5D1F168A1721
    |   ---------------------------------------------------
    |   APFS Volume Disk (Role):   disk3s3 (Recovery)
    |   Name:                      Recovery (Case-insensitive)
    |   Mount Point:               Not Mounted
    |   Capacity Consumed:         517791744 B (517.8 MB)
    |   FileVault:                 No
    |
    +-> Volume disk3s4 9993BAEA-A837-475C-B454-FBEC2C836374
        ---------------------------------------------------
        APFS Volume Disk (Role):   disk3s4 (VM)
        Name:                      VM (Case-insensitive)
        Mount Point:               /private/var/vm
        Capacity Consumed:         2147659776 B (2.1 GB)
        FileVault:                 No

I'm trying to, as a first step just to get my feet wet, move Détresse (disk1s2) into the same container (disk2) as Naviguer.

I'll post back, I'm setting up a machine like yours now.

Now clearly I didnt fill the partitions up as much as you, which brings up the other problem that you barely have enough free space to move things around even if it did work.

Well, yes and no. disk2 has 118,07 GB free, and disk1 is only 29,06 GB total, of which only 11,31 are used.

It does get more complicated with Renaissance, yes. But after the first step, I should have a disk2 with 400,07 GB total and 135,83 GB free. Renaissance is 100 GB total, 14,35 GB of which are free. So moving that into the combined new disk2 as a second step should work?

(If that second step doesn't work, such as because moving the current boot volume is hard, that's fine.)

Then you would have two more issues, one being that you would have to move everything into a container that is not the first container on the disk, and the second being that you would very likely have boot issues if you try to move around the main startup volume and ancillary preboot, recovery, and vm.

Right. Let's ignore those aspects for now and just merge the two non-booting containers.

The only other way I can thing of doing this would be to attempt making images of all three containers, imagescanning them and let them reside on an external drive. Then wipe the drive to only one apfs container, first restore the container with the main boot volume, then try using asr and the --restorenewvolume thing again since you now are doing it with an image and not as a "file copy."

I'll look into that.

Essentially I do not think you are getting around reformatting without extensive work and image creation which will likely take longer than just backing up contents of data volume and reformatting anyway. If its any consolation, you arent even supposed to be able to have multiple containers on the same physical disk, it straight up blocks creation through terminal. To get it I had to make two hfs+ partitions and convert them afterwards.

I'm pretty sure I ended up with these three containers because that's what Disk Utility did.

 

[dsemf]

I'm pretty sure I ended up with these three containers because that's what Disk Utility did.

If you use the Partition button to add new APFS volumes, it creates another Container. If you select a container and select Add Volume, the volume is created within the selected container.

DS

 

[chucker23n1]

If you use the Partition button to add new APFS volumes, it creates another Container. If you select a container and select Add Volume, the volume is created within the selected container.

View attachment 754679

DS

I didn't manually create partitions or volumes; I converted from HFS+ to APFS and incorrectly assumed Disk Utility would be smart enough to convert into an existing APFS container; instead, it created a new one each time.

 

[teskor]

the Restore button in Disk Utility, with the special Restore to new volume checkbox. Cool!
Except it doesn't work
It gets busy for a while, and then near what appears to be the finish line, it fails to "invert the target volume", whatever that means.

I found this thread by googling, because I had the exact same problem. For reference, I found a workaround and
posted it here: https://discussions.apple.com/thread/250029045.
The relevant part of that message is:
"The process of cloning an apfs volume with disk utility led to the error in the title and left
me with a volume with only one file "ContainerToInvert" in it.
I did some googling and found that there's an utility apfs_invert. So I played around and found
that this command would make the copy work:

sudo /System/Library/Filesystems/apfs.fs/Contents/Resources/apfs_invert -d /dev/disk3s3 -n ContainerToInvert -s 1

(where /dev/disk3s3 is the device of the container partition (check with "diskutil list" or "diskutil apfs list" for the right name).

Is it possible that one can circumvent the error that easily? If yes, couldn't Apple simply fix Disk Utility?"

 

[chucker23n1]

I ended up giving up after a year of trying to merge the partitions through sanctioned ways and instead:

• made sufficient room in the big APFS container B
• used Carbon Copy Cloner to clone my boot volume (in APFS container A) to a fresh volume (in APFS container B)
• booted to that to and used it for a while to see if it was a working clone
• deleted the original volume and its container
• resized the other container

Boom, much more disk space.

It's a bit disappointing that Apple's user interfaces — the GUI and the diskutil CLI — are still failing at this task.

 

[Fernand]

and nobody knows what "inverting" means?