Cannot delete symlink from root

[chacaito]

Hi, fully agree with Analog Kid,

MacOs since some previous versions, made 2 partitions for 1 installation. Since APFS use some container, two partitions can share space of one container. So what Apple did, they use a partition for the OS itself, which is in READ ONLY mode, and a second partition is for the User Data that is in READ WRITE mode.

The advantage : Nobody can delete some OS files by error or make some mistake to remove important file for the OS.
So the home directory you can see in the read only disk, is just a mount point to the READ AND WRITE directory for the users that can read and write to their home directory without touching anything in the OS filesystem itself. It's just a 'protection' again that.

So the OS itself, is on a READ ONLY drive, and the Users data is linked to the other partition that is READ and WRITE mode, and can be erase by some Users (for any reason).

MacOs is a Unix, and it's the only Unix that is addressed to general population to be able to use. Remark, Linux can be use by general public also.

The original question is fully justified, to get some explanations why there is link here, and there.

 

[svenmany]

by back

I’m not a file system expert by any stretch, and I know there are people here who are so maybe they’ll weigh in, but from the limited documentation I can find a firmlink looks less like a hardlink and more like a symlink with path traversal history.

A key feature of a hardlink is that it anchors a file’s data to the directory structure in two or more places. You can delete either reference to the file’s inode but the content will remain linked into the file system until you delete all of those references.

With a symlink, there’s only one anchor to the data and the symlink is basically a redirect to the original file by name. When you traverse a symlink to a directory and then back out, you’ll follow the hard path not the symbolic one.

This is part of what makes hardlinks to directories so treacherous. When you follow a hard link into a directory and then back out, which path do you follow? There are two hard paths out, both with equal standing, and both stateless.

A firmlink looks like a symlink that when you follow a path and then back out it follows the link in reverse, not the hard path. It makes the links between directores more transparent. It does not appear to include the key feature of a hard link though, being that there is only one anchor for the contents into the filesystem. If a firmlink is created to another path (and firmlinks are only permitted to directories), and then the original path is deleted, the content does not remain linked to the file system by the firmlink.

I don’t see any way for users to create firmlinks. The purpose is just to be able to make separate read only and read/write volumes appear as one unified directory structure to prevent malicious changes to the file system while retaining something close to a standard unix file tree.

Thanks for the reply.

By "back out", do you mean "cd .."?

 

[Analog Kid]

Thanks for the reply.

By "back out", do you mean "cd .."?

Yeah, return back up the directory tree to an unambiguous parent. ’cd ..’ is the typical way to do that as a user, but it seems to cause problems with other applications as well, thus the general unix-wide ban on hardlinks to directories. If you’re in a directory and have no state as to how you got there, but there are two possible parents, which one do you pick?

 

[svenmany]

Yeah, return back up the directory tree to an unambiguous parent. ’cd ..’ is the typical way to do that as a user, but it seems to cause problems with other applications as well, thus the general unix-wide ban on hardlinks to directories. If you’re in a directory and have no state as to how you got there, but there are two possible parents, which one do you pick?

First off, thanks for helping me think through this. I'm trying to understand what "A firmlink is described by Apple as a “bi-directional wormhole” between two filesystems" means. I'm also trying to develop an intuition about them, but I'm not even sure what my goal is.

A file system presents an API to an application. An application presents some mental model to the user. Finder and zsh present two different models to the user. When I think about this, I try to compare what I see to Windows junction points and symbolic links.

Windows Explorer, Command Prompt, and Powershell all present consistent behavior when descending down and then back up. They all remember how you got there and reverse the path you travelled. So, the state of the application, for any of the three, is not only the directory you're in, but also the path you used to get there (or the path you used to specify the location).

Shell (I use zsh) and Finder treat symbolic links differently. The shell behaves like the Windows shell. When I reverse my path navigation with "cd .." I navigate back from where I came. It doesn't follow the actual directory if I arrived at the directory through a link. In fact "pwd" shows the path you used to get there, which might include a symbolic link, and "cd .." respects what you see in "pwd". I tested Bash as well. (What a nightmare if they were different.)

Finder behaves differently. If you navigate into a symbolic link you are immediately told you are in the "actual" directory (the target of the link). You lose memory of the symbolic link itself. (I show the path bar in Finder, so it's easy to see.) So, in Finder reversing navigation down with "command up arrow" doesn't reverse the paths encountered to get there.

Firmlinks behave the same as symbolic links in terms of this navigation behavior; the Finder loses memory of the navigation and the shell remembers it. Navigating back in Finder takes you to what we might consider the "real" directory. Navigating back in the shell takes you back where you came from.

So why don't I think of firmlinks as symbolic links? Because the shell presents the user with the impression that both locations have equal status; they both link to the other. I use the command "ls -lO" to gain that impression. Both "System/Volumes/Data/Users" and "/Users" are shown to be directories with the "sunlnk" flag. The fact that they both link to each other might be what Apple means by "wormhole".

I interpret the fact that Finder gives you the impression that "/Users" is the real one to be just an illusion. I suspect that the shell's presentation more closely represents the actual state of the file system. Another illusion that the Finder presents is that "/System/Volumes/Data", as seen in the shell, is presented as "System/Volumes/<data volume name>" in Finder.

So, in my mind, Firmlinks are far different than Symbolic Links. But, they're also different than Hard Links since the two sunlnk directories are on two different volumes. I guess it's a just an opinion which of the two are a closer match to your intuition. Hard Link seems closer to me.

 

[Analog Kid]

First off, thanks for helping me think through this. I'm trying to understand what "A firmlink is described by Apple as a “bi-directional wormhole” between two filesystems" means. I'm also trying to develop an intuition about them, but I'm not even sure what my goal is.

A file system presents an API to an application. An application presents some mental model to the user. Finder and zsh present two different models to the user. When I think about this, I try to compare what I see to Windows junction points and symbolic links.

Windows Explorer, Command Prompt, and Powershell all present consistent behavior when descending down and then back up. They all remember how you got there and reverse the path you travelled. So, the state of the application, for any of the three, is not only the directory you're in, but also the path you used to get there (or the path you used to specify the location).

Shell (I use zsh) and Finder treat symbolic links differently. The shell behaves like the Windows shell. When I reverse my path navigation with "cd .." I navigate back from where I came. It doesn't follow the actual directory if I arrived at the directory through a link. In fact "pwd" shows the path you used to get there, which might include a symbolic link, and "cd .." respects what you see in "pwd". I tested Bash as well. (What a nightmare if they were different.)

Finder behaves differently. If you navigate into a symbolic link you are immediately told you are in the "actual" directory (the target of the link). You lose memory of the symbolic link itself. (I show the path bar in Finder, so it's easy to see.) So, in Finder reversing navigation down with "command up arrow" doesn't reverse the paths encountered to get there.

Firmlinks behave the same as symbolic links in terms of this navigation behavior; the Finder loses memory of the navigation and the shell remembers it. Navigating back in Finder takes you to what we might consider the "real" directory. Navigating back in the shell takes you back where you came from.

So why don't I think of firmlinks as symbolic links? Because the shell presents the user with the impression that both locations have equal status; they both link to the other. I use the command "ls -lO" to gain that impression. Both "System/Volumes/Data/Users" and "/Users" are shown to be directories with the "sunlnk" flag. The fact that they both link to each other might be what Apple means by "wormhole".

I interpret the fact that Finder gives you the impression that "/Users" is the real one to be just an illusion. I suspect that the shell's presentation more closely represents the actual state of the file system. Another illusion that the Finder presents is that "/System/Volumes/Data", as seen in the shell, is presented as "System/Volumes/<data volume name>" in Finder.

So, in my mind, Firmlinks are far different than Symbolic Links. But, they're also different than Hard Links since the two sunlnk directories are on two different volumes. I guess it's a just an opinion which of the two are a closer match to your intuition. Hard Link seems closer to me.

Sorry, I'm kind of at the limits of my understanding of how these things work under the hood, and my ignorance of Windows runs deep. I'm reading what I can find to catch up, but there's remarkably little documentation around firmlinks-- probably because they're meant to be an Apple hack to partition their filesystem, not a general purpose tool.

What I presume is that in the symlink case, the shell is maintaining the state. So you're able to back out the way you came in because the shell is trimming and appending to the `pwd` directory. I'd argue that Finder should do the same (though it doesn't). Apple should try to hide this nonsense from the user, and more importantly, users who have know interest in understanding the differences we're discussing here.

The problem, I think, is in applications that are given an inode and need to walk from that inode back up toward the root-- without state or history, they can't know which path to take.

I think the firmlinks solve this problem with inode hacks, by setting the high bits on one volume or the other in a group:

macOS 10.15 Volumes & Firmlink magic

With macOS 10.15 - Catalina, Apple has introduced a change in the way system and user data is stored on disk. In prior versions, the root '...

www.swiftforensics.com

Ignoring the high order bits makes the two paths the same, using the high order bits makes them distinct? I haven't seen that explicitly laid out, but it seems to be what's implied.

 

[svenmany]

The problem, I think, is in applications that are given an inode and need to walk from that inode back up toward the root-- without state or history, they can't know which path to take.

I've never considered that an application might be given an inode and then need to deduce a particular path that gets to that inode. In the APIs that I use they are completely distinct notions, path locations and files. But, the API's that I use are agnostic to the kind of file system under consideration.

I think the firmlinks solve this problem with inode hacks, by setting the high bits on one volume or the other in a group:

macOS 10.15 Volumes & Firmlink magic

With macOS 10.15 - Catalina, Apple has introduced a change in the way system and user data is stored on disk. In prior versions, the root '...

www.swiftforensics.com

Ignoring the high order bits makes the two paths the same, using the high order bits makes them distinct? I haven't seen that explicitly laid out, but it seems to be what's implied.

I read that post; thanks for the link. I hadn't noticed that inode values were crafted that way. My suspicion is that Apple wanted to keep the inode number on the two peer sunlnk files the same. For example

/System/Library/Caches has the same inode number as /System/Volumes/Data/System/Library/Caches,

have the same inode number, yet (I believe) they exist in two different file systems. Apple decided to allocate such sunlnk numbers from the range of inode numbers being assigned to the data volume's files. Then, to ensure that there is no collision with other files on the System volume (the non-sunlnk ones), they offset all inode numbers on the System volume by some large amount. Unless I'm missing something, I don't know why Yogesh finds that clever.

The subtlety is probably in the implementation details that required Apple to keep the inode numbers of peers the same.

Anyway, thanks for the indulgence. This really was a major deviation from what the original OP's issue was - just a symbolic link from the read-only system volume to a directory at the root of the data volume. One question I do have about that is why Apple used a symbolic link from /home to /System/Volumes/Data/home rather than using firmlinks like they did between /Users and /System/Volumes/Data/Users. But that might be a mystery for a different thread.