Shared Subtrees

개요와 네 가지 전파 유형

공유 서브트리(shared subtree)는 프로세스가 자신의 mount namespace를 복제한 뒤에도 부모 namespace에서 새로 마운트된 CD 같은 자원을 볼 수 있게 하는 전파 규칙이다. 이 규칙은 사용자별 namespace와 버전 파일 시스템 같은 기능을 조립하는 기반도 제공한다.

전파 속성은 정확히는 `struct vfsmount` 단위로 적용된다. 문서는 shared, slave, private, unbindable의 네 유형을 소개한 다음 상태 변경, 실제 사용 사례, 연산별 상세 의미, 구현 자료구조와 알고리즘을 설명한다.

.. SPDX-License-Identifier: GPL-2.0

===============
Shared Subtrees
===============

.. Contents:
        1) Overview
        2) Features
        3) Setting mount states
        4) Use-case
        5) Detailed semantics
        6) Quiz
        7) FAQ
        8) Implementation


1) Overview
-----------

Consider the following situation:

A process wants to clone its own namespace, but still wants to access the CD
that got mounted recently.  Shared subtree semantics provide the necessary
mechanism to accomplish the above.

It provides the necessary building blocks for features like per-user-namespace
and versioned filesystem.

2) Features
-----------

Shared subtree provides four different flavors of mounts; struct vfsmount to be
precise:

Shared mount의 양방향 전파

shared mount는 여러 mountpoint에 복제할 수 있으며, 모든 복제본은 같은 peer group에 속해 계속 동일한 전파 관계를 유지한다. `/mnt`를 shared로 만든 뒤 `/tmp`에 bind mount하면 두 위치의 기존 내용뿐 아니라 이후의 mount·umount 이벤트도 서로에게 전파된다.

예제에서 `/dev/sd0`를 `/tmp/a`에 마운트하면 `/mnt/a`에서도 `t1 t2 t3`을 볼 수 있다. 반대로 `/mnt/a`에 마운트해도 `/tmp/a`에 나타난다. 현재 `mount(8)`은 `--make-shared`를 직접 지원하므로 예전의 `smount` 예제 프로그램은 필요 없어 제거되었다.

a) A **shared mount** can be replicated to as many mountpoints and all the
   replicas continue to be exactly same.

   Here is an example:

   Let's say /mnt has a mount that is shared::

     # mount --make-shared /mnt

   .. note::
      mount(8) command now supports the --make-shared flag,
      so the sample 'smount' program is no longer needed and has been
      removed.

   ::

     # mount --bind /mnt /tmp

   The above command replicates the mount at /mnt to the mountpoint /tmp
   and the contents of both the mounts remain identical.

   ::

     #ls /mnt
     a b c

     #ls /tmp
     a b c

   Now let's say we mount a device at /tmp/a::

     # mount /dev/sd0  /tmp/a

     # ls /tmp/a
     t1 t2 t3

     # ls /mnt/a
     t1 t2 t3

   Note that the mount has propagated to the mount at /mnt as well.

   And the same is true even when /dev/sd0 is mounted on /mnt/a. The
   contents will be visible under /tmp/a too.

Slave mount의 단방향 전파

slave mount는 shared mount와 비슷하지만 mount·umount 이벤트가 master에서 slave 방향으로만 흐른다. 모든 slave에는 shared 상태인 master가 있다.

예제에서는 shared `/mnt`를 `/tmp`로 bind한 뒤 `/tmp`를 `--make-slave`로 바꾼다. `/mnt/a`에 마운트한 `/dev/sd0`는 `/tmp/a`에 전파되지만, slave인 `/tmp/b`에 마운트한 `/dev/sd1`은 master의 `/mnt/b`로 되돌아가지 않는다.

b) A **slave mount** is like a shared mount except that mount and umount events
   only propagate towards it.

   All slave mounts have a master mount which is a shared.

   Here is an example:

   Let's say /mnt has a mount which is shared::

     # mount --make-shared /mnt

   Let's bind mount /mnt to /tmp::

     # mount --bind /mnt /tmp

   the new mount at /tmp becomes a shared mount and it is a replica of
   the mount at /mnt.

   Now let's make the mount at /tmp; a slave of /mnt::

     # mount --make-slave /tmp

   let's mount /dev/sd0 on /mnt/a::

     # mount /dev/sd0 /mnt/a

     # ls /mnt/a
     t1 t2 t3

     # ls /tmp/a
     t1 t2 t3

   Note the mount event has propagated to the mount at /tmp

   However let's see what happens if we mount something on the mount at
   /tmp::

     # mount /dev/sd1 /tmp/b

     # ls /tmp/b
     s1 s2 s3

     # ls /mnt/b

   Note how the mount event has not propagated to the mount at
   /mnt

Private·unbindable과 상태 설정 명령

private mount는 전파 이벤트를 보내지도 받지도 않는 익숙한 기본 유형이다. unbindable mount는 private의 성질에 더해 bind mount의 원본으로 사용할 수도 없다.

따라서 `/mnt`에 `mount --make-unbindable /mnt`를 실행한 다음 이를 `/tmp`로 bind하려는 시도는 잘못된 연산으로 실패한다. util-linux의 `mount` 명령은 `--make-shared`, `--make-slave`, `--make-private`, `--make-unbindable`로 각 상태를 지정한다.

c) A **private mount** does not forward or receive propagation.

   This is the mount we are familiar with. Its the default type.


d) An **unbindable mount** is, as the name suggests, an unbindable private
   mount.

   let's say we have a mount at /mnt and we make it unbindable::

     # mount --make-unbindable /mnt

   Let's try to bind mount this mount somewhere else::

     # mount --bind /mnt /tmp mount: wrong fs type, bad option, bad
     superblock on /mnt, or too many mounted file systems

   Binding a unbindable mount is a invalid operation.


3) Setting mount states
-----------------------

The mount command (util-linux package) can be used to set mount
states::

    mount --make-shared mountpoint
    mount --make-slave mountpoint
    mount --make-private mountpoint
    mount --make-unbindable mountpoint

Namespace·개인 트리·버전 파일 사용 사례

첫 번째 사용 사례는 복제한 namespace에서도 새 CD mount를 보는 것이다. 관리자가 `/cdrom`을 자기 자신에 bind한 뒤 shared로 만들면, 자식 namespace의 `/cdrom`은 부모의 복제본이 된다. 이후 어느 namespace에서든 `/cdrom` 아래에 생긴 mount가 peer에 전파된다.

두 번째는 프로세스 자신의 mount는 감추되 시스템의 mount는 계속 받는 구성이다. 관리자는 전체 트리를 `mount --make-rshared /`로 공유하고, 새 namespace의 `/myprivatetree`만 `mount --make-rslave`로 바꾼다. 그러면 부모 쪽 변경은 자식에게 보이지만 자식의 사적 변경은 다른 namespace로 새지 않는다.

이 의미론은 사용자별 namespace에도 활용할 수 있다. namespace를 사용자 ID와 연결·해제할 수 있는 일급 객체로 만들고 PAM이 이를 지원한다면 사용자마다 요구에 맞는 mount view를 가질 수 있다.

버전 파일 시스템에서는 전체 mount tree를 `/view/v1`부터 `/view/v4`까지 `--rbind`한다. `/usr`의 기반 파일 시스템은 접근 경로를 보고 `/view/v3/usr/fs/namespace.c` 요청이 v3임을 판별해 해당 inode를 반환할 수 있다.

4) Use cases
------------

A) A process wants to clone its own namespace, but still wants to
   access the CD that got mounted recently.

   Solution:

   The system administrator can make the mount at /cdrom shared::

     mount --bind /cdrom /cdrom
     mount --make-shared /cdrom

   Now any process that clones off a new namespace will have a
   mount at /cdrom which is a replica of the same mount in the
   parent namespace.

   So when a CD is inserted and mounted at /cdrom that mount gets
   propagated to the other mount at /cdrom in all the other clone
   namespaces.

B) A process wants its mounts invisible to any other process, but
   still be able to see the other system mounts.

   Solution:

   To begin with, the administrator can mark the entire mount tree
   as shareable::

     mount --make-rshared /

   A new process can clone off a new namespace. And mark some part
   of its namespace as slave::

     mount --make-rslave /myprivatetree

   Hence forth any mounts within the /myprivatetree done by the
   process will not show up in any other namespace. However mounts
   done in the parent namespace under /myprivatetree still shows
   up in the process's namespace.


Apart from the above semantics this feature provides the
building blocks to solve the following problems:

C)  Per-user namespace

    The above semantics allows a way to share mounts across
    namespaces.  But namespaces are associated with processes. If
    namespaces are made first class objects with user API to
    associate/disassociate a namespace with userid, then each user
    could have his/her own namespace and tailor it to his/her
    requirements. This needs to be supported in PAM.

D)  Versioned files

    If the entire mount tree is visible at multiple locations, then
    an underlying versioning file system can return different
    versions of the file depending on the path used to access that
    file.

    An example is::

       mount --make-shared /
       mount --rbind / /view/v1
       mount --rbind / /view/v2
       mount --rbind / /view/v3
       mount --rbind / /view/v4

    and if /usr has a versioning filesystem mounted, then that
    mount appears at /view/v1/usr, /view/v2/usr, /view/v3/usr and
    /view/v4/usr too

    A user can request v3 version of the file /usr/fs/namespace.c
    by accessing /view/v3/usr/fs/namespace.c . The underlying
    versioning filesystem can then decipher that v3 version of the
    filesystem is being requested and return the corresponding
    inode.

전파 이벤트·peer group과 다섯 상태

이 문서에서 `vfsmount`와 mount라는 명사는 같은 뜻으로 사용한다. 전파 이벤트는 한 vfsmount에서 발생해 다른 vfsmount의 mount 또는 unmount 동작을 일으키는 사건이며, peer group은 서로에게 이벤트를 전파하는 vfsmount 집합이다.

shared mount는 peer group의 구성원이다. `/mnt`를 shared로 만들고 `/tmp`로 bind하면 두 mount가 같은 peer group이 되어 어느 쪽 아래의 변경도 다른 구성원에 반영된다.

slave mount는 전파 이벤트를 받지만 보내지 않는다. `mount --make-slave mount`는 shared mount를 slave로 바꿀 수 있다. 이때 peer group의 유일한 구성원이라 master가 없으면 private이 되며, 그렇지 않으면 일반적으로 shared 속성을 잃는다.

shared and slave 상태는 어떤 peer group의 slave이면서 동시에 자체 peer group을 가진다. master의 이벤트를 받아 자신의 peer와 slave로 전달한다. slave에 `--make-shared`를 실행하거나 shared mount 아래로 이동하면 이 상태가 될 수 있다.

private은 이벤트를 보내거나 받지 않으며, unbindable은 여기에 bind 복제 금지를 더한다.

5) Detailed semantics
---------------------
The section below explains the detailed semantics of
bind, rbind, move, mount, umount and clone-namespace operations.

.. Note::
   the word 'vfsmount' and the noun 'mount' have been used
   to mean the same thing, throughout this document.

a) Mount states

   A **propagation event** is defined as event generated on a vfsmount
   that leads to mount or unmount actions in other vfsmounts.

   A **peer group** is defined as a group of vfsmounts that propagate
   events to each other.

   A given mount can be in one of the following states:

   (1) Shared mounts

       A **shared mount** is defined as a vfsmount that belongs to a
       peer group.

       For example::

         mount --make-shared /mnt
         mount --bind /mnt /tmp

       The mount at /mnt and that at /tmp are both shared and belong
       to the same peer group. Anything mounted or unmounted under
       /mnt or /tmp reflect in all the other mounts of its peer
       group.


   (2) Slave mounts

       A **slave mount** is defined as a vfsmount that receives
       propagation events and does not forward propagation events.

       A slave mount as the name implies has a master mount from which
       mount/unmount events are received. Events do not propagate from
       the slave mount to the master.  Only a shared mount can be made
       a slave by executing the following command::

         mount --make-slave mount

       A shared mount that is made as a slave is no more shared unless
       modified to become shared.

   (3) Shared and Slave

       A vfsmount can be both **shared** as well as **slave**.  This state
       indicates that the mount is a slave of some vfsmount, and
       has its own peer group too.  This vfsmount receives propagation
       events from its master vfsmount, and also forwards propagation
       events to its 'peer group' and to its slave vfsmounts.

       Strictly speaking, the vfsmount is shared having its own
       peer group, and this peer-group is a slave of some other
       peer group.

       Only a slave vfsmount can be made as 'shared and slave' by
       either executing the following command::

         mount --make-shared mount

       or by moving the slave vfsmount under a shared vfsmount.

   (4) Private mount

       A **private mount** is defined as vfsmount that does not
       receive or forward any propagation events.

   (5) Unbindable mount

       A **unbindable mount** is defined as vfsmount that does not
       receive or forward any propagation events and cannot
       be bind mounted.

마운트 상태 전이표

다음 표는 현재 상태에 각 `--make-*` 명령을 적용한 결과다. shared mount를 slave로 바꿀 때 peer group에 자기 혼자뿐이면 연결할 master가 없어 private이 된다. 반대로 non-shared mount에 slave 명령을 적용해도 효과가 없다.

표에 없는 `mount --move`도 목적지 mount의 유형에 따라 상태를 바꿀 수 있으며, 그 세부 규칙은 move 의미론에서 설명한다.

`*` peer가 하나인 shared mount는 slave로 만들 master가 없어 자동으로 private이 된다. `**` non-shared mount를 slave로 만드는 명령은 상태를 바꾸지 않는다.

       State diagram:

       The state diagram below explains the state transition of a mount,
       in response to various commands::

            -----------------------------------------------------------------------
            |             |make-shared |  make-slave  | make-private |make-unbindab|
            --------------|------------|--------------|--------------|-------------|
            |shared       |shared      |*slave/private|   private    | unbindable  |
            |             |            |              |              |             |
            |-------------|------------|--------------|--------------|-------------|
            |slave        |shared      | **slave      |    private   | unbindable  |
            |             |and slave   |              |              |             |
            |-------------|------------|--------------|--------------|-------------|
            |shared       |shared      | slave        |    private   | unbindable  |
            |and slave    |and slave   |              |              |             |
            |-------------|------------|--------------|--------------|-------------|
            |private      |shared      |  **private   |    private   | unbindable  |
            |-------------|------------|--------------|--------------|-------------|
            |unbindable   |shared      |**unbindable  |    private   | unbindable  |
            ------------------------------------------------------------------------

            * if the shared mount is the only mount in its peer group, making it
            slave, makes it private automatically. Note that there is no master to
            which it can be slaved to.

            ** slaving a non-shared mount has no effect on the mount.

       Apart from the commands listed below, the 'move' operation also changes
       the state of a mount depending on type of the destination mount. Its
       explained in section 5d.

Bind 연산의 상태 조합

`mount --bind A/a B/b`에서 `A`는 원본 mount, `a`는 그 안의 dentry, `B`는 목적지 mount, `b`는 목적지 dentry다. 결과는 원본과 목적지의 전파 상태 조합에 따라 달라진다.

목적지 `B`가 shared이면 `A`의 clone `C`뿐 아니라 `B`가 전파하는 모든 위치에 `C1`부터 `Cn`까지 만들어진다. 이 새 전파 트리는 `B`의 전파 트리와 같은 모양이다. 원본이 shared이면 `C`의 peer group을 `A`와 합치고, private이면 새 clone들끼리 목적지와 같은 전파 구성을 만든다.

원본 `A`가 `Z`의 slave이면 shared 목적지에 생긴 `C`와 그 peer group도 `Z`의 slave가 되므로 `C`는 shared and slave 상태다. 원본이 unbindable이면 목적지가 shared인지와 관계없이 bind 자체가 잘못된 연산이다.

목적지가 non-shared라면 clone 하나만 `B/b`에 붙는다. shared 원본의 clone은 `A`의 peer group 구성원이 되고, `Z`의 slave 원본을 복제한 `C`는 `A`와 함께 `Z`의 slave가 된다. `A`와 `C`의 로컬 이벤트는 서로에게 전파되지 않는다.

b) Bind semantics

   Consider the following command::

     mount --bind A/a  B/b

   where 'A' is the source mount, 'a' is the dentry in the mount 'A', 'B'
   is the destination mount and 'b' is the dentry in the destination mount.

   The outcome depends on the type of mount of 'A' and 'B'. The table
   below contains quick reference::

            --------------------------------------------------------------------------
            |         BIND MOUNT OPERATION                                           |
            |************************************************************************|
            |source(A)->| shared      |       private  |       slave    | unbindable |
            | dest(B)  |              |                |                |            |
            |   |      |              |                |                |            |
            |   v      |              |                |                |            |
            |************************************************************************|
            |  shared  | shared       |     shared     | shared & slave |  invalid   |
            |          |              |                |                |            |
            |non-shared| shared       |      private   |      slave     |  invalid   |
            **************************************************************************

   Details:

   1. 'A' is a shared mount and 'B' is a shared mount. A new mount 'C'
      which is clone of 'A', is created. Its root dentry is 'a' . 'C' is
      mounted on mount 'B' at dentry 'b'. Also new mount 'C1', 'C2', 'C3' ...
      are created and mounted at the dentry 'b' on all mounts where 'B'
      propagates to. A new propagation tree containing 'C1',..,'Cn' is
      created. This propagation tree is identical to the propagation tree of
      'B'.  And finally the peer-group of 'C' is merged with the peer group
      of 'A'.

   2. 'A' is a private mount and 'B' is a shared mount. A new mount 'C'
      which is clone of 'A', is created. Its root dentry is 'a'. 'C' is
      mounted on mount 'B' at dentry 'b'. Also new mount 'C1', 'C2', 'C3' ...
      are created and mounted at the dentry 'b' on all mounts where 'B'
      propagates to. A new propagation tree is set containing all new mounts
      'C', 'C1', .., 'Cn' with exactly the same configuration as the
      propagation tree for 'B'.

   3. 'A' is a slave mount of mount 'Z' and 'B' is a shared mount. A new
      mount 'C' which is clone of 'A', is created. Its root dentry is 'a' .
      'C' is mounted on mount 'B' at dentry 'b'. Also new mounts 'C1', 'C2',
      'C3' ... are created and mounted at the dentry 'b' on all mounts where
      'B' propagates to. A new propagation tree containing the new mounts
      'C','C1',..  'Cn' is created. This propagation tree is identical to the
      propagation tree for 'B'. And finally the mount 'C' and its peer group
      is made the slave of mount 'Z'.  In other words, mount 'C' is in the
      state 'slave and shared'.

   4. 'A' is a unbindable mount and 'B' is a shared mount. This is a
      invalid operation.

   5. 'A' is a private mount and 'B' is a non-shared(private or slave or
      unbindable) mount. A new mount 'C' which is clone of 'A', is created.
      Its root dentry is 'a'. 'C' is mounted on mount 'B' at dentry 'b'.

   6. 'A' is a shared mount and 'B' is a non-shared mount. A new mount 'C'
      which is a clone of 'A' is created. Its root dentry is 'a'. 'C' is
      mounted on mount 'B' at dentry 'b'.  'C' is made a member of the
      peer-group of 'A'.

   7. 'A' is a slave mount of mount 'Z' and 'B' is a non-shared mount. A
      new mount 'C' which is a clone of 'A' is created. Its root dentry is
      'a'.  'C' is mounted on mount 'B' at dentry 'b'. Also 'C' is set as a
      slave mount of 'Z'. In other words 'A' and 'C' are both slave mounts of
      'Z'.  All mount/unmount events on 'Z' propagates to 'A' and 'C'. But
      mount/unmount on 'A' do not propagate anywhere else. Similarly
      mount/unmount on 'C' do not propagate anywhere else.

   8. 'A' is a unbindable mount and 'B' is a non-shared mount. This is a
      invalid operation. A unbindable mount cannot be bind mounted.

Rbind와 unbindable 가지치기

rbind는 bind를 지정 mount 하나가 아니라 그 아래 mount tree 전체에 적용한다. 원본 트리에 unbindable mount가 있으면 새 위치에서는 그 mount와 아래 subtree가 복제 대상에서 잘려 나간다.

예제 트리 `A` 아래에 `B`, `C`가 있고 각각 두 자식이 있을 때 `C`만 unbindable이라면, `Z`로 rbind한 결과는 `Z/A'/B'`와 `D'`, `E'`만 남는다. `C`, `F`, `G` 가지 전체는 복제되지 않는다.

c) Rbind semantics

   rbind is same as bind. Bind replicates the specified mount.  Rbind
   replicates all the mounts in the tree belonging to the specified mount.
   Rbind mount is bind mount applied to all the mounts in the tree.

   If the source tree that is rbind has some unbindable mounts,
   then the subtree under the unbindable mount is pruned in the new
   location.

   eg:

   let's say we have the following mount tree::

                A
              /   \
              B   C
             / \ / \
             D E F G

   Let's say all the mount except the mount C in the tree are
   of a type other than unbindable.

   If this tree is rbound to say Z

   We will have the following tree at the new location::

                Z
                |
                A'
               /
              B'                Note how the tree under C is pruned
             / \                in the new location.
            D' E'

Move 연산과 목적지 전파

`mount --move A B/b`는 원본 mount `A`를 목적지 `B`의 dentry `b`로 옮긴다. shared mount 아래에 놓여 있는 mount를 원본으로 이동하는 것은 허용되지 않는다.

shared 목적지에서는 `A`를 `B/b`에 붙이는 동시에 `B`의 모든 전파 대상에 `A1`부터 `An`을 생성한다. 원본이 shared이면 새 전파 트리를 기존 `A` 전파 트리에 덧붙인다. private 원본은 shared로 바뀌고, slave 원본은 master `Z`의 slave 관계를 유지하면서 shared도 된다.

unbindable 원본을 shared 목적지로 옮기면 목적지 전파 때문에 clone이 필요하지만 unbindable은 복제할 수 없으므로 연산이 실패한다. 목적지가 non-shared이면 추가 clone이 필요 없어 private·shared·slave·unbindable 원본 모두 자신의 상태를 유지한 채 이동할 수 있다.

d) Move semantics

   Consider the following command::

     mount --move A  B/b

   where 'A' is the source mount, 'B' is the destination mount and 'b' is
   the dentry in the destination mount.

   The outcome depends on the type of the mount of 'A' and 'B'. The table
   below is a quick reference::

            ---------------------------------------------------------------------------
            |                   MOVE MOUNT OPERATION                                 |
            |**************************************************************************
            | source(A)->| shared      |       private  |       slave    | unbindable |
            | dest(B)  |               |                |                |            |
            |   |      |               |                |                |            |
            |   v      |               |                |                |            |
            |**************************************************************************
            |  shared  | shared        |     shared     |shared and slave|  invalid   |
            |          |               |                |                |            |
            |non-shared| shared        |      private   |    slave       | unbindable |
            ***************************************************************************

   .. Note:: moving a mount residing under a shared mount is invalid.

   Details follow:

   1. 'A' is a shared mount and 'B' is a shared mount.  The mount 'A' is
      mounted on mount 'B' at dentry 'b'.  Also new mounts 'A1', 'A2'...'An'
      are created and mounted at dentry 'b' on all mounts that receive
      propagation from mount 'B'. A new propagation tree is created in the
      exact same configuration as that of 'B'. This new propagation tree
      contains all the new mounts 'A1', 'A2'...  'An'.  And this new
      propagation tree is appended to the already existing propagation tree
      of 'A'.

   2. 'A' is a private mount and 'B' is a shared mount. The mount 'A' is
      mounted on mount 'B' at dentry 'b'. Also new mount 'A1', 'A2'... 'An'
      are created and mounted at dentry 'b' on all mounts that receive
      propagation from mount 'B'. The mount 'A' becomes a shared mount and a
      propagation tree is created which is identical to that of
      'B'. This new propagation tree contains all the new mounts 'A1',
      'A2'...  'An'.

   3. 'A' is a slave mount of mount 'Z' and 'B' is a shared mount.  The
      mount 'A' is mounted on mount 'B' at dentry 'b'.  Also new mounts 'A1',
      'A2'... 'An' are created and mounted at dentry 'b' on all mounts that
      receive propagation from mount 'B'. A new propagation tree is created
      in the exact same configuration as that of 'B'. This new propagation
      tree contains all the new mounts 'A1', 'A2'...  'An'.  And this new
      propagation tree is appended to the already existing propagation tree of
      'A'.  Mount 'A' continues to be the slave mount of 'Z' but it also
      becomes 'shared'.

   4. 'A' is a unbindable mount and 'B' is a shared mount. The operation
      is invalid. Because mounting anything on the shared mount 'B' can
      create new mounts that get mounted on the mounts that receive
      propagation from 'B'.  And since the mount 'A' is unbindable, cloning
      it to mount at other mountpoints is not possible.

   5. 'A' is a private mount and 'B' is a non-shared(private or slave or
      unbindable) mount. The mount 'A' is mounted on mount 'B' at dentry 'b'.

   6. 'A' is a shared mount and 'B' is a non-shared mount.  The mount 'A'
      is mounted on mount 'B' at dentry 'b'.  Mount 'A' continues to be a
      shared mount.

   7. 'A' is a slave mount of mount 'Z' and 'B' is a non-shared mount.
      The mount 'A' is mounted on mount 'B' at dentry 'b'.  Mount 'A'
      continues to be a slave mount of mount 'Z'.

   8. 'A' is a unbindable mount and 'B' is a non-shared mount. The mount
      'A' is mounted on mount 'B' at dentry 'b'. Mount 'A' continues to be a
      unbindable mount.

일반 mount와 unmount 전파

`mount device B/b`는 원본 mount가 항상 private이라는 점을 제외하면 bind 연산과 같다. 따라서 목적지 `B`가 shared이면 새 device mount가 `B`의 전파 대상들에도 복제된다.

`umount A`에서 `A`가 shared `B`의 dentry `b`에 붙어 있으면, `B` 및 그 전파 대상의 같은 `b`에 가장 최근에 쌓인 mount 가운데 하위 mount가 없는 것들을 함께 해제한다.

예제에서 peer인 `B1`, `B2`, `B3`의 `b`에 먼저 `A1..A3`, 다음으로 `C1..C3`을 올린 뒤 `C1`을 해제하면 가장 최근 층인 `C1`, `C2`, `C3`이 대상이다. `C2`나 `C3`에 자식 mount가 있으면 그것만 남고 나머지는 해제된다. 그러나 직접 요청한 `C1` 자체에 자식이 있으면 전체 umount 연산이 실패한다.

e) Mount semantics

   Consider the following command::

     mount device  B/b

   'B' is the destination mount and 'b' is the dentry in the destination
   mount.

   The above operation is the same as bind operation with the exception
   that the source mount is always a private mount.


f) Unmount semantics

   Consider the following command::

     umount A

   where 'A' is a mount mounted on mount 'B' at dentry 'b'.

   If mount 'B' is shared, then all most-recently-mounted mounts at dentry
   'b' on mounts that receive propagation from mount 'B' and does not have
   sub-mounts within them are unmounted.

   Example: Let's say 'B1', 'B2', 'B3' are shared mounts that propagate to
   each other.

   let's say 'A1', 'A2', 'A3' are first mounted at dentry 'b' on mount
   'B1', 'B2' and 'B3' respectively.

   let's say 'C1', 'C2', 'C3' are next mounted at the same dentry 'b' on
   mount 'B1', 'B2' and 'B3' respectively.

   if 'C1' is unmounted, all the mounts that are most-recently-mounted on
   'B1' and on the mounts that 'B1' propagates-to are unmounted.

   'B1' propagates to 'B2' and 'B3'. And the most recently mounted mount
   on 'B2' at dentry 'b' is 'C2', and that of mount 'B3' is 'C3'.

   So all 'C1', 'C2' and 'C3' should be unmounted.

   If any of 'C2' or 'C3' has some child mounts, then that mount is not
   unmounted, but all other mounts are unmounted. However if 'C1' is told
   to be unmounted and 'C1' has some sub-mounts, the umount operation is
   failed entirely.

Mount namespace 복제 규칙

복제된 namespace는 부모 namespace의 모든 mount를 가진다. 부모의 대응 mount를 `A`, 자식의 clone을 `B`라고 하면 전파 상태도 원칙적으로 보존된다.

g) Clone Namespace

   A cloned namespace contains all the mounts as that of the parent
   namespace.

   Let's say 'A' and 'B' are the corresponding mounts in the parent and the
   child namespace.

   If 'A' is shared, then 'B' is also shared and 'A' and 'B' propagate to
   each other.

   If 'A' is a slave mount of 'Z', then 'B' is also the slave mount of
   'Z'.

   If 'A' is a private mount, then 'B' is a private mount too.

   If 'A' is unbindable mount, then 'B' is a unbindable mount too.

경계 조건을 묻는 세 가지 퀴즈

퀴즈 A는 shared `/mnt`의 bind 복제 `/tmp`를 `/mnt/1`로 이동했을 때 `/mnt`, `/mnt/1`, `/mnt/1/1`의 내용이 모두 같은지, 아니면 앞의 둘만 같은지를 묻는다. shared 목적지로 move할 때 생기는 전파와 재귀적 자기 포함 가능성을 점검하는 문제다.

퀴즈 B는 루트 전체를 rshared로 만든 뒤 `/`를 `/v/1`에 rbind했을 때 `/v/1/v/1`에 무엇이 보이는지 묻는다. shared 트리를 자기 하위로 재귀 복제할 때의 결과를 생각하게 한다.

퀴즈 C는 여러 bind와 `--make-slave`, `--make-shared`를 조합해 `A → B → C` slave 연쇄를 만든다. `/bin`을 `A`의 `/tmp/test`에 bind했을 때 이벤트가 `B`, `C`까지 전파되는지와 `/mnt/1/test`의 내용을 묻는다.

6) Quiz
-------

A. What is the result of the following command sequence?

   ::

       mount --bind /mnt /mnt
       mount --make-shared /mnt
       mount --bind /mnt /tmp
       mount --move /tmp /mnt/1

   what should be the contents of /mnt /mnt/1 /mnt/1/1 should be?
   Should they all be identical? or should /mnt and /mnt/1 be
   identical only?


B. What is the result of the following command sequence?

   ::

       mount --make-rshared /
       mkdir -p /v/1
       mount --rbind / /v/1

   what should be the content of /v/1/v/1 be?


C. What is the result of the following command sequence?

   ::

       mount --bind /mnt /mnt
       mount --make-shared /mnt
       mkdir -p /mnt/1/2/3 /mnt/1/test
       mount --bind /mnt/1 /tmp
       mount --make-slave /mnt
       mount --make-shared /mnt
       mount --bind /mnt/1/2 /tmp1
       mount --make-slave /mnt

   At this point we have the first mount at /tmp and
   its root dentry is 1. Let's call this mount 'A'
   And then we have a second mount at /tmp1 with root
   dentry 2. Let's call this mount 'B'
   Next we have a third mount at /mnt with root dentry
   mnt. Let's call this mount 'C'

   'B' is the slave of 'A' and 'C' is a slave of 'B'
   A -> B -> C

   at this point if we execute the following command::

     mount --bind /bin /tmp/test

   The mount is attempted on 'A'

   will the mount propagate to 'B' and 'C' ?

   what would be the contents of
   /mnt/1/test be?

7) FAQ

FAQ: bind mount와 exportfs

bind mount는 symbolic link와 달리 다른 쪽 mount가 해제되거나 이동해도 계속 존재한다. symbolic link는 대상 경로가 사라지거나 바뀌면 낡은 링크가 될 수 있다.

exportfs는 shared subtree 기능 일부를 흉내 내기에는 지나치게 무거운 수단이며, 특히 master에서 slave로만 흐르는 단방향 slave 의미론을 exportfs로 구현할 뚜렷한 방법이 없다.

------

1. Why is bind mount needed? How is it different from symbolic links?

   symbolic links can get stale if the destination mount gets
   unmounted or moved. Bind mounts continue to exist even if the
   other mount is unmounted or moved.

2. Why can't the shared subtree be implemented using exportfs?

   exportfs is a heavyweight way of accomplishing part of what
   shared subtree can do. I cannot imagine a way to implement the
   semantics of slave mount using exportfs?

3. Why is unbindable mount needed?

FAQ: 재귀 rbind의 폭발적 증가

unbindable mount는 같은 subtree 내부의 여러 위치에 mount tree를 반복 복제할 때 불필요한 재귀 복제를 잘라 내는 데 필요하다. 한 트리를 자기 하위에 `n`번 rbind하면 생성 mount 수가 `n`에 대해 급격히 증가한다.

초기 `/root`에는 `tmp`와 `usr`이 있고 vfsmount는 하나뿐이다. `/root`를 `/tmp/m1`에 rbind하면 복제본 안의 `tmp/m1`까지 다시 포함되어 vfsmount가 2개가 된다. 다시 `/tmp/m2`에 rbind하면 기존 복제 위치들까지 중첩 복제되어 6개가 되고, `/tmp/m3` 단계에서는 24개가 된다.

문서가 제시하는 점화식은 `V[i] = i*V[i-1]`이다. 매 단계에서 일련의 umount로 불필요한 가지를 제거할 수 있지만, unbindable로 애초에 clone 대상에서 제외하는 편이 낫다.

   Let's say we want to replicate the mount tree at multiple
   locations within the same subtree.

   if one rbind mounts a tree within the same subtree 'n' times
   the number of mounts created is an exponential function of 'n'.
   Having unbindable mount can help prune the unneeded bind
   mounts. Here is an example.

   step 1:
      let's say the root tree has just two directories with
      one vfsmount::

                                    root
                                   /    \
                                  tmp    usr

      And we want to replicate the tree at multiple
      mountpoints under /root/tmp

   step 2:
      ::


                        mount --make-shared /root

                        mkdir -p /tmp/m1

                        mount --rbind /root /tmp/m1

      the new tree now looks like this::

                                    root
                                   /    \
                                 tmp    usr
                                /
                               m1
                              /  \
                             tmp  usr
                             /
                            m1

      it has two vfsmounts

   step 3:
      ::

                            mkdir -p /tmp/m2
                            mount --rbind /root /tmp/m2

      the new tree now looks like this::

                                      root
                                     /    \
                                   tmp     usr
                                  /    \
                                m1       m2
                               / \       /  \
                             tmp  usr   tmp  usr
                             / \          /
                            m1  m2      m1
                                / \     /  \
                              tmp usr  tmp   usr
                              /        / \
                             m1       m1  m2
                            /  \
                          tmp   usr
                          /  \
                         m1   m2

                    it has 6 vfsmounts

   step 4:
      ::

                          mkdir -p /tmp/m3
                          mount --rbind /root /tmp/m3

      I won't draw the tree..but it has 24 vfsmounts


   at step i the number of vfsmounts is V[i] = i*V[i-1].
   This is an exponential function. And this tree has way more
   mounts than what we really needed in the first place.

   One could use a series of umount at each step to prune
   out the unneeded mounts. But there is a better solution.
   Unclonable mounts come in handy here.

Unbindable로 재귀 복제 가지치기

개선된 구성은 먼저 `/root/tmp`를 자기 자신에 bind해 별도 mountpoint로 만든다. 이어 `/root`를 재귀 shared로 만들고 `/root/tmp`는 unbindable로 바꾼다.

이제 `/root`를 `/tmp/m1`, `/tmp/m2`, `/tmp/m3`에 차례로 rbind해도 원본의 unbindable `tmp` 아래 subtree는 복제본 안에서 더 깊게 복제되지 않는다. 결과 트리는 root 아래 `m1`, `m2`, `m3`이 나란히 있고, 각 복제본에는 `tmp`와 `usr` 한 쌍만 있는 선형적인 구조가 된다.

   step 1:
      let's say the root tree has just two directories with
      one vfsmount::

                                    root
                                   /    \
                                  tmp    usr

         How do we set up the same tree at multiple locations under
         /root/tmp

   step 2:
      ::


                        mount --bind /root/tmp /root/tmp

                        mount --make-rshared /root
                        mount --make-unbindable /root/tmp

                        mkdir -p /tmp/m1

                        mount --rbind /root /tmp/m1

      the new tree now looks like this::

                                    root
                                   /    \
                                 tmp    usr
                                /
                               m1
                              /  \
                             tmp  usr

   step 3:
      ::

                            mkdir -p /tmp/m2
                            mount --rbind /root /tmp/m2

      the new tree now looks like this::

                                    root
                                   /    \
                                 tmp    usr
                                /   \
                               m1     m2
                              /  \     / \
                             tmp  usr tmp usr

   step 4:
      ::

                            mkdir -p /tmp/m3
                            mount --rbind /root /tmp/m3

      the new tree now looks like this::

                                          root
                                      /           \
                                     tmp           usr
                                 /    \    \
                               m1     m2     m3
                              /  \     / \    /  \
                             tmp  usr tmp usr tmp usr

`struct vfsmount` 전파 자료구조

구현은 `struct vfsmount`에 전파 관계를 나타내는 필드를 둔다. `mnt_share`는 서로 이벤트를 보내고 받는 peer들을, `mnt_slave_list`는 이 mount가 이벤트를 보내는 slave들을, `mnt_slave`는 같은 master를 둔 slave들을 연결한다. `mnt_master`는 이벤트를 받는 원천 master를 가리킨다.

`mnt_flags`에는 shared 상태를 나타내는 `MNT_SHARE`와 복제 불가를 나타내는 `MNT_UNCLONABLE`이 추가된다. 한 peer group의 shared mount는 `mnt_share`를 통해 원형 목록을 이룬다.

같은 `mnt_master`를 가진 vfsmount들은 master의 `mnt_slave_list`를 머리로 하고 `mnt_slave`를 따라가는 원형 목록을 이룬다. `mnt_master`는 master peer group의 임의 구성원을 가리킬 수 있으므로, peer group의 모든 직접 slave를 찾으려면 모든 구성원의 `mnt_slave_list`를 순회해야 한다.

개념적으로 slave들은 하나의 집합이며 개별 목록에 어떻게 분산됐는지는 전파 의미나 트리 변경에 영향을 주지 않는다. 한 peer group의 모든 vfsmount는 같은 `mnt_master`를 가지며, non-NULL이면 slave list에서 연속된 정렬 구간을 이룬다.

8) Implementation
-----------------

A) Datastructure

   Several new fields are introduced to struct vfsmount:

   ->mnt_share
           Links together all the mount to/from which this vfsmount
           send/receives propagation events.

   ->mnt_slave_list
           Links all the mounts to which this vfsmount propagates
           to.

   ->mnt_slave
           Links together all the slaves that its master vfsmount
           propagates to.

   ->mnt_master
           Points to the master vfsmount from which this vfsmount
           receives propagation.

   ->mnt_flags
           Takes two more flags to indicate the propagation status of
           the vfsmount.  MNT_SHARE indicates that the vfsmount is a shared
           vfsmount.  MNT_UNCLONABLE indicates that the vfsmount cannot be
           replicated.

   All the shared vfsmounts in a peer group form a cyclic list through
   ->mnt_share.

   All vfsmounts with the same ->mnt_master form on a cyclic list anchored
   in ->mnt_master->mnt_slave_list and going through ->mnt_slave.

   ->mnt_master can point to arbitrary (and possibly different) members
   of master peer group.  To find all immediate slaves of a peer group
   you need to go through _all_ ->mnt_slave_list of its members.
   Conceptually it's just a single set - distribution among the
   individual lists does not affect propagation or the way propagation
   tree is modified by operations.

   All vfsmounts in a peer group have the same ->mnt_master.  If it is
   non-NULL, they form a contiguous (ordered) segment of slave list.

Peer와 slave로 이루어진 전파 트리

그림의 `A`, `B`, `C`, `D`는 서로 전파하는 하나의 shared peer group이다. `A`에는 slave `E`, `F`, `G`가 있고, `C`에는 `J`, `K`, `D`에는 `H`, `I`가 있다. `E`와 `K`도 서로 shared이며, `K`에는 다시 slave `M`, `L`, `N`이 있다.

`A.mnt_share`는 `B`, `C`, `D`의 `mnt_share`와 연결된다. `A.mnt_slave_list`는 `E`, `K`, `F`, `G`의 `mnt_slave`를 연결하고 이들의 `mnt_master`는 `A`를 가리킨다. 마찬가지로 `K`의 slave list는 `M`, `L`, `N`, `C`의 목록은 `J`, `K`, `D`의 목록은 `H`, `I`를 연결한다.

겉보기에는 forest처럼 보이지만 shared mount 전체를 `pnode`라는 하나의 개념적 노드로 묶으면 트리다. 이 propagation tree는 파일 경로의 mount tree와 서로 직교하는 별도 관계다.

   A example propagation tree looks as shown in the figure below.

   .. note::
      Though it looks like a forest, if we consider all the shared
      mounts as a conceptual entity called 'pnode', it becomes a tree.

   ::


                        A <--> B <--> C <---> D
                       /|\            /|      |\
                      / F G          J K      H I
                     /
                    E<-->K
                        /|\
                       M L N

   In the above figure  A,B,C and D all are shared and propagate to each
   other.   'A' has got 3 slave mounts 'E' 'F' and 'G' 'C' has got 2 slave
   mounts 'J' and 'K'  and  'D' has got two slave mounts 'H' and 'I'.
   'E' is also shared with 'K' and they propagate to each other.  And
   'K' has 3 slaves 'M', 'L' and 'N'

   A's ->mnt_share links with the ->mnt_share of 'B' 'C' and 'D'

   A's ->mnt_slave_list links with ->mnt_slave of 'E', 'K', 'F' and 'G'

   E's ->mnt_share links with ->mnt_share of K

   'E', 'K', 'F', 'G' have their ->mnt_master point to struct vfsmount of 'A'

   'M', 'L', 'N' have their ->mnt_master point to struct vfsmount of 'K'

   K's ->mnt_slave_list links with ->mnt_slave of 'M', 'L' and 'N'

   C's ->mnt_slave_list links with ->mnt_slave of 'J' and 'K'

   J and K's ->mnt_master points to struct vfsmount of C

   and finally D's ->mnt_slave_list links with ->mnt_slave of 'H' and 'I'

   'H' and 'I' have their ->mnt_master pointing to struct vfsmount of 'D'.


   NOTE: The propagation tree is orthogonal to the mount tree.

잠금과 prepare·commit·abort 알고리즘

`mnt_share`, `mnt_slave`, `mnt_slave_list`, `mnt_master`는 `namespace_sem`으로 보호한다. 수정할 때는 exclusive, 읽을 때는 shared로 잡는다. 보통 `mnt_flags` 변경은 `vfsmount_lock`으로 직렬화한다.

예외는 `do_add_mount()`와 `clone_mnt()`다. 전자는 아직 어떤 shared 자료구조에도 공개되지 않은 vfsmount를 바꾼다. 후자는 `namespace_sem`을 잡고 있으며 해당 vfsmount의 참조는 이 semaphore 없이는 순회할 수 없는 목록에만 있다.

구현의 핵심은 rbind와 move이며 `attach_recursive_mnt()`와 `propagate_mnt()`에서 세 단계로 나뉜다. prepare 단계는 원본 트리의 각 mount마다 목적지의 모든 전파 수신 mount에 붙일 mount tree를 필요한 수만큼 생성한다. 아직 붙이지 않고 `mnt_parent`와 `mnt_mountpoint`만 기록한 뒤 목적지와 같은 propagation tree로 연결한다.

prepare가 성공하면 원본 트리의 mount 수 `n`개에 대응하는 새 propagation tree가 생기고, 목적지가 전파하는 mount 수 `m`개에 대응하는 새 mount tree가 생긴다. 메모리 할당이 실패하면 abort로 간다. commit은 각 mount tree를 해당 목적지에 붙이고, abort는 새로 만든 tree를 모두 삭제한다.

전파 관련 기능은 `pnode.c`에 있다. 문서 이력은 Ram Pai가 만든 version 0.1과 Al Viro의 의견을 반영한 version 0.2를 기록한다.

B) Locking:

   ->mnt_share, ->mnt_slave, ->mnt_slave_list, ->mnt_master are protected
   by namespace_sem (exclusive for modifications, shared for reading).

   Normally we have ->mnt_flags modifications serialized by vfsmount_lock.
   There are two exceptions: do_add_mount() and clone_mnt().
   The former modifies a vfsmount that has not been visible in any shared
   data structures yet.
   The latter holds namespace_sem and the only references to vfsmount
   are in lists that can't be traversed without namespace_sem.

C) Algorithm:

   The crux of the implementation resides in rbind/move operation.

   The overall algorithm breaks the operation into 3 phases: (look at
   attach_recursive_mnt() and propagate_mnt())

   1. Prepare phase.

      For each mount in the source tree:

      a) Create the necessary number of mount trees to
         be attached to each of the mounts that receive
         propagation from the destination mount.
      b) Do not attach any of the trees to its destination.
         However note down its ->mnt_parent and ->mnt_mountpoint
      c) Link all the new mounts to form a propagation tree that
         is identical to the propagation tree of the destination
         mount.

      If this phase is successful, there should be 'n' new
      propagation trees; where 'n' is the number of mounts in the
      source tree.  Go to the commit phase

      Also there should be 'm' new mount trees, where 'm' is
      the number of mounts to which the destination mount
      propagates to.

      If any memory allocations fail, go to the abort phase.

   2. Commit phase.

      Attach each of the mount trees to their corresponding
      destination mounts.

   3. Abort phase.

      Delete all the newly created trees.

   .. Note::
      all the propagation related functionality resides in the file pnode.c


------------------------------------------------------------------------

version 0.1  (created the initial document, Ram Pai [email protected])

version 0.2  (Incorporated comments from Al Viro)