요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
=============
FUSE Overview
=============
Definitions
===========
Userspace filesystem:
A filesystem in which data and metadata are provided by an ordinary
userspace process. The filesystem can be accessed normally through
the kernel interface.
Filesystem daemon:
The process(es) providing the data and metadata of the filesystem.
Non-privileged mount (or user mount):
A userspace filesystem mounted by a non-privileged (non-root) user.
The filesystem daemon is running with the privileges of the mounting
user. NOTE: this is not the same as mounts allowed with the "user"
option in /etc/fstab, which is not discussed here.
Filesystem connection:
A connection between the filesystem daemon and the kernel. The
connection exists until either the daemon dies, or the filesystem is
umounted. Note that detaching (or lazy umounting) the filesystem
does *not* break the connection, in this case it will exist until
the last reference to the filesystem is released.
Mount owner:
The user who does the mounting.
User:
The user who is performing filesystem operations.
What is FUSE?
=============
FUSE is a userspace filesystem framework. It consists of a kernel
module (fuse.ko), a userspace library (libfuse.*) and a mount utility
(fusermount).
One of the most important features of FUSE is allowing secure,
non-privileged mounts. This opens up new possibilities for the use of
filesystems. A good example is sshfs: a secure network filesystem
using the sftp protocol.
The userspace library and utilities are available from the
`FUSE homepage: <https://github.com/libfuse/>`_
Filesystem type
===============
The filesystem type given to mount(2) can be one of the following:
fuse
This is the usual way to mount a FUSE filesystem. The first
argument of the mount system call may contain an arbitrary string,
which is not interpreted by the kernel.
fuseblk
The filesystem is block device based. The first argument of the
mount system call is interpreted as the name of the device.
Mount options
=============
fd=N
The file descriptor to use for communication between the userspace
filesystem and the kernel. The file descriptor must have been
obtained by opening the FUSE device ('/dev/fuse').
rootmode=M
The file mode of the filesystem's root in octal representation.
user_id=N
The numeric user id of the mount owner.
group_id=N
The numeric group id of the mount owner.
default_permissions
By default FUSE doesn't check file access permissions, the
filesystem is free to implement its access policy or leave it to
the underlying file access mechanism (e.g. in case of network
filesystems). This option enables permission checking, restricting
access based on file mode. It is usually useful together with the
'allow_other' mount option.
allow_other
This option overrides the security measure restricting file access
to the user mounting the filesystem. This option is by default only
allowed to root, but this restriction can be removed with a
(userspace) configuration option.
max_read=N
With this option the maximum size of read operations can be set.
The default is infinite. Note that the size of read requests is
limited anyway to 32 pages (which is 128kbyte on i386).
blksize=N
Set the block size for the filesystem. The default is 512. This
option is only valid for 'fuseblk' type mounts.
Control filesystem
==================
There's a control filesystem for FUSE, which can be mounted by::
mount -t fusectl none /sys/fs/fuse/connections
Mounting it under the '/sys/fs/fuse/connections' directory makes it
backwards compatible with earlier versions.
Under the fuse control filesystem each connection has a directory
named by a unique number.
For each connection the following files exist within this directory:
waiting
The number of requests which are waiting to be transferred to
userspace or being processed by the filesystem daemon. If there is
no filesystem activity and 'waiting' is non-zero, then the
filesystem is hung or deadlocked.
abort
Writing anything into this file will abort the filesystem
connection. This means that all waiting requests will be aborted an
error returned for all aborted and new requests.
max_background
The maximum number of background requests that can be outstanding
at a time. When the number of background requests reaches this limit,
further requests will be blocked until some are completed, potentially
causing I/O operations to stall.
congestion_threshold
The threshold of background requests at which the kernel considers
the filesystem to be congested. When the number of background requests
exceeds this value, the kernel will skip asynchronous readahead
operations, reducing read-ahead optimizations but preserving essential
I/O, as well as suspending non-synchronous writeback operations
(WB_SYNC_NONE), delaying page cache flushing to the filesystem.
Only the owner of the mount may read or write these files.
Interrupting filesystem operations
##################################
If a process issuing a FUSE filesystem request is interrupted, the
following will happen:
- If the request is not yet sent to userspace AND the signal is
fatal (SIGKILL or unhandled fatal signal), then the request is
dequeued and returns immediately.
- If the request is not yet sent to userspace AND the signal is not
fatal, then an interrupted flag is set for the request. When
the request has been successfully transferred to userspace and
this flag is set, an INTERRUPT request is queued.
- If the request is already sent to userspace, then an INTERRUPT
request is queued.
INTERRUPT requests take precedence over other requests, so the
userspace filesystem will receive queued INTERRUPTs before any others.
The userspace filesystem may ignore the INTERRUPT requests entirely,
or may honor them by sending a reply to the *original* request, with
the error set to EINTR.
It is also possible that there's a race between processing the
original request and its INTERRUPT request. There are two possibilities:
1. The INTERRUPT request is processed before the original request is
processed
2. The INTERRUPT request is processed after the original request has
been answered
If the filesystem cannot find the original request, it should wait for
some timeout and/or a number of new requests to arrive, after which it
should reply to the INTERRUPT request with an EAGAIN error. In case
1) the INTERRUPT request will be requeued. In case 2) the INTERRUPT
reply will be ignored.
Aborting a filesystem connection
================================
It is possible to get into certain situations where the filesystem is
not responding. Reasons for this may be:
a) Broken userspace filesystem implementation
b) Network connection down
c) Accidental deadlock
d) Malicious deadlock
(For more on c) and d) see later sections)
In either of these cases it may be useful to abort the connection to
the filesystem. There are several ways to do this:
- Kill the filesystem daemon. Works in case of a) and b)
- Kill the filesystem daemon and all users of the filesystem. Works
in all cases except some malicious deadlocks
- Use forced umount (umount -f). Works in all cases but only if
filesystem is still attached (it hasn't been lazy unmounted)
- Abort filesystem through the FUSE control filesystem. Most
powerful method, always works.
How do non-privileged mounts work?
==================================
Since the mount() system call is a privileged operation, a helper
program (fusermount) is needed, which is installed setuid root.
The implication of providing non-privileged mounts is that the mount
owner must not be able to use this capability to compromise the
system. Obvious requirements arising from this are:
A) mount owner should not be able to get elevated privileges with the
help of the mounted filesystem
B) mount owner should not get illegitimate access to information from
other users' and the super user's processes
C) mount owner should not be able to induce undesired behavior in
other users' or the super user's processes
How are requirements fulfilled?
===============================
A) The mount owner could gain elevated privileges by either:
1. creating a filesystem containing a device file, then opening this device
2. creating a filesystem containing a suid or sgid application, then executing this application
The solution is not to allow opening device files and ignore
setuid and setgid bits when executing programs. To ensure this
fusermount always adds "nosuid" and "nodev" to the mount options
for non-privileged mounts.
B) If another user is accessing files or directories in the
filesystem, the filesystem daemon serving requests can record the
exact sequence and timing of operations performed. This
information is otherwise inaccessible to the mount owner, so this
counts as an information leak.
The solution to this problem will be presented in point 2) of C).
C) There are several ways in which the mount owner can induce
undesired behavior in other users' processes, such as:
1) mounting a filesystem over a file or directory which the mount
owner could otherwise not be able to modify (or could only
make limited modifications).
This is solved in fusermount, by checking the access
permissions on the mountpoint and only allowing the mount if
the mount owner can do unlimited modification (has write
access to the mountpoint, and mountpoint is not a "sticky"
directory)
2) Even if 1) is solved the mount owner can change the behavior
of other users' processes.
i) It can slow down or indefinitely delay the execution of a
filesystem operation creating a DoS against the user or the
whole system. For example a suid application locking a
system file, and then accessing a file on the mount owner's
filesystem could be stopped, and thus causing the system
file to be locked forever.
ii) It can present files or directories of unlimited length, or
directory structures of unlimited depth, possibly causing a
system process to eat up diskspace, memory or other
resources, again causing *DoS*.
The solution to this as well as B) is not to allow processes
to access the filesystem, which could otherwise not be
monitored or manipulated by the mount owner. Since if the
mount owner can ptrace a process, it can do all of the above
without using a FUSE mount, the same criteria as used in
ptrace can be used to check if a process is allowed to access
the filesystem or not.
Note that the *ptrace* check is not strictly necessary to
prevent C/2/i, it is enough to check if mount owner has enough
privilege to send signal to the process accessing the
filesystem, since *SIGSTOP* can be used to get a similar effect.
I think these limitations are unacceptable?
===========================================
If a sysadmin trusts the users enough, or can ensure through other
measures, that system processes will never enter non-privileged
mounts, it can relax the last limitation in several ways:
- With the 'user_allow_other' config option. If this config option is
set, the mounting user can add the 'allow_other' mount option which
disables the check for other users' processes.
User namespaces have an unintuitive interaction with 'allow_other':
an unprivileged user - normally restricted from mounting with
'allow_other' - could do so in a user namespace where they're
privileged. If any process could access such an 'allow_other' mount
this would give the mounting user the ability to manipulate
processes in user namespaces where they're unprivileged. For this
reason 'allow_other' restricts access to users in the same userns
or a descendant.
- With the 'allow_sys_admin_access' module option. If this option is
set, super user's processes have unrestricted access to mounts
irrespective of allow_other setting or user namespace of the
mounting user.
Note that both of these relaxations expose the system to potential
information leak or *DoS* as described in points B and C/2/i-ii in the
preceding section.
Kernel - userspace interface
============================
The following diagram shows how a filesystem operation (in this
example unlink) is performed in FUSE. ::
| "rm /mnt/fuse/file" | FUSE filesystem daemon
| |
| | >sys_read()
| | >fuse_dev_read()
| | >request_wait()
| | [sleep on fc->waitq]
| |
| >sys_unlink() |
| >fuse_unlink() |
| [get request from |
| fc->unused_list] |
| >request_send() |
| [queue req on fc->pending] |
| [wake up fc->waitq] | [woken up]
| >request_wait_answer() |
| [sleep on req->waitq] |
| | <request_wait()
| | [remove req from fc->pending]
| | [copy req to read buffer]
| | [add req to fc->processing]
| | <fuse_dev_read()
| | <sys_read()
| |
| | [perform unlink]
| |
| | >sys_write()
| | >fuse_dev_write()
| | [look up req in fc->processing]
| | [remove from fc->processing]
| | [copy write buffer to req]
| [woken up] | [wake up req->waitq]
| | <fuse_dev_write()
| | <sys_write()
| <request_wait_answer() |
| <request_send() |
| [add request to |
| fc->unused_list] |
| <fuse_unlink() |
| <sys_unlink() |
.. note:: Everything in the description above is greatly simplified
There are a couple of ways in which to deadlock a FUSE filesystem.
Since we are talking about unprivileged userspace programs,
something must be done about these.
**Scenario 1 - Simple deadlock**::
| "rm /mnt/fuse/file" | FUSE filesystem daemon
| |
| >sys_unlink("/mnt/fuse/file") |
| [acquire inode semaphore |
| for "file"] |
| >fuse_unlink() |
| [sleep on req->waitq] |
| | <sys_read()
| | >sys_unlink("/mnt/fuse/file")
| | [acquire inode semaphore
| | for "file"]
| | *DEADLOCK*
The solution for this is to allow the filesystem to be aborted.
**Scenario 2 - Tricky deadlock**
This one needs a carefully crafted filesystem. It's a variation on
the above, only the call back to the filesystem is not explicit,
but is caused by a pagefault. ::
| Kamikaze filesystem thread 1 | Kamikaze filesystem thread 2
| |
| [fd = open("/mnt/fuse/file")] | [request served normally]
| [mmap fd to 'addr'] |
| [close fd] | [FLUSH triggers 'magic' flag]
| [read a byte from addr] |
| >do_page_fault() |
| [find or create page] |
| [lock page] |
| >fuse_readpage() |
| [queue READ request] |
| [sleep on req->waitq] |
| | [read request to buffer]
| | [create reply header before addr]
| | >sys_write(addr - headerlength)
| | >fuse_dev_write()
| | [look up req in fc->processing]
| | [remove from fc->processing]
| | [copy write buffer to req]
| | >do_page_fault()
| | [find or create page]
| | [lock page]
| | * DEADLOCK *
The solution is basically the same as above.
An additional problem is that while the write buffer is being copied
to the request, the request must not be interrupted/aborted. This is
because the destination address of the copy may not be valid after the
request has returned.
This is solved with doing the copy atomically, and allowing abort
while the page(s) belonging to the write buffer are faulted with
get_user_pages(). The 'req->locked' flag indicates when the copy is
taking place, and abort is delayed until this flag is unset.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
용어, 구성 요소와 filesystem type
1-65userspace filesystem은 일반 사용자 공간 process가 data와 metadata를 제공하지만 kernel interface를 통해 보통 filesystem처럼 접근할 수 있는 파일시스템입니다.
filesystem daemon은 filesystem의 data와 metadata를 제공하는 하나 이상의 process입니다.
non-privileged mount 또는 user mount는 non-root 사용자가 마운트한 userspace filesystem이며 daemon도 mount 사용자의 권한으로 실행됩니다. `/etc/fstab`의 `user` option으로 허용하는 mount와는 다른 개념이며 이 문서는 후자를 다루지 않습니다.
filesystem connection은 daemon과 kernel 사이 연결입니다. daemon이 죽거나 filesystem이 unmount될 때까지 존재합니다. detach 또는 lazy unmount만으로는 연결이 끊기지 않으며 마지막 filesystem reference가 해제될 때까지 남습니다.
mount owner는 마운트를 수행한 사용자이고, user는 실제 filesystem operation을 수행하는 사용자입니다. 두 주체가 다를 수 있다는 점이 비권한 mount 접근 제어의 핵심입니다.
FUSE는 userspace filesystem framework이며 kernel module `fuse.ko`, 사용자 공간 library `libfuse.*`, mount utility `fusermount`로 구성됩니다.
중요한 기능은 안전한 non-privileged mount입니다. 예를 들어 sshfs는 sftp protocol을 사용하는 안전한 network filesystem을 비권한 사용자가 제공하게 합니다. library와 utility는 FUSE homepage인 `https://github.com/libfuse/`에서 구할 수 있습니다.
`mount(2)`에 지정하는 filesystem type은 `fuse` 또는 `fuseblk`입니다. `fuse`는 일반 FUSE mount이며 mount syscall의 첫 인수는 kernel이 해석하지 않는 임의 문자열일 수 있습니다. `fuseblk`는 block-device 기반이고 첫 인수를 device name으로 해석합니다.
mount를 제공하는 주체와 사용하는 주체를 분리해 정리했습니다.
`mount(2)` 첫 인수의 해석 차이입니다.
.. SPDX-License-Identifier: GPL-2.0
=============
FUSE Overview
=============
Definitions
===========
Userspace filesystem:
A filesystem in which data and metadata are provided by an ordinary
userspace process. The filesystem can be accessed normally through
the kernel interface.
Filesystem daemon:
The process(es) providing the data and metadata of the filesystem.
Non-privileged mount (or user mount):
A userspace filesystem mounted by a non-privileged (non-root) user.
The filesystem daemon is running with the privileges of the mounting
user. NOTE: this is not the same as mounts allowed with the "user"
option in /etc/fstab, which is not discussed here.
Filesystem connection:
A connection between the filesystem daemon and the kernel. The
connection exists until either the daemon dies, or the filesystem is
umounted. Note that detaching (or lazy umounting) the filesystem
does *not* break the connection, in this case it will exist until
the last reference to the filesystem is released.
Mount owner:
The user who does the mounting.
User:
The user who is performing filesystem operations.
What is FUSE?
=============
FUSE is a userspace filesystem framework. It consists of a kernel
module (fuse.ko), a userspace library (libfuse.*) and a mount utility
(fusermount).
One of the most important features of FUSE is allowing secure,
non-privileged mounts. This opens up new possibilities for the use of
filesystems. A good example is sshfs: a secure network filesystem
using the sftp protocol.
The userspace library and utilities are available from the
`FUSE homepage: <https://github.com/libfuse/>`_
Filesystem type
===============
The filesystem type given to mount(2) can be one of the following:
fuse
This is the usual way to mount a FUSE filesystem. The first
argument of the mount system call may contain an arbitrary string,
which is not interpreted by the kernel.
fuseblk
The filesystem is block device based. The first argument of the
mount system call is interpreted as the name of the device.
Mount option과 접근 정책
66-105`fd=N`은 userspace filesystem과 kernel 통신에 사용할 file descriptor입니다. 반드시 FUSE device `/dev/fuse`를 열어 얻은 fd여야 합니다.
`rootmode=M`은 filesystem root의 file mode를 8진수로 지정합니다. `user_id=N`과 `group_id=N`은 mount owner의 numeric UID와 GID입니다.
기본적으로 FUSE kernel은 file access permission을 검사하지 않습니다. filesystem이 자체 policy를 구현하거나 network filesystem처럼 하위 접근 mechanism에 맡길 수 있습니다.
`default_permissions`는 file mode를 기준으로 kernel permission check를 활성화합니다. 보통 mount owner 외 사용자 접근을 허용하는 `allow_other`와 함께 유용합니다.
`allow_other`는 file access를 mount owner로 제한하는 기본 보안 조치를 해제합니다. 기본적으로 root만 사용할 수 있지만 사용자 공간 configuration option으로 제한을 풀 수 있습니다.
`max_read=N`은 read operation 최대 크기를 지정하며 기본값은 무한입니다. 다만 read request는 어차피 32 page로 제한되고 i386의 4K page에서는 128 KiB입니다.
`blksize=N`은 filesystem block size를 설정하며 기본값은 512입니다. `fuseblk` type mount에서만 유효합니다.
통신, 소유권, 접근 제어, I/O 크기 설정을 구분했습니다.
Mount options
=============
fd=N
The file descriptor to use for communication between the userspace
filesystem and the kernel. The file descriptor must have been
obtained by opening the FUSE device ('/dev/fuse').
rootmode=M
The file mode of the filesystem's root in octal representation.
user_id=N
The numeric user id of the mount owner.
group_id=N
The numeric group id of the mount owner.
default_permissions
By default FUSE doesn't check file access permissions, the
filesystem is free to implement its access policy or leave it to
the underlying file access mechanism (e.g. in case of network
filesystems). This option enables permission checking, restricting
access based on file mode. It is usually useful together with the
'allow_other' mount option.
allow_other
This option overrides the security measure restricting file access
to the user mounting the filesystem. This option is by default only
allowed to root, but this restriction can be removed with a
(userspace) configuration option.
max_read=N
With this option the maximum size of read operations can be set.
The default is infinite. Note that the size of read requests is
limited anyway to 32 pages (which is 128kbyte on i386).
blksize=N
Set the block size for the filesystem. The default is 512. This
option is only valid for 'fuseblk' type mounts.
fusectl 상태·제어와 INTERRUPT 처리
106-187FUSE control filesystem은 `mount -t fusectl none /sys/fs/fuse/connections`로 마운트합니다. 이 경로를 사용하면 이전 version과 호환됩니다.
fusectl 아래에는 각 filesystem connection마다 고유 번호를 이름으로 한 directory가 생깁니다. connection별 control file은 mount owner만 읽고 쓸 수 있습니다.
`waiting`은 사용자 공간으로 전달되기를 기다리거나 daemon이 처리 중인 request 수입니다. filesystem activity가 없는데 0이 아니면 filesystem이 hang 또는 deadlock 상태일 가능성이 큽니다.
`abort`에 아무 값이나 쓰면 connection을 abort합니다. 대기 중인 모든 request를 중단하고 중단된 request와 이후 새 request 모두에 error를 반환합니다.
`max_background`는 동시에 outstanding할 수 있는 background request 최대 수입니다. 한도에 도달하면 일부가 완료될 때까지 추가 request가 block되어 I/O가 멈출 수 있습니다.
`congestion_threshold`를 넘으면 kernel은 filesystem을 congested로 봅니다. asynchronous readahead를 건너뛰어 최적화를 줄이지만 필수 I/O는 유지하고, `WB_SYNC_NONE` non-synchronous writeback을 일시 중단해 page cache flush를 늦춥니다.
FUSE request를 발생시킨 process가 interrupt될 때 아직 사용자 공간에 보내지 않았고 signal이 `SIGKILL` 또는 처리되지 않는 fatal signal이면 queue에서 제거하고 즉시 반환합니다.
아직 보내지 않았지만 signal이 fatal이 아니면 request에 interrupted flag를 설정합니다. 이후 사용자 공간 전달에 성공했을 때 flag가 있으면 `INTERRUPT` request를 queue합니다. 이미 사용자 공간에 보낸 request도 즉시 `INTERRUPT`를 queue합니다.
INTERRUPT는 다른 request보다 우선하므로 daemon은 queue된 INTERRUPT를 먼저 받습니다. daemon은 무시할 수도 있고 원래 request에 `EINTR` error reply를 보내 존중할 수도 있습니다.
원래 request와 INTERRUPT 처리에는 race가 있습니다. INTERRUPT가 원래 request보다 먼저 처리되거나, 원래 request 답변 뒤에 처리될 수 있습니다.
daemon이 원래 request를 찾지 못하면 일정 timeout이나 새 request 수를 기다린 뒤 INTERRUPT에 `EAGAIN`으로 답해야 합니다. 앞선 경우 INTERRUPT가 requeue되고, 뒤늦은 경우 INTERRUPT reply는 무시됩니다.
운영 상태 진단과 queue 제어 항목입니다.
request 전달 상태와 signal 성격에 따른 분기입니다.
Control filesystem
==================
There's a control filesystem for FUSE, which can be mounted by::
mount -t fusectl none /sys/fs/fuse/connections
Mounting it under the '/sys/fs/fuse/connections' directory makes it
backwards compatible with earlier versions.
Under the fuse control filesystem each connection has a directory
named by a unique number.
For each connection the following files exist within this directory:
waiting
The number of requests which are waiting to be transferred to
userspace or being processed by the filesystem daemon. If there is
no filesystem activity and 'waiting' is non-zero, then the
filesystem is hung or deadlocked.
abort
Writing anything into this file will abort the filesystem
connection. This means that all waiting requests will be aborted an
error returned for all aborted and new requests.
max_background
The maximum number of background requests that can be outstanding
at a time. When the number of background requests reaches this limit,
further requests will be blocked until some are completed, potentially
causing I/O operations to stall.
congestion_threshold
The threshold of background requests at which the kernel considers
the filesystem to be congested. When the number of background requests
exceeds this value, the kernel will skip asynchronous readahead
operations, reducing read-ahead optimizations but preserving essential
I/O, as well as suspending non-synchronous writeback operations
(WB_SYNC_NONE), delaying page cache flushing to the filesystem.
Only the owner of the mount may read or write these files.
Interrupting filesystem operations
##################################
If a process issuing a FUSE filesystem request is interrupted, the
following will happen:
- If the request is not yet sent to userspace AND the signal is
fatal (SIGKILL or unhandled fatal signal), then the request is
dequeued and returns immediately.
- If the request is not yet sent to userspace AND the signal is not
fatal, then an interrupted flag is set for the request. When
the request has been successfully transferred to userspace and
this flag is set, an INTERRUPT request is queued.
- If the request is already sent to userspace, then an INTERRUPT
request is queued.
INTERRUPT requests take precedence over other requests, so the
userspace filesystem will receive queued INTERRUPTs before any others.
The userspace filesystem may ignore the INTERRUPT requests entirely,
or may honor them by sending a reply to the *original* request, with
the error set to EINTR.
It is also possible that there's a race between processing the
original request and its INTERRUPT request. There are two possibilities:
1. The INTERRUPT request is processed before the original request is
processed
2. The INTERRUPT request is processed after the original request has
been answered
If the filesystem cannot find the original request, it should wait for
some timeout and/or a number of new requests to arrive, after which it
should reply to the INTERRUPT request with an EAGAIN error. In case
1) the INTERRUPT request will be requeued. In case 2) the INTERRUPT
reply will be ignored.
Connection abort와 비권한 mount 요구 조건
188-236filesystem이 응답하지 않는 원인은 broken userspace implementation, network 단절, 우발적 deadlock, 악의적 deadlock일 수 있습니다.
daemon만 kill하는 방법은 구현 오류와 network 단절에는 효과가 있습니다. daemon과 filesystem 사용자 모두를 kill하면 일부 악의적 deadlock을 제외한 모든 경우에 효과가 있습니다.
`umount -f` 강제 unmount는 filesystem이 아직 attach되어 있고 lazy unmount되지 않았다면 모든 경우에 동작합니다. FUSE control filesystem의 abort는 가장 강력하며 항상 동작합니다.
`mount()` syscall은 권한 operation이므로 non-privileged mount에는 setuid root로 설치된 helper `fusermount`가 필요합니다.
비권한 mount가 system compromise에 쓰이지 않으려면 세 조건이 필요합니다. mount owner가 mounted filesystem으로 권한 상승을 해서는 안 되고, 다른 사용자나 superuser process의 정보에 부당하게 접근해서도 안 되며, 그 process에 원치 않는 동작을 유발해서도 안 됩니다.
원인과 filesystem attach 상태에 따른 효과입니다.
mount owner 능력을 제한하는 세 원칙입니다.
Aborting a filesystem connection
================================
It is possible to get into certain situations where the filesystem is
not responding. Reasons for this may be:
a) Broken userspace filesystem implementation
b) Network connection down
c) Accidental deadlock
d) Malicious deadlock
(For more on c) and d) see later sections)
In either of these cases it may be useful to abort the connection to
the filesystem. There are several ways to do this:
- Kill the filesystem daemon. Works in case of a) and b)
- Kill the filesystem daemon and all users of the filesystem. Works
in all cases except some malicious deadlocks
- Use forced umount (umount -f). Works in all cases but only if
filesystem is still attached (it hasn't been lazy unmounted)
- Abort filesystem through the FUSE control filesystem. Most
powerful method, always works.
How do non-privileged mounts work?
==================================
Since the mount() system call is a privileged operation, a helper
program (fusermount) is needed, which is installed setuid root.
The implication of providing non-privileged mounts is that the mount
owner must not be able to use this capability to compromise the
system. Obvious requirements arising from this are:
A) mount owner should not be able to get elevated privileges with the
help of the mounted filesystem
B) mount owner should not get illegitimate access to information from
other users' and the super user's processes
C) mount owner should not be able to induce undesired behavior in
other users' or the super user's processes
비권한 mount 방어와 제한 완화
237-328mount owner는 device file을 만들고 열거나 suid·sgid application을 만들어 실행해 권한을 올리려 할 수 있습니다. `fusermount`는 non-privileged mount에 항상 `nosuid`와 `nodev`를 추가해 device open을 막고 실행 시 setuid·setgid bit를 무시합니다.
다른 사용자가 filesystem을 접근하면 daemon은 operation의 정확한 순서와 timing을 기록할 수 있습니다. 이 정보는 원래 mount owner가 볼 수 없으므로 information leak입니다.
mount owner는 원래 수정하지 못하거나 제한적으로만 수정할 수 있는 file·directory 위에 filesystem을 mount해 다른 process 행동을 바꿀 수 있습니다. fusermount는 mountpoint write access가 있고 sticky directory가 아니어서 owner가 제한 없이 수정할 수 있을 때만 mount를 허용합니다.
mountpoint 검사 뒤에도 daemon은 operation을 느리게 하거나 무한정 지연해 user 또는 system 전체 DoS를 만들 수 있습니다. 예를 들어 suid application이 system file lock을 잡은 뒤 mount owner의 FUSE file을 접근하면 daemon이 응답을 멈춰 lock을 영원히 보유하게 할 수 있습니다.
또한 길이가 제한 없는 file·directory 또는 깊이가 제한 없는 tree를 보여 system process가 disk, memory, 기타 resource를 소진하게 할 수 있습니다.
정보 유출과 이런 조작을 막으려면 mount owner가 원래 monitor하거나 manipulate할 수 없는 process에는 filesystem 접근을 허용하지 않습니다. mount owner가 어떤 process를 `ptrace`할 수 있다면 FUSE 없이도 같은 행동이 가능하므로 ptrace와 같은 기준으로 접근을 판정합니다.
DoS 지연만 막는 데는 엄격한 ptrace check까지 필요하지 않을 수 있습니다. mount owner가 process에 signal을 보낼 권한이 있는지만 확인해도 `SIGSTOP`으로 비슷한 효과를 낼 수 있기 때문입니다.
sysadmin이 사용자를 신뢰하거나 system process가 non-privileged mount에 들어가지 않음을 다른 수단으로 보장할 수 있다면 제한을 완화할 수 있습니다.
사용자 공간 config의 `user_allow_other`를 켜면 mounting user가 `allow_other` mount option을 추가해 다른 user process 검사를 비활성화할 수 있습니다.
user namespace와 `allow_other`의 상호작용은 직관적이지 않습니다. 일반적으로 비권한인 사용자가 자신이 privileged인 userns 안에서는 allow_other로 mount할 수 있습니다. 아무 process나 이 mount에 접근하면 mounting user가 자신에게 권한이 없는 다른 userns process를 조작할 수 있습니다.
이 때문에 `allow_other`는 mounting user와 같은 user namespace 또는 그 descendant의 사용자로 접근을 제한합니다.
module option `allow_sys_admin_access`를 설정하면 superuser process는 `allow_other` 설정이나 mounting user의 user namespace와 무관하게 mount에 제한 없이 접근합니다.
두 완화 모두 앞서 설명한 B의 information leak과 C/2/i-ii의 DoS 위험에 system을 노출합니다. 기능 편의가 보안 경계를 없앤다는 점을 명시적으로 수용해야 합니다.
권한 상승·mountpoint·process 접근을 순서대로 제한합니다.
범위와 user namespace 제약을 비교합니다.
How are requirements fulfilled?
===============================
A) The mount owner could gain elevated privileges by either:
1. creating a filesystem containing a device file, then opening this device
2. creating a filesystem containing a suid or sgid application, then executing this application
The solution is not to allow opening device files and ignore
setuid and setgid bits when executing programs. To ensure this
fusermount always adds "nosuid" and "nodev" to the mount options
for non-privileged mounts.
B) If another user is accessing files or directories in the
filesystem, the filesystem daemon serving requests can record the
exact sequence and timing of operations performed. This
information is otherwise inaccessible to the mount owner, so this
counts as an information leak.
The solution to this problem will be presented in point 2) of C).
C) There are several ways in which the mount owner can induce
undesired behavior in other users' processes, such as:
1) mounting a filesystem over a file or directory which the mount
owner could otherwise not be able to modify (or could only
make limited modifications).
This is solved in fusermount, by checking the access
permissions on the mountpoint and only allowing the mount if
the mount owner can do unlimited modification (has write
access to the mountpoint, and mountpoint is not a "sticky"
directory)
2) Even if 1) is solved the mount owner can change the behavior
of other users' processes.
i) It can slow down or indefinitely delay the execution of a
filesystem operation creating a DoS against the user or the
whole system. For example a suid application locking a
system file, and then accessing a file on the mount owner's
filesystem could be stopped, and thus causing the system
file to be locked forever.
ii) It can present files or directories of unlimited length, or
directory structures of unlimited depth, possibly causing a
system process to eat up diskspace, memory or other
resources, again causing *DoS*.
The solution to this as well as B) is not to allow processes
to access the filesystem, which could otherwise not be
monitored or manipulated by the mount owner. Since if the
mount owner can ptrace a process, it can do all of the above
without using a FUSE mount, the same criteria as used in
ptrace can be used to check if a process is allowed to access
the filesystem or not.
Note that the *ptrace* check is not strictly necessary to
prevent C/2/i, it is enough to check if mount owner has enough
privilege to send signal to the process accessing the
filesystem, since *SIGSTOP* can be used to get a similar effect.
I think these limitations are unacceptable?
===========================================
If a sysadmin trusts the users enough, or can ensure through other
measures, that system processes will never enter non-privileged
mounts, it can relax the last limitation in several ways:
- With the 'user_allow_other' config option. If this config option is
set, the mounting user can add the 'allow_other' mount option which
disables the check for other users' processes.
User namespaces have an unintuitive interaction with 'allow_other':
an unprivileged user - normally restricted from mounting with
'allow_other' - could do so in a user namespace where they're
privileged. If any process could access such an 'allow_other' mount
this would give the mounting user the ability to manipulate
processes in user namespaces where they're unprivileged. For this
reason 'allow_other' restricts access to users in the same userns
or a descendant.
- With the 'allow_sys_admin_access' module option. If this option is
set, super user's processes have unrestricted access to mounts
irrespective of allow_other setting or user namespace of the
mounting user.
Note that both of these relaxations expose the system to potential
information leak or *DoS* as described in points B and C/2/i-ii in the
preceding section.
Kernel-daemon unlink 요청 왕복
329-377원문의 sequence diagram은 `rm /mnt/fuse/file`이 kernel과 FUSE daemon 사이에서 처리되는 단순화된 흐름을 보여줍니다.
daemon은 `sys_read()`에서 `fuse_dev_read()`와 `request_wait()`를 호출하고 `fc->waitq`에서 request를 기다립니다.
사용자 command는 `sys_unlink()`에서 `fuse_unlink()`로 들어갑니다. kernel은 `fc->unused_list`에서 request를 얻고 `request_send()`가 `fc->pending`에 queue한 뒤 `fc->waitq`를 깨웁니다.
요청 thread는 `request_wait_answer()`에서 `req->waitq`에 잠듭니다. 깨어난 daemon의 read 경로는 `fc->pending`에서 request를 제거하고 read buffer에 복사한 뒤 `fc->processing`에 넣어 사용자 공간에 반환합니다.
daemon이 실제 unlink를 수행한 뒤 `sys_write()`와 `fuse_dev_write()`로 result를 보냅니다. kernel은 `fc->processing`에서 해당 request를 찾아 제거하고 write buffer를 request로 복사한 뒤 `req->waitq`를 깨웁니다.
요청 thread는 `request_wait_answer()`와 `request_send()`에서 돌아오고 request를 `fc->unused_list`에 다시 넣습니다. 그 뒤 `fuse_unlink()`와 `sys_unlink()`가 반환합니다. 원문은 실제 구현이 이 설명보다 훨씬 복잡하다고 명시합니다.
세 queue와 두 wait queue 사이의 request 이동을 보존했습니다.
정상 unlink 동안 request가 이동하는 목록입니다.
Kernel - userspace interface
============================
The following diagram shows how a filesystem operation (in this
example unlink) is performed in FUSE. ::
| "rm /mnt/fuse/file" | FUSE filesystem daemon
| |
| | >sys_read()
| | >fuse_dev_read()
| | >request_wait()
| | [sleep on fc->waitq]
| |
| >sys_unlink() |
| >fuse_unlink() |
| [get request from |
| fc->unused_list] |
| >request_send() |
| [queue req on fc->pending] |
| [wake up fc->waitq] | [woken up]
| >request_wait_answer() |
| [sleep on req->waitq] |
| | <request_wait()
| | [remove req from fc->pending]
| | [copy req to read buffer]
| | [add req to fc->processing]
| | <fuse_dev_read()
| | <sys_read()
| |
| | [perform unlink]
| |
| | >sys_write()
| | >fuse_dev_write()
| | [look up req in fc->processing]
| | [remove from fc->processing]
| | [copy write buffer to req]
| [woken up] | [wake up req->waitq]
| | <fuse_dev_write()
| | <sys_write()
| <request_wait_answer() |
| <request_send() |
| [add request to |
| fc->unused_list] |
| <fuse_unlink() |
| <sys_unlink() |
.. note:: Everything in the description above is greatly simplified
단순·pagefault deadlock과 abort 안전성
378-440비권한 사용자 공간 program이 제공하는 FUSE filesystem에는 여러 deadlock 가능성이 있으므로 kernel이 이를 해소할 수 있어야 합니다.
단순 deadlock에서 `sys_unlink('/mnt/fuse/file')`은 file의 inode semaphore를 획득하고 `fuse_unlink()`가 `req->waitq`에서 daemon 답을 기다립니다.
daemon이 request를 읽은 뒤 같은 `/mnt/fuse/file`에 다시 `sys_unlink()`를 호출하면 동일 inode semaphore를 획득하려 하지만 원래 thread가 daemon 답을 기다리며 lock을 놓지 않아 deadlock입니다. 해결책은 filesystem connection을 abort할 수 있게 하는 것입니다.
더 까다로운 deadlock은 명시적 callback이 아니라 pagefault로 filesystem에 재진입합니다. 악의적으로 설계된 filesystem thread 1은 FUSE file을 열고 fd를 address에 mmap한 뒤 fd를 닫습니다. 다른 thread의 FLUSH는 특정 magic flag를 trigger합니다.
thread 1이 mapping에서 byte를 읽으면 `do_page_fault()`가 page를 찾거나 만들고 lock한 뒤 `fuse_readpage()`가 READ request를 queue하고 `req->waitq`에서 잠듭니다.
thread 2는 read request를 buffer로 받고 reply header를 mapped address 바로 앞에 만든 뒤 `sys_write(addr - headerlength)`로 result를 씁니다. `fuse_dev_write()`가 processing request를 찾고 result를 복사하는 동안 write buffer 접근이 다시 `do_page_fault()`를 일으킵니다.
두 번째 pagefault는 thread 1이 이미 lock한 같은 page를 lock하려 하므로 deadlock입니다. 기본 해결은 역시 filesystem abort입니다.
추가로 write buffer를 request로 복사하는 중에는 request를 interrupt하거나 abort하면 안 됩니다. request가 반환된 뒤에는 copy destination address가 더 이상 유효하지 않을 수 있기 때문입니다.
kernel은 copy를 atomic하게 수행하고 write buffer의 page를 `get_user_pages()`로 fault하는 동안에는 abort를 허용합니다. `req->locked` flag가 실제 copy 중임을 표시하며 flag가 해제될 때까지 abort를 지연합니다.
inode semaphore와 daemon callback의 순환 대기입니다.
reply buffer가 FUSE mapping을 가리킬 때 생기는 간접 재진입입니다.
어느 시점에 abort를 허용하거나 지연하는지 정리했습니다.
There are a couple of ways in which to deadlock a FUSE filesystem.
Since we are talking about unprivileged userspace programs,
something must be done about these.
**Scenario 1 - Simple deadlock**::
| "rm /mnt/fuse/file" | FUSE filesystem daemon
| |
| >sys_unlink("/mnt/fuse/file") |
| [acquire inode semaphore |
| for "file"] |
| >fuse_unlink() |
| [sleep on req->waitq] |
| | <sys_read()
| | >sys_unlink("/mnt/fuse/file")
| | [acquire inode semaphore
| | for "file"]
| | *DEADLOCK*
The solution for this is to allow the filesystem to be aborted.
**Scenario 2 - Tricky deadlock**
This one needs a carefully crafted filesystem. It's a variation on
the above, only the call back to the filesystem is not explicit,
but is caused by a pagefault. ::
| Kamikaze filesystem thread 1 | Kamikaze filesystem thread 2
| |
| [fd = open("/mnt/fuse/file")] | [request served normally]
| [mmap fd to 'addr'] |
| [close fd] | [FLUSH triggers 'magic' flag]
| [read a byte from addr] |
| >do_page_fault() |
| [find or create page] |
| [lock page] |
| >fuse_readpage() |
| [queue READ request] |
| [sleep on req->waitq] |
| | [read request to buffer]
| | [create reply header before addr]
| | >sys_write(addr - headerlength)
| | >fuse_dev_write()
| | [look up req in fc->processing]
| | [remove from fc->processing]
| | [copy write buffer to req]
| | >do_page_fault()
| | [find or create page]
| | [lock page]
| | * DEADLOCK *
The solution is basically the same as above.
An additional problem is that while the write buffer is being copied
to the request, the request must not be interrupted/aborted. This is
because the destination address of the copy may not be valid after the
request has returned.
This is solved with doing the copy atomically, and allowing abort
while the page(s) belonging to the write buffer are faulted with
get_user_pages(). The 'req->locked' flag indicates when the copy is
taking place, and abort is delayed until this flag is unset.
요약·해설
fuse.rst:1-440FUSE는 `fuse.ko`, libfuse, fusermount를 결합해 일반 process가 filesystem data와 metadata를 제공하게 합니다. 비권한 mount의 안전성은 nosuid·nodev, mountpoint 검증, mount owner가 monitor할 수 없는 process의 접근 제한으로 성립합니다.
운영자는 fusectl의 waiting·abort·max_background·congestion_threshold로 connection을 진단하고 제어할 수 있습니다. 정상 request는 unused, pending, processing queue와 두 wait queue를 왕복하며, 재진입·pagefault deadlock을 해소하려면 항상 동작하는 abort 경로와 copy 중 `req->locked` 보호가 필요합니다.
mount 생성부터 request 처리와 장애 복구까지의 핵심 흐름입니다.