KSMBD - SMB3 Kernel Server

KSMBD의 커널·사용자 공간 분리

KSMBD는 네트워크 파일 공유용 SMB3 프로토콜을 커널 공간에서 구현하는 Linux 서버다. 성능에 직접 관련된 open·read·write·close 같은 파일 연산은 커널의 `ksmbd`가 처리해 VFS와 쉽게 결합한다.

성능과 직접 관계가 적고 과거 buffer overflow와 위험한 보안 결함이 잦았던 DCE/RPC 관리, 사용자 계정 관리는 사용자 공간 `ksmbd.mountd`가 맡는다. 이 분리는 고속 데이터 경로를 커널에 두면서 복잡한 관리 로직의 공격면을 분리한다.

.. SPDX-License-Identifier: GPL-2.0

==========================
KSMBD - SMB3 Kernel Server
==========================

KSMBD is a linux kernel server which implements SMB3 protocol in kernel space
for sharing files over network.

KSMBD architecture
==================

The subset of performance related operations belong in kernelspace and
the other subset which belong to operations which are not really related with
performance in userspace. So, DCE/RPC management that has historically resulted
into a number of buffer overflow issues and dangerous security bugs and user
account management are implemented in user space as ksmbd.mountd.
File operations that are related with performance (open/read/write/close etc.)
in kernel space (ksmbd). This also allows for easier integration with VFS
interface for all file operations.

포트 445 연결과 병렬 workitem

서버 daemon이 시작되면 초기화 때 `ksmbd/interface-name` forker thread가 생기고 전용 TCP 포트 445에서 SMB 요청을 기다린다. 새 client 연결을 받아 각 통신 channel 전용 `ksmbd/1`부터 `ksmbd/n` thread를 만든다.

전용 thread는 연결 뒤 SMB 명령의 수신과 parsing을 맡는다. DCE/RPC처럼 사용자 공간이 처리할 명령은 `ksmbd.mountd`로 넘기고, 일반 명령은 각각 독립 kernel workitem으로 만들어 `ksmbd-io` kworker queue에 넣는다.

여러 client 명령을 병렬로 처리할 수 있으며, 기본 kernel worker 관리가 부하 증가 때 thread를 늘리고 감소 때 여분을 없앤다. 이 multiplexing은 load sharing과 client 처리량을 함께 개선한다.

ksmbd (kernel daemon)
---------------------

When the server daemon is started, It starts up a forker thread
(ksmbd/interface name) at initialization time and open a dedicated port 445
for listening to SMB requests. Whenever new clients make a request, the Forker
thread will accept the client connection and fork a new thread for a dedicated
communication channel between the client and the server. It allows for parallel
processing of SMB requests(commands) from clients as well as allowing for new
clients to make new connections. Each instance is named ksmbd/1~n(port number)
to indicate connected clients. Depending on the SMB request types, each new
thread can decide to pass through the commands to the user space (ksmbd.mountd),
currently DCE/RPC commands are identified to be handled through the user space.
To further utilize the linux kernel, it has been chosen to process the commands
as workitems and to be executed in the handlers of the ksmbd-io kworker threads.
It allows for multiplexing of the handlers as the kernel takes care of initiating
extra worker threads if the load is increased and vice versa, if the load is
decreased it destroys the extra worker threads. So, after the connection is
established with the client. Dedicated ksmbd/1..n(port number) takes complete
ownership of receiving/parsing of SMB commands. Each received command is worked
in parallel i.e., there can be multiple client commands which are worked in
parallel. After receiving each command a separated kernel workitem is prepared
for each command which is further queued to be handled by ksmbd-io kworkers.
So, each SMB workitem is queued to the kworkers. This allows the benefit of load
sharing to be managed optimally by the default kernel and optimizing client
performance by handling client commands in parallel.

`ksmbd.mountd`의 관리 경로

`ksmbd.mountd`는 `ksmbd.adduser`로 등록한 사용자 계정과 암호를 커널에 전달하고, `smb.conf`에서 읽은 share 매개변수도 제공하는 사용자 공간 프로세스다.

daemon은 netlink socket으로 커널 interface에 계속 연결된 채 DCE/RPC와 share·user 정보 요청을 기다린다. 파일 서버에 중요한 `NetShareEnum`, `NetServerGetInfo` 등 최소 수십 개 RPC 호출을 처리하고, 완성된 DCE/RPC 응답을 해당 client의 kernel thread로 돌려준다.

ksmbd.mountd (user space daemon)
--------------------------------

ksmbd.mountd is a userspace process to, transfer the user account and password that
are registered using ksmbd.adduser (part of utils for user space). Further it
allows sharing information parameters that are parsed from smb.conf to ksmbd in
kernel. For the execution part it has a daemon which is continuously running
and connected to the kernel interface using netlink socket, it waits for the
requests (dcerpc and share/user info). It handles RPC calls (at a minimum few
dozen) that are most important for file server from NetShareEnum and
NetServerGetInfo. Complete DCE/RPC response is prepared from the user space
and passed over to the associated kernel thread for the client.

KSMBD 기능 지원 상태

KSMBD는 SMB2.1, SMB3.0, SMB3.1.1 dialect를 지원하며 보안에 취약한 SMB1은 의도적으로 제외한다. 자동 협상, compound request, oplock, v1·v2 lease, multi-credit, NTLM 계열 인증, signing, pre-auth integrity, CCM·GCM 암호화, SMB Direct, RSS, SMB3.1.1 POSIX extension과 Kerberos는 지원된다.

SMB3 multi-channel은 replay·retry가 아직 계획 단계라 부분 지원이다. ACL은 DACL만 제공하고 SACL auditing과 domain member 지원은 향후 과제다. 로컬 domain SID는 무작위 subauth를 디스크에 저장하고 inode의 uid·gid를 RID로 사용한다.

DCE/RPC는 파일 서버에 필요한 `NetShareEnumAll`, `NetServerGetInfo`, SAMR, LSARPC 일부를 `ksmbd.mountd`의 netlink interface로 처리한다. 더 많은 관리 호출과 Witness protocol을 위해 Samba 도구·library upcall 통합을 조사 중이다.

durable·persistent handle, SMB2 notify, ksmbd/nfsd 상호운용, SMB3.1.1 compression·QUIC, RDMA 위 signing/encryption, GMAC signing은 이 표에서 미래 계획으로 분류된다.

KSMBD Feature Status
====================

============================== =================================================
Feature name                   Status
============================== =================================================
Dialects                       Supported. SMB2.1 SMB3.0, SMB3.1.1 dialects
                               (intentionally excludes security vulnerable SMB1
                               dialect).
Auto Negotiation               Supported.
Compound Request               Supported.
Oplock Cache Mechanism         Supported.
SMB2 leases(v1 lease)          Supported.
Directory leases(v2 lease)     Supported.
Multi-credits                  Supported.
NTLM/NTLMv2                    Supported.
HMAC-SHA256 Signing            Supported.
Secure negotiate               Supported.
Signing Update                 Supported.
Pre-authentication integrity   Supported.
SMB3 encryption(CCM, GCM)      Supported. (CCM/GCM128 and CCM/GCM256 supported)
SMB direct(RDMA)               Supported.
SMB3 Multi-channel             Partially Supported. Planned to implement
                               replay/retry mechanisms for future.
Receive Side Scaling mode      Supported.
SMB3.1.1 POSIX extension       Supported.
ACLs                           Partially Supported. only DACLs available, SACLs
                               (auditing) is planned for the future. For
                               ownership (SIDs) ksmbd generates random subauth
                               values(then store it to disk) and use uid/gid
                               get from inode as RID for local domain SID.
                               The current acl implementation is limited to
                               standalone server, not a domain member.
                               Integration with Samba tools is being worked on
                               to allow future support for running as a domain
                               member.
Kerberos                       Supported.
Durable handle v1,v2           Planned for future.
Persistent handle              Planned for future.
SMB2 notify                    Planned for future.
Sparse file support            Supported.
DCE/RPC support                Partially Supported. a few calls(NetShareEnumAll,
                               NetServerGetInfo, SAMR, LSARPC) that are needed
                               for file server handled via netlink interface
                               from ksmbd.mountd. Additional integration with
                               Samba tools and libraries via upcall is being
                               investigated to allow support for additional
                               DCE/RPC management calls (and future support
                               for Witness protocol e.g.)
ksmbd/nfsd interoperability    Planned for future. The features that ksmbd
                               support are Leases, Notify, ACLs and Share modes.
SMB3.1.1 Compression           Planned for future.
SMB3.1.1 over QUIC             Planned for future.
Signing/Encryption over RDMA   Planned for future.
SMB3.1.1 GMAC signing support  Planned for future.
============================== =================================================

빌드·구성·실행 절차

먼저 release에서 `ksmbd-tools`를 받아 `autogen.sh`, `configure --with-rundir=/run`, `make && sudo make install` 순으로 빌드한다. 사용법은 프로젝트 README에서 `ksmbd.mountd`, `adduser`, `addshare`, `control` 도구를 확인한다.

`/usr/local/etc/ksmbd/ksmbd.conf`를 만들고 share를 추가한다. 자세한 형식은 `ksmbd.conf.example`과 `man ksmbd.conf`을 따른다. `man ksmbd.adduser`를 확인한 뒤 `sudo ksmbd.adduser -a <USERNAME>`으로 SMB 접근 계정과 암호를 만든다.

커널 menuconfig의 `Network File Systems → SMB3 server support (EXPERIMENTAL)`을 module로 선택했다면 빌드 후 `sudo modprobe ksmbd.ko`로 삽입한다. built-in이면 별도 load는 필요 없다. 이어 `sudo ksmbd.mountd`를 시작하고 Windows 또는 Linux의 `cifs.ko`·Samba `smbclient`로 share에 접근한다.

How to run
==========

1. Download ksmbd-tools(https://github.com/cifsd-team/ksmbd-tools/releases) and
   compile them.

   - Refer to README(https://github.com/cifsd-team/ksmbd-tools/blob/master/README.md)
     to know how to use ksmbd.mountd/adduser/addshare/control utils

     $ ./autogen.sh
     $ ./configure --with-rundir=/run
     $ make && sudo make install

2. Create /usr/local/etc/ksmbd/ksmbd.conf file, add SMB share in ksmbd.conf file.

   - Refer to ksmbd.conf.example in ksmbd-utils, See ksmbd.conf manpage
     for details to configure shares.

        $ man ksmbd.conf

3. Create user/password for SMB share.

   - See ksmbd.adduser manpage.

     $ man ksmbd.adduser
     $ sudo ksmbd.adduser -a <Enter USERNAME for SMB share access>

4. Insert the ksmbd.ko module after you build your kernel. No need to load the module
   if ksmbd is built into the kernel.

   - Set ksmbd in menuconfig(e.g. $ make menuconfig)
       [*] Network File Systems  --->
           <M> SMB3 server support (EXPERIMENTAL)

        $ sudo modprobe ksmbd.ko

5. Start ksmbd user space daemon

        $ sudo ksmbd.mountd

6. Access share from Windows or Linux using SMB3 client (cifs.ko or smbclient of samba)

종료와 component별 debug 출력

KSMBD의 사용자·커널 daemon은 `sudo ksmbd.control -s`로 함께 종료한다.

debug 계층의 상태는 `/sys/class/ksmbd-control/debug`에 나타난다. `sudo ksmbd.control -d "all"`은 모든 component 출력을 활성화하고, `-d "smb"`처럼 `smb`, `auth`, `vfs`, `oplock`, `ipc`, `conn`, `rdma` 중 하나를 고를 수 있다.

sysfs 파일을 읽으면 활성 component가 대괄호로 표시된다. 선택된 component를 같은 명령으로 다시 지정하면 비활성화되어 대괄호가 사라진다.

Shutdown KSMBD
==============

1. kill user and kernel space daemon
        # sudo ksmbd.control -s

How to turn debug print on
==========================

Each layer
/sys/class/ksmbd-control/debug

1. Enable all component prints
        # sudo ksmbd.control -d "all"

2. Enable one of the components (smb, auth, vfs, oplock, ipc, conn, rdma)
        # sudo ksmbd.control -d "smb"

3. Show what prints are enabled.
        # cat /sys/class/ksmbd-control/debug
          [smb] auth vfs oplock ipc conn [rdma]

4. Disable prints:
        If you try the selected component once more, It is disabled without brackets.