NVMe PCI endpoint function target

PCI transport NVMe target controller

저자는 Damien Le Moal입니다. NVMe PCI endpoint function target driver는 PCI transport type으로 구성한 NVMe fabrics target controller를 이용해 NVMe PCIe controller를 구현합니다.

.. SPDX-License-Identifier: GPL-2.0

=================================
NVMe PCI Endpoint Function Target
=================================

:Author: Damien Le Moal <[email protected]>

The NVMe PCI endpoint function target driver implements an NVMe PCIe controller
using an NVMe fabrics target controller configured with the PCI transport type.

구조와 command data path

이 driver는 NVMe target controller를 PCIe link에 노출해 일반 M.2 SSD와 유사한 NVMe PCIe device를 구현합니다. Controller는 NVMe over Fabrics와 같은 방식으로 생성하며, port를 통해 NVMe subsystem의 interface 역할을 합니다. Port transfer type은 `pci`여야 합니다.

Subsystem namespace는 일반 file이나 block device를 backing store로 쓸 수 있습니다. 또는 NVMe passthrough를 이용해 기존 physical NVMe device나 NVMe TCP host controller 같은 NVMe fabrics host controller를 PCI host에 노출할 수 있습니다.

Driver는 PCIe host가 제출한 NVMe command의 parsing과 실행을 가능한 한 NVMe target core에 맡깁니다. 다만 PCI endpoint framework API와 DMA API를 사용해 PCIe link의 모든 data transfer를 관리하므로 일부 NVMe data structure 관리와 command parsing도 직접 구현합니다.

Overview
========

The NVMe PCI endpoint function target driver allows exposing an NVMe target
controller over a PCIe link, thus implementing an NVMe PCIe device similar to a
regular M.2 SSD. The target controller is created in the same manner as when
using NVMe over fabrics: the controller represents the interface to an NVMe
subsystem using a port. The port transfer type must be configured to be
"pci". The subsystem can be configured to have namespaces backed by regular
files or block devices, or can use NVMe passthrough to expose to the PCI host an
existing physical NVMe device or an NVMe fabrics host controller (e.g. a NVMe
TCP host controller).

The NVMe PCI endpoint function target driver relies as much as possible on the
NVMe target core code to parse and execute NVMe commands submitted by the PCIe
host. However, using the PCI endpoint framework API and DMA API, the driver is
also responsible for managing all data transfers over the PCIe link. This
implies that the NVMe PCI endpoint function target driver implements several
NVMe data structure management and some NVMe command parsing.

1) The driver manages retrieval of NVMe commands in submission queues using DMA
   if supported, or MMIO otherwise. Each command retrieved is then executed
   using a work item to maximize performance with the parallel execution of
   multiple commands on different CPUs. The driver uses a work item to
   constantly poll the doorbell of all submission queues to detect command
   submissions from the PCIe host.

2) The driver transfers completion queues entries of completed commands to the
   PCIe host using MMIO copy of the entries in the host completion queue.
   After posting completion entries in a completion queue, the driver uses the
   PCI endpoint framework API to raise an interrupt to the host to signal the
   commands completion.

3) For any command that has a data buffer, the NVMe PCI endpoint target driver
   parses the command PRPs or SGLs lists to create a list of PCI address
   segments representing the mapping of the command data buffer on the host.
   The command data buffer is transferred over the PCIe link using this list of
   PCI address segments using DMA, if supported. If DMA is not supported, MMIO
   is used, which results in poor performance. For write commands, the command
   data buffer is transferred from the host into a local memory buffer before
   executing the command using the target core code. For read commands, a local
   memory buffer is allocated to execute the command and the content of that
   buffer is transferred to the host once the command completes.

Controller capability 예외

BAR 0 register로 PCIe host에 공개하는 NVMe capability는 target core controller와 거의 같지만 다음 예외가 있습니다.

Controller Capabilities
-----------------------

The NVMe capabilities exposed to the PCIe host through the BAR 0 registers
are almost identical to the capabilities of the NVMe target controller
implemented by the target core code. There are some exceptions.

1) The NVMe PCI endpoint target driver always sets the controller capability
   CQR bit to request "Contiguous Queues Required". This is to facilitate the
   mapping of a queue PCI address range to the local CPU address space.

2) The doorbell stride (DSTRB) is always set to be 4B

3) Since the PCI endpoint framework does not provide a way to handle PCI level
   resets, the controller capability NSSR bit (NVM Subsystem Reset Supported)
   is always cleared.

4) The boot partition support (BPS), Persistent Memory Region Supported (PMRS)
   and Controller Memory Buffer Supported (CMBS) capabilities are never
   reported.

지원 기능과 MDTS

Driver는 PRP와 SGL을 모두 지원하며 IRQ vector coalescing과 submission queue arbitration burst도 구현합니다.

최대 queue 수와 maximum data transfer size(`MDTS`)는 controller를 시작하기 전에 configfs에서 설정합니다. Command 실행용 local memory의 과도한 사용을 피하려고 MDTS 기본값은 512KB이고 임의의 상한인 2MB로 제한됩니다.

Supported Features
------------------

The NVMe PCI endpoint target driver implements support for both PRPs and SGLs.
The driver also implements IRQ vector coalescing and submission queue
arbitration burst.

The maximum number of queues and the maximum data transfer size (MDTS) are
configurable through configfs before starting the controller. To avoid issues
with excessive local memory usage for executing commands, MDTS defaults to 512
KB and is limited to a maximum of 2 MB (arbitrary limit).

필요한 PCI address mapping window

대부분의 PCI endpoint controller는 PCI address range를 local CPU memory address에 mapping하는 window 수가 제한돼 있습니다.

비동기 동작 때문에 이 window들이 항상 동시에 쓰이지는 않지만 동시에 필요할 가능성은 있습니다. 안전하게 지원할 수 있는 completion queue 최대 수는 `endpoint controller의 전체 mapping window 수 - 2`입니다.

예를 들어 outbound memory window가 32개면 window 부족으로 PCI address mapping error가 발생할 위험 없이 completion queue를 최대 30개 운용할 수 있습니다.

Minimum number of PCI Address Mapping Windows Required
------------------------------------------------------

Most PCI endpoint controllers provide a limited number of mapping windows for
mapping a PCI address range to local CPU memory addresses. The NVMe PCI
endpoint target controllers uses mapping windows for the following.

1) One memory window for raising MSI or MSI-X interrupts
2) One memory window for MMIO transfers
3) One memory window for each completion queue

Given the highly asynchronous nature of the NVMe PCI endpoint target driver
operation, the memory windows as described above will generally not be used
simultaneously, but that may happen. So a safe maximum number of completion
queues that can be supported is equal to the total number of memory mapping
windows of the PCI endpoint controller minus two. E.g. for an endpoint PCI
controller with 32 outbound memory windows available, up to 30 completion
queues can be safely operated without any risk of getting PCI address mapping
errors due to the lack of memory windows.

최대 queue pair 수

Driver를 PCI endpoint controller에 bind할 때 admin queue와 여러 I/O queue를 수용할 만큼 BAR 0 공간을 할당합니다. 지원 가능한 I/O queue pair 최대값은 여러 제한의 최솟값입니다.

NVMe endpoint function driver는 configfs를 통해 최대 queue pair 수를 설정할 수 있습니다.

Maximum Number of Queue Pairs
-----------------------------

Upon binding of the NVMe PCI endpoint target driver to the PCI endpoint
controller, BAR 0 is allocated with enough space to accommodate the admin queue
and multiple I/O queues. The maximum of number of I/O queues pairs that can be
supported is limited by several factors.

1) The NVMe target core code limits the maximum number of I/O queues to the
   number of online CPUs.
2) The total number of queue pairs, including the admin queue, cannot exceed
   the number of MSI-X or MSI vectors available.
3) The total number of completion queues must not exceed the total number of
   PCI mapping windows minus 2 (see above).

The NVMe endpoint function driver allows configuring the maximum number of
queue pairs through configfs.

Specification 비준수 제약

NVMe target core와 마찬가지로 이 driver는 여러 submission queue가 하나의 completion queue를 공유하는 구성을 지원하지 않습니다. 모든 submission queue는 서로 고유한 completion queue를 지정해야 합니다.

Limitations and NVMe Specification Non-Compliance
-------------------------------------------------

Similar to the NVMe target core code, the NVMe PCI endpoint target driver does
not support multiple submission queues using the same completion queue. All
submission queues must specify a unique completion queue.

사용자 안내와 kernel 요건

NVMe PCI endpoint target을 구성하려면 kernel에서 `CONFIG_PCI_ENDPOINT`, `CONFIG_PCI_ENDPOINT_CONFIGFS`, `CONFIG_NVME_TARGET_PCI_EPF`를 활성화해야 합니다.

또한 `CONFIG_PCI`, `CONFIG_BLK_DEV_NVME`, `CONFIG_NVME_TARGET`과 endpoint hardware에 맞는 PCI endpoint controller driver가 필요합니다.

시험을 간단히 하려면 `CONFIG_BLK_DEV_NULL_BLK`를 켜는 것이 좋습니다. 그러면 `null_blk` block device를 subsystem namespace로 사용할 수 있습니다.

User Guide
==========

This section describes the hardware requirements and how to setup an NVMe PCI
endpoint target device.

Kernel Requirements
-------------------

The kernel must be compiled with the configuration options CONFIG_PCI_ENDPOINT,
CONFIG_PCI_ENDPOINT_CONFIGFS, and CONFIG_NVME_TARGET_PCI_EPF enabled.
CONFIG_PCI, CONFIG_BLK_DEV_NVME and CONFIG_NVME_TARGET must also be enabled
(obviously).

In addition to this, at least one PCI endpoint controller driver should be
available for the endpoint hardware used.

To facilitate testing, enabling the null-blk driver (CONFIG_BLK_DEV_NULL_BLK)
is also recommended. With this, a simple setup using a null_blk block device
as a subsystem namespace can be used.

Hardware 연결 요건

최소 하나의 endpoint controller device가 필요합니다. System의 endpoint controller는 다음 두 위치에서 확인할 수 있습니다.

# ls /sys/class/pci_epc/
a40000000.pcie-ep

# ls /sys/kernel/config/pci_ep/controllers
a40000000.pcie-ep

두 번째 경로는 `PCI_ENDPOINT_CONFIGFS`가 활성화됐을 때 사용할 수 있습니다.

Endpoint board는 RX-TX signal이 교차된 PCI cable로 host와 연결해야 합니다. Host PCI slot에 plug-and-play 기능이 없다면 NVMe PCI endpoint device를 구성할 때 host 전원을 꺼야 합니다.

Hardware Requirements
---------------------

To use the NVMe PCI endpoint target driver, at least one endpoint controller
device is required.

To find the list of endpoint controller devices in the system::

       # ls /sys/class/pci_epc/
        a40000000.pcie-ep

If PCI_ENDPOINT_CONFIGFS is enabled::

       # ls /sys/kernel/config/pci_ep/controllers
        a40000000.pcie-ep

The endpoint board must of course also be connected to a host with a PCI cable
with RX-TX signal swapped. If the host PCI slot used does not have
plug-and-play capabilities, the host should be powered off when the NVMe PCI
endpoint device is configured.

Endpoint device 생성의 두 단계

NVMe endpoint device는 두 단계로 만듭니다. 먼저 NVMe target subsystem과 port를 정의하고, 다음으로 NVMe PCI endpoint device를 구성해 앞서 만든 subsystem과 port에 bind합니다.

NVMe Endpoint Device
--------------------

Creating an NVMe endpoint device is a two step process. First, an NVMe target
subsystem and port must be defined. Second, the NVMe PCI endpoint device must
be setup and bound to the subsystem and port created.

NVMe subsystem과 port 생성

다음은 memory backing이 없는 기본 250GB `null_blk` device 하나를 namespace로 사용하는 간단한 subsystem과 port 예제입니다. 먼저 configfs를 mount하고 `null_blk`와 `nvmet_pci_epf` module을 load합니다.

# mount -t configfs none /sys/kernel/config
# modprobe null_blk
# ls /dev/nullb0
/dev/nullb0
# modprobe nvmet_pci_epf
# lsmod | grep nvmet

PCI target controller에 사용할 subsystem을 만들고 model, vendor ID, host 접근, 최대 4개 I/O queue pair를 설정합니다.

# cd /sys/kernel/config/nvmet/subsystems
# mkdir nvmepf.0.nqn
# echo -n "Linux-pci-epf" > nvmepf.0.nqn/attr_model
# echo "0x1b96" > nvmepf.0.nqn/attr_vendor_id
# echo "0x1b96" > nvmepf.0.nqn/attr_subsys_vendor_id
# echo 1 > nvmepf.0.nqn/attr_allow_any_host
# echo 4 > nvmepf.0.nqn/attr_qid_max

`/dev/nullb0`를 backing device로 하는 namespace 1을 만들고 활성화합니다.

# mkdir nvmepf.0.nqn/namespaces/1
# echo -n "/dev/nullb0" > nvmepf.0.nqn/namespaces/1/device_path
# echo 1 > nvmepf.0.nqn/namespaces/1/enable

마지막으로 target port 1을 만들고 transport type을 `pci`로 지정한 뒤 subsystem을 symbolic link로 연결합니다.

# cd /sys/kernel/config/nvmet/ports
# mkdir 1
# echo -n "pci" > 1/addr_trtype
# ln -s /sys/kernel/config/nvmet/subsystems/nvmepf.0.nqn \
+        /sys/kernel/config/nvmet/ports/1/subsystems/nvmepf.0.nqn

Creating an NVMe Subsystem and Port
-----------------------------------

Details about how to configure an NVMe target subsystem and port are outside the
scope of this document. The following only provides a simple example of a port
and subsystem with a single namespace backed by a null_blk device.

First, make sure that configfs is enabled::

       # mount -t configfs none /sys/kernel/config

Next, create a null_blk device (default settings give a 250 GB device without
memory backing). The block device created will be /dev/nullb0 by default::

        # modprobe null_blk
        # ls /dev/nullb0
        /dev/nullb0

The NVMe PCI endpoint function target driver must be loaded::

        # modprobe nvmet_pci_epf
        # lsmod | grep nvmet
        nvmet_pci_epf          32768  0
        nvmet                 118784  1 nvmet_pci_epf
        nvme_core             131072  2 nvmet_pci_epf,nvmet

Now, create a subsystem and a port that we will use to create a PCI target
controller when setting up the NVMe PCI endpoint target device. In this
example, the port is created with a maximum of 4 I/O queue pairs::

        # cd /sys/kernel/config/nvmet/subsystems
        # mkdir nvmepf.0.nqn
        # echo -n "Linux-pci-epf" > nvmepf.0.nqn/attr_model
        # echo "0x1b96" > nvmepf.0.nqn/attr_vendor_id
        # echo "0x1b96" > nvmepf.0.nqn/attr_subsys_vendor_id
        # echo 1 > nvmepf.0.nqn/attr_allow_any_host
        # echo 4 > nvmepf.0.nqn/attr_qid_max

Next, create and enable the subsystem namespace using the null_blk block
device::

        # mkdir nvmepf.0.nqn/namespaces/1
        # echo -n "/dev/nullb0" > nvmepf.0.nqn/namespaces/1/device_path
        # echo 1 > "nvmepf.0.nqn/namespaces/1/enable"

Finally, create the target port and link it to the subsystem::

        # cd /sys/kernel/config/nvmet/ports
        # mkdir 1
        # echo -n "pci" > 1/addr_trtype
        # ln -s /sys/kernel/config/nvmet/subsystems/nvmepf.0.nqn \
                /sys/kernel/config/nvmet/ports/1/subsystems/nvmepf.0.nqn

NVMe PCI endpoint function 생성과 시작

Subsystem과 port가 준비되면 configfs의 `nvmet_pci_epf` 아래에 function 0을 만듭니다. Port 생성 시 driver가 자동으로 load되어 있어야 합니다.

# ls /sys/kernel/config/pci_ep/functions
nvmet_pci_epf
# cd /sys/kernel/config/pci_ep/functions/nvmet_pci_epf
# mkdir nvmepf.0
# ls nvmepf.0/

Device ID는 임의 값을 사용할 수 있습니다. Vendor ID는 NVMe target subsystem vendor ID와 같은 값으로 자동 설정됩니다. MSI-X vector 32개를 설정하고, endpoint controller가 MSI-X를 지원하지 않으면 대신 MSI를 설정합니다.

# echo 0xBEEF > nvmepf.0/deviceid
# echo 32 > nvmepf.0/msix_interrupts
# echo 32 > nvmepf.0/msi_interrupts

Function을 target port 1과 subsystem NQN에 연결합니다.

# echo 1 > nvmepf.0/nvme/portid
# echo "nvmepf.0.nqn" > nvmepf.0/nvme/subsysnqn

그 다음 endpoint function을 endpoint controller에 symbolic link로 bind하고 controller를 시작합니다.

# cd /sys/kernel/config/pci_ep
# ln -s functions/nvmet_pci_epf/nvmepf.0 controllers/a40000000.pcie-ep/
# echo 1 > controllers/a40000000.pcie-ep/start

Endpoint machine의 kernel message에는 `nullb0` 생성, namespace 추가, MSI-X vector 수, NVMe controller와 PCI controller 생성 정보가 나타납니다. 예제에서는 I/O queue 4개와 MDTS 524288B가 보고됩니다.

Creating an NVMe PCI Endpoint Device
------------------------------------

With the NVMe target subsystem and port ready for use, the NVMe PCI endpoint
device can now be created and enabled. The NVMe PCI endpoint target driver
should already be loaded (that is done automatically when the port is created)::

        # ls /sys/kernel/config/pci_ep/functions
        nvmet_pci_epf

Next, create function 0::

        # cd /sys/kernel/config/pci_ep/functions/nvmet_pci_epf
        # mkdir nvmepf.0
        # ls nvmepf.0/
        baseclass_code    msix_interrupts   secondary
        cache_line_size   nvme              subclass_code
        deviceid          primary           subsys_id
        interrupt_pin     progif_code       subsys_vendor_id
        msi_interrupts    revid             vendorid

Configure the function using any device ID (the vendor ID for the device will
be automatically set to the same value as the NVMe target subsystem vendor
ID)::

        # cd /sys/kernel/config/pci_ep/functions/nvmet_pci_epf
        # echo 0xBEEF > nvmepf.0/deviceid
        # echo 32 > nvmepf.0/msix_interrupts

If the PCI endpoint controller used does not support MSI-X, MSI can be
configured instead::

        # echo 32 > nvmepf.0/msi_interrupts

Next, let's bind our endpoint device with the target subsystem and port that we
created::

        # echo 1 > nvmepf.0/nvme/portid
        # echo "nvmepf.0.nqn" > nvmepf.0/nvme/subsysnqn

The endpoint function can then be bound to the endpoint controller and the
controller started::

        # cd /sys/kernel/config/pci_ep
        # ln -s functions/nvmet_pci_epf/nvmepf.0 controllers/a40000000.pcie-ep/
        # echo 1 > controllers/a40000000.pcie-ep/start

On the endpoint machine, kernel messages will show information as the NVMe
target device and endpoint device are created and connected.

.. code-block:: text

        null_blk: disk nullb0 created
        null_blk: module loaded
        nvmet: adding nsid 1 to subsystem nvmepf.0.nqn
        nvmet_pci_epf nvmet_pci_epf.0: PCI endpoint controller supports MSI-X, 32 vectors
        nvmet: Created nvm controller 1 for subsystem nvmepf.0.nqn for NQN nqn.2014-08.org.nvmexpress:uuid:2ab90791-2246-4fbb-961d-4c3d5a5a0176.
        nvmet_pci_epf nvmet_pci_epf.0: New PCI ctrl "nvmepf.0.nqn", 4 I/O queues, mdts 524288 B

PCI Root-Complex host 확인

PCI host를 boot하면 PCIe link가 초기화되고, host NVMe driver가 device controller를 활성화할 때 endpoint에 `Enabling controller` message가 표시됩니다.

Host에서는 설정한 vendor ID와 device ID를 가진 PCI device로 검색됩니다.

# lspci -n
0000:01:00.0 0108: 1b96:beef

NVMe device는 namespace 하나를 가진 250GB block device로 인식됩니다.

# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
nvme0n1     259:0    0   250G  0 disk

Endpoint block device는 일반 NVMe namespace block device처럼 사용할 수 있습니다. `nvme id-ctrl /dev/nvme0`로 vendor, subsystem vendor, serial, model, firmware, MDTS, controller ID, NVMe version 등의 상세 정보를 조회할 수 있습니다.

PCI Root-Complex Host
---------------------

Booting the PCI host will result in the initialization of the PCIe link (this
may be signaled by the PCI endpoint driver with a kernel message). A kernel
message on the endpoint will also signal when the host NVMe driver enables the
device controller::

        nvmet_pci_epf nvmet_pci_epf.0: Enabling controller

On the host side, the NVMe PCI endpoint function target device is
discoverable as a PCI device, with the vendor ID and device ID as configured::

        # lspci -n
        0000:01:00.0 0108: 1b96:beef

An this device will be recognized as an NVMe device with a single namespace::

        # lsblk
        NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
        nvme0n1     259:0    0   250G  0 disk

The NVMe endpoint block device can then be used as any other regular NVMe
namespace block device. The *nvme* command line utility can be used to get more
detailed information about the endpoint device::

        # nvme id-ctrl /dev/nvme0
        NVME Identify Controller:
        vid       : 0x1b96
        ssvid     : 0x1b96
        sn        : 94993c85650ef7bcd625
        mn        : Linux-pci-epf
        fr        : 6.13.0-r
        rab       : 6
        ieee      : 000000
        cmic      : 0xb
        mdts      : 7
        cntlid    : 0x1
        ver       : 0x20100
        ...

Configfs endpoint binding 속성

NVMe PCI endpoint target driver가 사용하는 PCI endpoint configfs device attribute는 다음과 같습니다.

Function directory의 `nvme` 하위 directory에는 target 전용 설정 field도 있습니다.

Endpoint Bindings
=================

The NVMe PCI endpoint target driver uses the PCI endpoint configfs device
attributes as follows.

================   ===========================================================
vendorid           Ignored (the vendor id of the NVMe target subsystem is used)
deviceid           Anything is OK (e.g. PCI_ANY_ID)
revid              Do not care
progif_code        Must be 0x02 (NVM Express)
baseclass_code     Must be 0x01 (PCI_BASE_CLASS_STORAGE)
subclass_code      Must be 0x08 (Non-Volatile Memory controller)
cache_line_size    Do not care
subsys_vendor_id   Ignored (the subsystem vendor id of the NVMe target subsystem
                   is used)
subsys_id          Anything is OK (e.g. PCI_ANY_ID)
msi_interrupts     At least equal to the number of queue pairs desired
msix_interrupts    At least equal to the number of queue pairs desired
interrupt_pin      Interrupt PIN to use if MSI and MSI-X are not supported
================   ===========================================================

The NVMe PCI endpoint target function also has some specific configurable
fields defined in the *nvme* subdirectory of the function directory. These
fields are as follows.

================   ===========================================================
mdts_kb            Maximum data transfer size in KiB (default: 512)
portid             The ID of the target port to use
subsysnqn          The NQN of the target subsystem to use
================   ===========================================================