PCI endpoint framework

문서 목적

저자는 Kishon Vijay Abraham I입니다. 이 문서는 PCI Endpoint Framework로 endpoint controller driver와 endpoint function driver를 만들고, configfs interface로 function driver를 controller driver에 bind하는 방법을 안내합니다.

.. SPDX-License-Identifier: GPL-2.0

:Author: Kishon Vijay Abraham I <[email protected]>

This document is a guide to use the PCI Endpoint Framework in order to create
endpoint controller driver, endpoint function driver, and using configfs
interface to bind the function driver to the controller driver.

Root Complex와 endpoint mode

Linux PCI subsystem은 Root Complex mode로 동작하는 PCI controller를 폭넓게 지원합니다. PCI bus scan, memory·IRQ resource 할당, vendor ID·device ID 기반 driver load, hotplug, power management, advanced error reporting, virtual channel 등의 기능이 있습니다.

일부 SoC의 PCI controller IP는 Root Complex mode와 Endpoint mode 중 하나로 동작할 수 있습니다. PCI Endpoint Framework는 Linux에 endpoint mode를 추가해 test·validation이나 co-processor accelerator 등 다양한 EP system use case를 지원합니다.

Introduction
============

Linux has a comprehensive PCI subsystem to support PCI controllers that
operates in Root Complex mode. The subsystem has capability to scan PCI bus,
assign memory resources and IRQ resources, load PCI driver (based on
vendor ID, device ID), support other services like hot-plug, power management,
advanced error reporting and virtual channels.

However the PCI controller IP integrated in some SoCs is capable of operating
either in Root Complex mode or Endpoint mode. PCI Endpoint Framework will
add endpoint mode support in Linux. This will help to run Linux in an
EP system which can have a wide variety of use cases from testing or
validation, co-processor accelerator, etc.

Endpoint Core와 EPC library

PCI Endpoint Core layer는 Endpoint Controller(`EPC`) library, Endpoint Function(`EPF`) library, EPF를 EPC에 bind하는 configfs layer의 세 component로 구성됩니다.

EPC library는 endpoint mode로 동작하는 controller driver가 사용할 API와 특정 endpoint function을 구현하는 function driver/library가 사용할 API를 제공합니다.

PCI Endpoint Core
=================

The PCI Endpoint Core layer comprises 3 components: the Endpoint Controller
library, the Endpoint Function library, and the configfs layer to bind the
endpoint function with the endpoint controller.

PCI Endpoint Controller(EPC) Library
------------------------------------

The EPC library provides APIs to be used by the controller that can operate
in endpoint mode. It also provides APIs to be used by function driver/library
in order to implement a particular endpoint function.

PCI controller driver용 EPC API

PCI Endpoint Core가 PCI controller driver에 제공하는 API입니다.

APIs for the PCI controller Driver
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This section lists the APIs that the PCI Endpoint core provides to be used
by the PCI controller driver.

* devm_pci_epc_create()/pci_epc_create()

   The PCI controller driver should implement the following ops:

         * write_header: ops to populate configuration space header
         * set_bar: ops to configure the BAR
         * clear_bar: ops to reset the BAR
         * alloc_addr_space: ops to allocate in PCI controller address space
         * free_addr_space: ops to free the allocated address space
         * raise_irq: ops to raise a legacy, MSI or MSI-X interrupt
         * start: ops to start the PCI link
         * stop: ops to stop the PCI link

   The PCI controller driver can then create a new EPC device by invoking
   devm_pci_epc_create()/pci_epc_create().

* pci_epc_destroy()

   The PCI controller driver can destroy the EPC device created by
   pci_epc_create() using pci_epc_destroy().

* pci_epc_linkup()

   In order to notify all the function devices that the EPC device to which
   they are linked has established a link with the host, the PCI controller
   driver should invoke pci_epc_linkup().

* pci_epc_mem_init()

   Initialize the pci_epc_mem structure used for allocating EPC addr space.

* pci_epc_mem_exit()

   Cleanup the pci_epc_mem structure allocated during pci_epc_mem_init().

EPF driver가 사용하는 EPC API

PCI endpoint function driver는 아래 EPC API로 configuration header, BAR, interrupt와 RC memory mapping을 제어합니다.

일부 endpoint controller는 mapping 가능한 RC PCI address에 제약을 둡니다. `pci_epc_mem_map()`은 이 제약을 처리하면서 controller memory를 할당하고 mapping합니다.

`pci_epc_mem_map()`은 RC PCI address range를 mapping하는 데 필요한 allocation size, 실제 mapping된 PCI range 크기, mapping 접근에 사용할 allocated memory 내부 offset을 반환합니다. 실제 mapping 크기는 요청보다 작을 수 있습니다.

`pci_epc_mem_unmap()`은 `pci_epc_mem_map()`으로 할당하고 mapping한 controller memory의 mapping을 해제하고 memory를 반환합니다.

EPC APIs for the PCI Endpoint Function Driver
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This section lists the APIs that the PCI Endpoint core provides to be used
by the PCI endpoint function driver.

* pci_epc_write_header()

   The PCI endpoint function driver should use pci_epc_write_header() to
   write the standard configuration header to the endpoint controller.

* pci_epc_set_bar()

   The PCI endpoint function driver should use pci_epc_set_bar() to configure
   the Base Address Register in order for the host to assign PCI addr space.
   Register space of the function driver is usually configured
   using this API.

* pci_epc_clear_bar()

   The PCI endpoint function driver should use pci_epc_clear_bar() to reset
   the BAR.

* pci_epc_raise_irq()

   The PCI endpoint function driver should use pci_epc_raise_irq() to raise
   Legacy Interrupt, MSI or MSI-X Interrupt.

* pci_epc_mem_alloc_addr()

   The PCI endpoint function driver should use pci_epc_mem_alloc_addr(), to
   allocate memory address from EPC addr space which is required to access
   RC's buffer

* pci_epc_mem_free_addr()

   The PCI endpoint function driver should use pci_epc_mem_free_addr() to
   free the memory space allocated using pci_epc_mem_alloc_addr().

* pci_epc_map_addr()

  A PCI endpoint function driver should use pci_epc_map_addr() to map to a RC
  PCI address the CPU address of local memory obtained with
  pci_epc_mem_alloc_addr().

* pci_epc_unmap_addr()

  A PCI endpoint function driver should use pci_epc_unmap_addr() to unmap the
  CPU address of local memory mapped to a RC address with pci_epc_map_addr().

* pci_epc_mem_map()

  A PCI endpoint controller may impose constraints on the RC PCI addresses that
  can be mapped. The function pci_epc_mem_map() allows endpoint function
  drivers to allocate and map controller memory while handling such
  constraints. This function will determine the size of the memory that must be
  allocated with pci_epc_mem_alloc_addr() for successfully mapping a RC PCI
  address range. This function will also indicate the size of the PCI address
  range that was actually mapped, which can be less than the requested size, as
  well as the offset into the allocated memory to use for accessing the mapped
  RC PCI address range.

* pci_epc_mem_unmap()

  A PCI endpoint function driver can use pci_epc_mem_unmap() to unmap and free
  controller memory that was allocated and mapped using pci_epc_mem_map().

기타 EPC binding API

EPC library의 나머지 API는 EPF device를 EPC device에 bind할 때 사용하며 `pci-ep-cfs.c`가 참고 구현입니다.

Other EPC APIs
~~~~~~~~~~~~~~

There are other APIs provided by the EPC library. These are used for binding
the EPF device with EPC device. pci-ep-cfs.c can be used as reference for
using these APIs.

* pci_epc_get()

   Get a reference to the PCI endpoint controller based on the device name of
   the controller.

* pci_epc_put()

   Release the reference to the PCI endpoint controller obtained using
   pci_epc_get()

* pci_epc_add_epf()

   Add a PCI endpoint function to a PCI endpoint controller. A PCIe device
   can have up to 8 functions according to the specification.

* pci_epc_remove_epf()

   Remove the PCI endpoint function from PCI endpoint controller.

* pci_epc_start()

   The PCI endpoint function driver should invoke pci_epc_start() once it
   has configured the endpoint function and wants to start the PCI link.

* pci_epc_stop()

   The PCI endpoint function driver should invoke pci_epc_stop() to stop
   the PCI LINK.

EPF library

EPF library는 function driver와 EPC library가 endpoint mode 기능을 제공할 때 사용할 API를 제공합니다.

PCI Endpoint Function(EPF) Library
----------------------------------

The EPF library provides APIs to be used by the function driver and the EPC
library to provide endpoint mode functionality.

Endpoint function driver용 EPF API

EPF driver는 `pci_epf_register_driver()`로 등록하며 다음 operation을 구현합니다.

`pci_epf_unregister_driver()`는 EPF driver 등록을 해제합니다.

`pci_epf_alloc_space()`는 특정 BAR용 space를 할당하고 `pci_epf_free_space()`는 그 space를 해제합니다.

EPF APIs for the PCI Endpoint Function Driver
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This section lists the APIs that the PCI Endpoint core provides to be used
by the PCI endpoint function driver.

* pci_epf_register_driver()

   The PCI Endpoint Function driver should implement the following ops:
         * bind: ops to perform when an EPC device has been bound to EPF device
         * unbind: ops to perform when a binding has been lost between an EPC
           device and EPF device
         * add_cfs: optional ops to create function specific configfs
           attributes

  The PCI Function driver can then register the PCI EPF driver by using
  pci_epf_register_driver().

* pci_epf_unregister_driver()

  The PCI Function driver can unregister the PCI EPF driver by using
  pci_epf_unregister_driver().

* pci_epf_alloc_space()

  The PCI Function driver can allocate space for a particular BAR using
  pci_epf_alloc_space().

* pci_epf_free_space()

  The PCI Function driver can free the allocated space
  (using pci_epf_alloc_space) by invoking pci_epf_free_space().

EPC library의 EPF link notification

EPC device가 host와 connection을 맺으면 PCI endpoint controller library가 `pci_epf_linkup()`을 호출해 function 측에 link-up을 알립니다.

APIs for the PCI Endpoint Controller Library
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This section lists the APIs that the PCI Endpoint core provides to be used
by the PCI endpoint controller library.

* pci_epf_linkup()

   The PCI endpoint controller library invokes pci_epf_linkup() when the
   EPC device has established the connection to the host.

기타 EPF device API

다른 EPF library API는 EPF device가 EPC device에 bind되거나 unbind될 때 function driver에 알리는 데 사용하며 `pci-ep-cfs.c`가 참고 구현입니다.

Other EPF APIs
~~~~~~~~~~~~~~

There are other APIs provided by the EPF library. These are used to notify
the function driver when the EPF device is bound to the EPC device.
pci-ep-cfs.c can be used as reference for using these APIs.

* pci_epf_create()

   Create a new PCI EPF device by passing the name of the PCI EPF device.
   This name will be used to bind the EPF device to a EPF driver.

* pci_epf_destroy()

   Destroy the created PCI EPF device.

* pci_epf_bind()

   pci_epf_bind() should be invoked when the EPF device has been bound to
   an EPC device.

* pci_epf_unbind()

   pci_epf_unbind() should be invoked when the binding between EPC device
   and EPF device is lost.