AF_XDP TX metadata

AF_XDP 송신 offload metadata

이 문서는 `af_xdp`를 통해 packet을 송신할 때 offload를 활성화하는 방법을 설명합니다. 수신 측의 유사한 metadata를 읽는 방법은 `xdp-rx-metadata` 문서를 참조합니다.

.. SPDX-License-Identifier: GPL-2.0

==================
AF_XDP TX Metadata
==================

This document describes how to enable offloads when transmitting packets
via :doc:`af_xdp`. Refer to :doc:`xdp-rx-metadata` on how to access similar
metadata on the receive side.

UMEM metadata 예약과 배치

Metadata용 headroom은 `struct xdp_umem_reg`의 `tx_metadata_len`과 `XDP_UMEM_TX_METADATA_LEN` flag로 예약합니다. 따라서 같은 UMEM을 공유하는 모든 socket은 동일한 metadata 길이를 사용합니다.

Metadata layout은 고정 UAPI이며 `include/uapi/linux/if_xdp.h`의 `union xsk_tx_metadata`에 정의됩니다. 일반적으로 `tx_metadata_len`에는 `sizeof(union xsk_tx_metadata)`를 넣어야 합니다.

초기 구현에서는 `XDP_UMEM_TX_METADATA_LEN` flag가 필수가 아니었습니다. 호환성이 필요한 application은 먼저 flag를 넣어 UMEM 생성을 시도하고, 실패하면 flag 없이 다시 시도할 수 있습니다.

Headroom과 metadata는 UMEM frame 안에서 `xdp_desc->addr` 바로 앞에 위치해야 합니다. Kernel은 `xdp_desc->addr - tx_metadata_len`에서 `xsk_tx_metadata`를 찾습니다.

Application은 `sizeof(struct xsk_tx_metadata)`보다 큰 headroom을 요청할 수 있습니다. Kernel은 여분의 padding을 무시합니다. `XDP_TX_METADATA` option이 없는 frame에서는 metadata 영역 전체를 무시합니다.

General Design
==============

The headroom for the metadata is reserved via ``tx_metadata_len`` and
``XDP_UMEM_TX_METADATA_LEN`` flag in ``struct xdp_umem_reg``. The metadata
length is therefore the same for every socket that shares the same umem.
The metadata layout is a fixed UAPI, refer to ``union xsk_tx_metadata`` in
``include/uapi/linux/if_xdp.h``. Thus, generally, the ``tx_metadata_len``
field above should contain ``sizeof(union xsk_tx_metadata)``.

Note that in the original implementation the ``XDP_UMEM_TX_METADATA_LEN``
flag was not required. Applications might attempt to create a umem
with a flag first and if it fails, do another attempt without a flag.

The headroom and the metadata itself should be located right before
``xdp_desc->addr`` in the umem frame. Within a frame, the metadata
layout is as follows::

           tx_metadata_len
     /                         \
    +-----------------+---------+----------------------------+
    | xsk_tx_metadata | padding |          payload           |
    +-----------------+---------+----------------------------+
                                ^
                                |
                          xdp_desc->addr

An AF_XDP application can request headrooms larger than ``sizeof(struct
xsk_tx_metadata)``. The kernel will ignore the padding (and will still
use ``xdp_desc->addr - tx_metadata_len`` to locate
the ``xsk_tx_metadata``). For the frames that shouldn't carry
any metadata (i.e., the ones that don't have ``XDP_TX_METADATA`` option),
the metadata area is ignored by the kernel as well.

TX metadata flag와 multi-buffer 규칙

`flags` field는 사용할 offload를 선택합니다.

Checksum offload에서 `csum_start`는 checksum 계산을 시작할 byte offset이고, `csum_offset`은 계산 결과를 device가 저장할 byte offset입니다. Launch-time offload의 예약 시각은 `union xsk_tx_metadata`의 `launch_time`에 넣습니다.

Offload를 실제로 작동시키려면 첫 packet의 `struct xdp_desc` descriptor가 `options` field에 `XDP_TX_METADATA` bit를 설정해야 합니다. Multi-buffer packet에서는 첫 chunk에만 metadata를 둡니다.

The flags field enables the particular offload:

- ``XDP_TXMD_FLAGS_TIMESTAMP``: requests the device to put transmission
  timestamp into ``tx_timestamp`` field of ``union xsk_tx_metadata``.
- ``XDP_TXMD_FLAGS_CHECKSUM``: requests the device to calculate L4
  checksum. ``csum_start`` specifies byte offset of where the checksumming
  should start and ``csum_offset`` specifies byte offset where the
  device should store the computed checksum.
- ``XDP_TXMD_FLAGS_LAUNCH_TIME``: requests the device to schedule the
  packet for transmission at a pre-determined time called launch time. The
  value of launch time is indicated by ``launch_time`` field of
  ``union xsk_tx_metadata``.

Besides the flags above, in order to trigger the offloads, the first
packet's ``struct xdp_desc`` descriptor should set ``XDP_TX_METADATA``
bit in the ``options`` field. Also note that in a multi-buffer packet
only the first chunk should carry the metadata.

Software TX checksum

개발과 시험 목적으로 `XDP_UMEM_REG` UMEM 등록 호출에 `XDP_UMEM_TX_SW_CSUM` flag를 전달할 수 있습니다. 이 경우 `XDK_COPY` mode에서 CPU가 TX checksum을 계산합니다.

이 option은 성능을 떨어뜨리므로 production에서는 활성화하지 않아야 합니다.

Software TX Checksum
====================

For development and testing purposes its possible to pass
``XDP_UMEM_TX_SW_CSUM`` flag to ``XDP_UMEM_REG`` UMEM registration call.
In this case, when running in ``XDK_COPY`` mode, the TX checksum
is calculated on the CPU. Do not enable this option in production because
it will negatively affect performance.

Launch time의 시계·순서·horizon

정확도를 확보하려면 요청한 launch time을 device의 PTP Hardware Clock(PHC)을 기준으로 표현해야 합니다.

AF_XDP는 packet을 정렬하고 송신을 지연하는 ETF queuing discipline과 다른 data path를 사용합니다. AF_XDP는 순서를 재배치하거나 보류하지 않고 즉시 device driver에 packet을 넘깁니다.

Driver가 FIFO 순서를 유지하므로 launch time을 요청한 packet은 실제로 송신될 때까지 같은 Tx Queue의 뒤 packet을 막습니다. 미래 송신을 예약하는 traffic에는 별도 queue를 할당하는 것이 좋습니다.

Launch-time offload가 꺼져 있으면 device driver는 launch-time 요청을 무시해야 합니다. 값은 device가 미래에 programming할 수 있는 최장 시점인 horizon을 넘으면 안 되며, 정확한 한계는 hardware마다 다릅니다.

Launch Time
===========

The value of the requested launch time should be based on the device's PTP
Hardware Clock (PHC) to ensure accuracy. AF_XDP takes a different data path
compared to the ETF queuing discipline, which organizes packets and delays
their transmission. Instead, AF_XDP immediately hands off the packets to
the device driver without rearranging their order or holding them prior to
transmission. Since the driver maintains FIFO behavior and does not perform
packet reordering, a packet with a launch time request will block other
packets in the same Tx Queue until it is sent. Therefore, it is recommended
to allocate separate queue for scheduling traffic that is intended for
future transmission.

In scenarios where the launch time offload feature is disabled, the device
driver is expected to disregard the launch time request. For correct
interpretation and meaningful operation, the launch time should never be
set to a value larger than the farthest programmable time in the future
(the horizon). Different devices have different hardware limitations on the
launch time offload feature.

stmmac driver

stmmac에서는 개별 Tx Queue마다 TSO와 launch time(TBS)을 동시에 사용할 수 없습니다. 기본 설정은 Tx Queue 0에서 TSO를 지원하고 나머지 Tx Queue에서 TBS를 지원합니다.

`tc-etf` 명령이 driver의 `ndo_setup_tc()` callback을 호출하도록 해 launch-time hardware offload를 켜거나 끌 수 있습니다.

Enhanced Normal Transmit Descriptor의 launch time은 32bit입니다. 상위 8bit는 초, 나머지 24bit는 256ns 단위 시간을 나타냅니다. 값은 PTP time의 `bits[39:8]`과 비교되며 256초마다 rollover됩니다.

따라서 dwmac4와 dwxlgmac2의 미래 launch-time horizon은 128초입니다.

stmmac driver
-------------

For stmmac, TSO and launch time (TBS) features are mutually exclusive for
each individual Tx Queue. By default, the driver configures Tx Queue 0 to
support TSO and the rest of the Tx Queues to support TBS. The launch time
hardware offload feature can be enabled or disabled by using the tc-etf
command to call the driver's ndo_setup_tc() callback.

The value of the launch time that is programmed in the Enhanced Normal
Transmit Descriptors is a 32-bit value, where the most significant 8 bits
represent the time in seconds and the remaining 24 bits represent the time
in 256 ns increments. The programmed launch time is compared against the
PTP time (bits[39:8]) and rolls over after 256 seconds. Therefore, the
horizon of the launch time for dwmac4 and dwxlgmac2 is 128 seconds in the
future.

igc driver

igc에서는 네 Tx Queue 모두 launch-time 기능을 지원합니다. `tc-etf`가 `ndo_setup_tc()` callback을 호출하도록 해 hardware offload를 켜거나 끕니다.

TSN mode에 진입하면 igc driver는 device를 reset하고 cycle time이 1초이며 모든 Tx Queue가 항상 열린 기본 Qbv schedule을 만듭니다.

Advanced Transmit Context Descriptor에 programming되는 launch time은 해당 queue의 Qbv transmission window 시작 시각에 대한 상대 offset입니다. Descriptor의 `Frst` flag를 설정하면 다음 Qbv cycle에 packet을 예약할 수 있습니다.

i225와 i226의 horizon은 queue의 다음 Qbv transmission window cycle 종료 시점입니다. 예를 들어 cycle이 1초라면 현재 cycle 위치에 따라 horizon은 1초에서 2초 사이입니다.

igc driver
----------

For igc, all four Tx Queues support the launch time feature. The launch
time hardware offload feature can be enabled or disabled by using the
tc-etf command to call the driver's ndo_setup_tc() callback. When entering
TSN mode, the igc driver will reset the device and create a default Qbv
schedule with a 1-second cycle time, with all Tx Queues open at all times.

The value of the launch time that is programmed in the Advanced Transmit
Context Descriptor is a relative offset to the starting time of the Qbv
transmission window of the queue. The Frst flag of the descriptor can be
set to schedule the packet for the next Qbv cycle. Therefore, the horizon
of the launch time for i225 and i226 is the ending time of the next cycle
of the Qbv transmission window of the queue. For example, when the Qbv
cycle time is set to 1 second, the horizon of the launch time ranges
from 1 second to 2 seconds, depending on where the Qbv cycle is currently
running.

Device capability 조회

각 device는 netlink `netdev` family를 통해 offload capability를 공개합니다. `Documentation/netlink/specs/netdev.yaml`의 `xsk-flags` feature bitmask를 참조합니다.

조회 구현 예시는 `tools/net/ynl/samples/netdev.c`에서 확인할 수 있습니다.

Querying Device Capabilities
============================

Every devices exports its offloads capabilities via netlink netdev family.
Refer to ``xsk-flags`` features bitmask in
``Documentation/netlink/specs/netdev.yaml``.

- ``tx-timestamp``: device supports ``XDP_TXMD_FLAGS_TIMESTAMP``
- ``tx-checksum``: device supports ``XDP_TXMD_FLAGS_CHECKSUM``
- ``tx-launch-time-fifo``: device supports ``XDP_TXMD_FLAGS_LAUNCH_TIME``

See ``tools/net/ynl/samples/netdev.c`` on how to query this information.

예제

TX metadata를 처리하는 예제 program은 `tools/testing/selftests/bpf/xdp_hw_metadata.c`에 있습니다. 더 단순한 예제는 `https://github.com/fomichev/xskgen`을 참조합니다.

Example
=======

See ``tools/testing/selftests/bpf/xdp_hw_metadata.c`` for an example
program that handles TX metadata. Also see https://github.com/fomichev/xskgen
for a more bare-bones example.