Multi-PF Netdev

배경과 netdev 통합 model

Multi-PF NIC는 multi-socket server의 여러 CPU가 각자의 전용 PCIe interface를 통해 network에 직접 연결되게 합니다. 두 card 사이에서 PCIe lane을 나누는 harness 또는 한 card용 PCIe slot bifurcation을 사용해 socket 사이 internal bus를 건너는 network traffic을 없앱니다. 그 결과 overhead와 latency, CPU 사용률을 줄이고 throughput을 높입니다.

기능은 같은 port에 속한 여러 PF를 netdev layer에서 하나의 netdev instance로 결합합니다. PCI function, sysfs entry, devlink 같은 lower-layer instance는 분리된 채 유지됩니다. 서로 다른 NUMA socket의 device로 traffic을 통과시키면 cross-NUMA traffic을 줄이고, 같은 netdev를 쓰는 각 NUMA의 application이 가까운 device를 이용해 성능을 높일 수 있습니다.

.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>

===============
Multi-PF Netdev
===============

Contents
========

- `Background`_
- `Overview`_
- `mlx5 implementation`_
- `Channels distribution`_
- `Observability`_
- `Steering`_
- `Mutually exclusive features`_

Background
==========

The Multi-PF NIC technology enables several CPUs within a multi-socket server to connect directly to
the network, each through its own dedicated PCIe interface. Through either a connection harness that
splits the PCIe lanes between two cards or by bifurcating a PCIe slot for a single card. This
results in eliminating the network traffic traversing over the internal bus between the sockets,
significantly reducing overhead and latency, in addition to reducing CPU utilization and increasing
network throughput.

Overview
========

The feature adds support for combining multiple PFs of the same port in a Multi-PF environment under
one netdev instance. It is implemented in the netdev layer. Lower-layer instances like pci func,
sysfs entry, and devlink are kept separate.
Passing traffic through different devices belonging to different NUMA sockets saves cross-NUMA
traffic and allows apps running on the same netdev from different NUMAs to still feel a sense of
proximity to the device and achieve improved performance.

mlx5 socket-direct 구현

mlx5의 Multi-PF 또는 Socket-direct는 같은 NIC에 속하고 socket-direct property가 켜진 PF를 group으로 묶습니다. 모든 PF probe가 끝나면 전체를 대표하는 netdev 하나를 만들고 PF 하나라도 제거되면 netdev를 파괴합니다. Network channel은 모든 device에 나누며 application과 CPU가 있는 가까운 NUMA node를 사용해야 최적 성능을 냅니다.

PF 하나를 primary leader로 고르고 secondary는 chip level에서 network와 끊긴 silent mode로 둡니다. Secondary에는 south↔north traffic이 직접 흐르지 않고 leader PF의 east↔west assistance를 사용합니다. 모든 Rx/Tx traffic은 primary를 통해 secondary로 또는 secondary에서 steering됩니다. 현재는 PF만 지원하고 최대 두 PF, 즉 두 socket으로 제한합니다.

mlx5 implementation
===================

Multi-PF or Socket-direct in mlx5 is achieved by grouping PFs together which belong to the same
NIC and has the socket-direct property enabled, once all PFs are probed, we create a single netdev
to represent all of them, symmetrically, we destroy the netdev whenever any of the PFs is removed.

The netdev network channels are distributed between all devices, a proper configuration would utilize
the correct close NUMA node when working on a certain app/CPU.

We pick one PF to be a primary (leader), and it fills a special role. The other devices
(secondaries) are disconnected from the network at the chip level (set to silent mode). In silent
mode, no south <-> north traffic flowing directly through a secondary PF. It needs the assistance of
the leader PF (east <-> west traffic) to function. All Rx/Tx traffic is steered through the primary
to/from the secondaries.

Currently, we limit the support to PFs only, and up to two PFs (sockets).

Channel의 round-robin 분배

여러 NUMA node에서 local 성능을 얻도록 channel을 PF 사이에 round-robin으로 분배합니다. 결합된 channel 하나는 특정 PF의 core device instance인 `mdev` 하나를 사용하고 모든 datapath queue도 그 PF에 만듭니다. 모든 channel이 `priv->mdev` 하나를 공유하지 않습니다.

Round-robin을 택하면 user가 channel 수를 바꾸어도 channel index와 PF mapping이 고정됩니다. Channel을 닫아도 statistics는 유지되므로 매번 mapping이 바뀌면 누적 통계가 해당 channel의 역사를 제대로 나타내지 못합니다.

Channels distribution
=====================

We distribute the channels between the different PFs to achieve local NUMA node performance
on multiple NUMA nodes.

Each combined channel works against one specific PF, creating all its datapath queues against it. We
distribute channels to PFs in a round-robin policy.

::

        Example for 2 PFs and 5 channels:
        +--------+--------+
        | ch idx | PF idx |
        +--------+--------+
        |    0   |    0   |
        |    1   |    1   |
        |    2   |    0   |
        |    3   |    1   |
        |    4   |    0   |
        +--------+--------+


The reason we prefer round-robin is, it is less influenced by changes in the number of channels. The
mapping between a channel index and a PF is fixed, no matter how many channels the user configures.
As the channel stats are persistent across channel's closure, changing the mapping every single time
would turn the accumulative stats less representing of the channel's history.

This is achieved by using the correct core device instance (mdev) in each channel, instead of them
all using the same instance under "priv->mdev".

Queue·NAPI·IRQ 관찰

PF, IRQ, NAPI, queue 관계는 `Documentation/netlink/specs/netdev.yaml`을 사용하는 YNL `queue-get`과 `napi-get` dump로 관찰할 수 있습니다. 예제의 RX와 TX queue 0~4는 NAPI ID 539~543에 연결되고, 각 NAPI ID는 IRQ 36, 39, 40, 41, 42에 대응합니다.

`/proc/irq/.../mlx5*` 경로를 보면 IRQ가 `pci:0000:08:00.0`과 `pci:0000:09:00.0` 사이에 번갈아 배치되어 channel round-robin policy가 실제 PF에 반영된 것을 확인할 수 있습니다. Function과 숫자 및 source path는 원문 예제 그대로 보존합니다.

Observability
=============
The relation between PF, irq, napi, and queue can be observed via netlink spec::

  $ ./tools/net/ynl/pyynl/cli.py --spec Documentation/netlink/specs/netdev.yaml --dump queue-get --json='{"ifindex": 13}'
  [{'id': 0, 'ifindex': 13, 'napi-id': 539, 'type': 'rx'},
   {'id': 1, 'ifindex': 13, 'napi-id': 540, 'type': 'rx'},
   {'id': 2, 'ifindex': 13, 'napi-id': 541, 'type': 'rx'},
   {'id': 3, 'ifindex': 13, 'napi-id': 542, 'type': 'rx'},
   {'id': 4, 'ifindex': 13, 'napi-id': 543, 'type': 'rx'},
   {'id': 0, 'ifindex': 13, 'napi-id': 539, 'type': 'tx'},
   {'id': 1, 'ifindex': 13, 'napi-id': 540, 'type': 'tx'},
   {'id': 2, 'ifindex': 13, 'napi-id': 541, 'type': 'tx'},
   {'id': 3, 'ifindex': 13, 'napi-id': 542, 'type': 'tx'},
   {'id': 4, 'ifindex': 13, 'napi-id': 543, 'type': 'tx'}]

  $ ./tools/net/ynl/pyynl/cli.py --spec Documentation/netlink/specs/netdev.yaml --dump napi-get --json='{"ifindex": 13}'
  [{'id': 543, 'ifindex': 13, 'irq': 42},
   {'id': 542, 'ifindex': 13, 'irq': 41},
   {'id': 541, 'ifindex': 13, 'irq': 40},
   {'id': 540, 'ifindex': 13, 'irq': 39},
   {'id': 539, 'ifindex': 13, 'irq': 36}]

Here you can clearly observe our channels distribution policy::

  $ ls /proc/irq/{36,39,40,41,42}/mlx5* -d -1
  /proc/irq/36/mlx5_comp0@pci:0000:08:00.0
  /proc/irq/39/mlx5_comp0@pci:0000:09:00.0
  /proc/irq/40/mlx5_comp1@pci:0000:08:00.0
  /proc/irq/41/mlx5_comp1@pci:0000:09:00.0
  /proc/irq/42/mlx5_comp2@pci:0000:08:00.0

Primary steering과 기본 XPS

Secondary PF는 silent mode라 network에서 분리됩니다. RX steering table은 primary PF만 소유하며 cross-vhca steering으로 incoming traffic을 다른 PF에 분배합니다. 기본 RSS table은 하나지만 다른 PF의 receive queue를 가리킬 수 있습니다. TX에서는 primary가 새 Tx flow table을 만들고 secondary가 alias하여 network로 나갑니다.

기본 XPS configuration은 송신 CPU와 같은 NUMA node의 PF에 속한 SQ를 선택합니다. 예제는 node0 CPU 0~11과 PF0, node1 CPU 12~23과 PF1을 짝지으며 tx queue 0~23의 `xps_cpus` mask가 두 node 사이에 교대로 배치됩니다. 모든 sysfs mask와 queue path는 아래 보존된 원문에서 확인할 수 있습니다.

Steering
========
Secondary PFs are set to "silent" mode, meaning they are disconnected from the network.

In Rx, the steering tables belong to the primary PF only, and it is its role to distribute incoming
traffic to other PFs, via cross-vhca steering capabilities. Still maintain a single default RSS table,
that is capable of pointing to the receive queues of a different PF.

In Tx, the primary PF creates a new Tx flow table, which is aliased by the secondaries, so they can
go out to the network through it.

In addition, we set default XPS configuration that, based on the CPU, selects an SQ belonging to the
PF on the same node as the CPU.

XPS default config example:

NUMA node(s):          2
NUMA node0 CPU(s):     0-11
NUMA node1 CPU(s):     12-23

PF0 on node0, PF1 on node1.

- /sys/class/net/eth2/queues/tx-0/xps_cpus:000001
- /sys/class/net/eth2/queues/tx-1/xps_cpus:001000
- /sys/class/net/eth2/queues/tx-2/xps_cpus:000002
- /sys/class/net/eth2/queues/tx-3/xps_cpus:002000
- /sys/class/net/eth2/queues/tx-4/xps_cpus:000004
- /sys/class/net/eth2/queues/tx-5/xps_cpus:004000
- /sys/class/net/eth2/queues/tx-6/xps_cpus:000008
- /sys/class/net/eth2/queues/tx-7/xps_cpus:008000
- /sys/class/net/eth2/queues/tx-8/xps_cpus:000010
- /sys/class/net/eth2/queues/tx-9/xps_cpus:010000
- /sys/class/net/eth2/queues/tx-10/xps_cpus:000020
- /sys/class/net/eth2/queues/tx-11/xps_cpus:020000
- /sys/class/net/eth2/queues/tx-12/xps_cpus:000040
- /sys/class/net/eth2/queues/tx-13/xps_cpus:040000
- /sys/class/net/eth2/queues/tx-14/xps_cpus:000080
- /sys/class/net/eth2/queues/tx-15/xps_cpus:080000
- /sys/class/net/eth2/queues/tx-16/xps_cpus:000100
- /sys/class/net/eth2/queues/tx-17/xps_cpus:100000
- /sys/class/net/eth2/queues/tx-18/xps_cpus:000200
- /sys/class/net/eth2/queues/tx-19/xps_cpus:200000
- /sys/class/net/eth2/queues/tx-20/xps_cpus:000400
- /sys/class/net/eth2/queues/tx-21/xps_cpus:400000
- /sys/class/net/eth2/queues/tx-22/xps_cpus:000800
- /sys/class/net/eth2/queues/tx-23/xps_cpus:800000

Stateful offload와의 상호 배제

Channel마다 서로 다른 PF를 사용하는 Multi-PF 특성은 state를 특정 PF에 보관하는 stateful feature와 충돌합니다. 예를 들어 TLS device offload는 connection별 context object를 PF에 만들기 때문에 다른 RQ나 SQ로 이동하면 동작이 깨집니다. 현재는 이 조합을 비활성화합니다.

Mutually exclusive features
===========================

The nature of Multi-PF, where different channels work with different PFs, conflicts with
stateful features where the state is maintained in one of the PFs.
For example, in the TLS device-offload feature, special context objects are created per connection
and maintained in the PF.  Transitioning between different RQs/SQs would break the feature. Hence,
we disable this combination for now.