DPAA2 Switch driver

DPAA2 Switch 드라이버와 포트 토폴로지

이 문서는 `GPL-2.0` 라이선스를 따르며 저작권은 © 2021 NXP에 있습니다.

DPAA2 Switch 드라이버

DPAA2 Switch 드라이버는 Datapath Switch(`DPSW`) 객체를 probe합니다. DPSW는 `LS2088A`, `LX2160A`와 그 변형 DPAA2 SoC에서 만들 수 있습니다.

드라이버는 switch device driver model을 사용하고 각 switch port를 네트워크 인터페이스로 노출합니다. 포트는 bridge에 포함하거나 독립 인터페이스로 사용할 수 있습니다.

포트 사이에서 switching되는 트래픽은 하드웨어로 offload됩니다.

DPSW 포트는 DPNI에 연결할 수도 있고 외부 접근을 위해 DPMAC에 연결할 수도 있습니다.

.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>

===================
DPAA2 Switch driver
===================

:Copyright: |copy| 2021 NXP

The DPAA2 Switch driver probes on the Datapath Switch (DPSW) object which can
be instantiated on the following DPAA2 SoCs and their variants: LS2088A and
LX2160A.

The driver uses the switch device driver model and exposes each switch port as
a network interface, which can be included in a bridge or used as a standalone
interface. Traffic switched between ports is offloaded into the hardware.

The DPSW can have ports connected to DPNIs or to DPMACs for external access.
::

         [ethA]     [ethB]      [ethC]     [ethD]     [ethE]     [ethF]
            :          :          :          :          :          :
            :          :          :          :          :          :
       [dpaa2-eth]  [dpaa2-eth]  [              dpaa2-switch              ]
            :          :          :          :          :          :        kernel
       =============================================================================
            :          :          :          :          :          :        hardware
         [DPNI]      [DPNI]     [============= DPSW =================]
            |          |          |          |          |          |
            |           ----------           |       [DPMAC]    [DPMAC]
             -------------------------------            |          |
                                                        |          |
                                                      [PHY]      [PHY]

Ethernet switch 생성 조건과 필요한 객체

Ethernet switch 생성

`dpaa2-switch` 드라이버는 `fsl-mc` bus에서 발견한 DPSW 장치를 probe합니다.

DPSW 장치는 boot-time 구성 파일인 DataPath Layout(`DPL`)로 정적으로 만들거나, 이미 `restool` userspace 도구에 포함된 DPAA2 object API로 실행 중 동적으로 만들 수 있습니다.

현재 `dpaa2-switch` 드라이버는 probe할 DPSW 객체에 다음 제약을 둡니다.

FDB 수

최소 FDB 수는 switch interface 수 이상이어야 합니다. bridge 아래에 있지 않을 때 각 switch port가 자체 FDB를 가져야 포트를 서로 분리할 수 있기 때문입니다.

fsl_dpaa2_switch dpsw.0: The number of FDBs is lower than the number of ports, cannot probe

broadcast와 flooding domain

broadcast와 flooding 구성이 모두 FDB별로 이루어져야 합니다. 그래야 같은 bridge 아래에 있어 FDB를 공유하는 switch port에 맞춰 각 FDB의 broadcast와 flooding domain을 제한할 수 있습니다.

fsl_dpaa2_switch dpsw.0: Flooding domain is not per FDB, cannot probe
fsl_dpaa2_switch dpsw.0: Broadcast domain is not per FDB, cannot probe

control interface

switch control interface가 비활성화되어서는 안 됩니다. 즉 생성 옵션으로 `DPSW_OPT_CTRL_IF_DIS`를 전달하면 안 됩니다.

control interface가 없으면 드라이버는 switch port netdevice에 올바른 Rx/Tx 트래픽 지원을 제공할 수 없습니다.

fsl_dpaa2_switch dpsw.0: Control Interface is disabled, cannot probe

실제 DPSW 객체 구성 외에도 `dpaa2-switch` 드라이버에는 다음 DPAA2 객체가 필요합니다.

• DPMCP 1개: MC firmware와 상호 작용하기 위한 Management Command Portal
• DPBP 1개: control interface의 Rx 경로용 버퍼를 채우는 Buffer Pool
• DPIO 최소 1개에 대한 접근: control interface 큐에서 enqueue/dequeue 작업을 수행하는 Software Portal. 전용 객체가 아니라 공유 객체를 사용합니다.

Creating an Ethernet Switch
===========================

The dpaa2-switch driver probes on DPSW devices found on the fsl-mc bus. These
devices can be either created statically through the boot time configuration
file - DataPath Layout (DPL) - or at runtime using the DPAA2 object APIs
(incorporated already into the restool userspace tool).

At the moment, the dpaa2-switch driver imposes the following restrictions on
the DPSW object that it will probe:

 * The minimum number of FDBs should be at least equal to the number of switch
   interfaces. This is necessary so that separation of switch ports can be
   done, ie when not under a bridge, each switch port will have its own FDB.
   ::

        fsl_dpaa2_switch dpsw.0: The number of FDBs is lower than the number of ports, cannot probe

 * Both the broadcast and flooding configuration should be per FDB. This
   enables the driver to restrict the broadcast and flooding domains of each
   FDB depending on the switch ports that are sharing it (aka are under the
   same bridge).
   ::

        fsl_dpaa2_switch dpsw.0: Flooding domain is not per FDB, cannot probe
        fsl_dpaa2_switch dpsw.0: Broadcast domain is not per FDB, cannot probe

 * The control interface of the switch should not be disabled
   (DPSW_OPT_CTRL_IF_DIS not passed as a create time option). Without the
   control interface, the driver is not capable to provide proper Rx/Tx traffic
   support on the switch port netdevices.
   ::

        fsl_dpaa2_switch dpsw.0: Control Interface is disabled, cannot probe

Besides the configuration of the actual DPSW object, the dpaa2-switch driver
will need the following DPAA2 objects:

 * 1 DPMCP - A Management Command Portal object is needed for any interaction
   with the MC firmware.

 * 1 DPBP - A Buffer Pool is used for seeding buffers intended for the Rx path
   on the control interface.

 * Access to at least one DPIO object (Software Portal) is needed for any
   enqueue/dequeue operation to be performed on the control interface queues.
   The DPIO object will be shared, no need for a private one.

L2 switching, VLAN·STP·FDB와 flooding

Switching 기능

드라이버는 port bridging과 각 switch interface의 독립 사용을 위한 L2 forwarding rule을 하드웨어에 구성할 수 있습니다.

하드웨어의 VLAN awareness는 구성할 수 없습니다. 따라서 DPAA2 switch port는 VLAN-aware bridge를 사용하는 사례에서만 사용해야 합니다.

ip link add dev br0 type bridge vlan_filtering 1

VLAN-unaware bridge에 연결하려 하면 실패합니다.

ip link add dev br1 type bridge
ip link set dev ethX master br1
Error: fsl_dpaa2_switch: Cannot join a VLAN-unaware bridge

bridge 생성 시 `stp_state 1`을 사용하면 STP를 통한 topology와 loop detection을 지원합니다.

ip link add dev br0 type bridge vlan_filtering 1 stp_state 1

L2 FDB의 add, delete, dump를 지원합니다.

각 switch port의 하드웨어 FDB learning은 bridge 명령으로 독립 구성할 수 있습니다. hardware learning을 끄면 fast age 절차를 실행해 이전에 학습한 주소를 모두 제거합니다.

bridge link set dev ethX learning off
bridge link set dev ethX learning on

unknown unicast와 multicast flooding domain을 제한할 수 있지만 둘을 서로 독립적으로 구성할 수는 없습니다.

ip link set dev ethX type bridge_slave flood off mcast_flood off
ip link set dev ethX type bridge_slave flood off mcast_flood on
Error: fsl_dpaa2_switch: Cannot configure multicast flooding independently of unicast.

switch port의 broadcast flooding은 `brport` sysfs를 통해 끄고 켤 수 있습니다.

echo 0 > /sys/bus/fsl-mc/devices/dpsw.Y/net/ethX/brport/broadcast_flood

Switching features
==================

The driver supports the configuration of L2 forwarding rules in hardware for
port bridging as well as standalone usage of the independent switch interfaces.

The hardware is not configurable with respect to VLAN awareness, thus any DPAA2
switch port should be used only in usecases with a VLAN aware bridge::

        $ ip link add dev br0 type bridge vlan_filtering 1

        $ ip link add dev br1 type bridge
        $ ip link set dev ethX master br1
        Error: fsl_dpaa2_switch: Cannot join a VLAN-unaware bridge

Topology and loop detection through STP is supported when ``stp_state 1`` is
used at bridge create ::

        $ ip link add dev br0 type bridge vlan_filtering 1 stp_state 1

L2 FDB manipulation (add/delete/dump) is supported.

HW FDB learning can be configured on each switch port independently through
bridge commands. When the HW learning is disabled, a fast age procedure will be
run and any previously learnt addresses will be removed.
::

        $ bridge link set dev ethX learning off
        $ bridge link set dev ethX learning on

Restricting the unknown unicast and multicast flooding domain is supported, but
not independently of each other::

        $ ip link set dev ethX type bridge_slave flood off mcast_flood off
        $ ip link set dev ethX type bridge_slave flood off mcast_flood on
        Error: fsl_dpaa2_switch: Cannot configure multicast flooding independently of unicast.

Broadcast flooding on a switch port can be disabled/enabled through the brport sysfs::

        $ echo 0 > /sys/bus/fsl-mc/devices/dpsw.Y/net/ethX/brport/broadcast_flood

ACL 기반 redirect·trap·drop offload

Offload

Routing action: redirect, trap, drop

DPAA2 switch는 ACL table을 사용해 flow 기반 packet redirection을 offload할 수 있습니다. 여러 port가 ACL table 하나를 공유하는 shared filter block도 지원합니다.

지원 flow key

`matchall` filter를 사용하면 port에서 받은 모든 트래픽을 redirect할 수 있습니다.

지원 flow action

• `drop`
• `mirred egress redirect`
• `trap`

ACL entry, 즉 filter 하나에는 위 action 중 하나만 설정할 수 있습니다.

예제 1: `eth4`에서 받은 frame 중 source address가 `00:01:02:03:04:05`인 frame을 CPU로 보냅니다.

tc qdisc add dev eth4 clsact
tc filter add dev eth4 ingress flower src_mac 00:01:02:03:04:05 skip_sw action trap

예제 2: `eth4`에서 받은 frame 중 VID 100이고 PCP가 3인 frame을 drop합니다.

tc filter add dev eth4 ingress protocol 802.1q flower skip_sw vlan_id 100 vlan_prio 3 action drop

예제 3: `eth4`에서 받은 모든 frame을 `eth1`로 redirect합니다.

tc filter add dev eth4 ingress matchall action mirred egress redirect dev eth1

예제 4: `eth5`와 `eth6`에서 shared filter block 하나를 사용합니다.

tc qdisc add dev eth5 ingress_block 1 clsact
tc qdisc add dev eth6 ingress_block 1 clsact
tc filter add block 1 ingress flower dst_mac 00:01:02:03:04:04 skip_sw \
  action trap
tc filter add block 1 ingress protocol ipv4 flower src_ip 192.168.1.1 skip_sw \
  action mirred egress redirect dev eth3

Offloads
========

Routing actions (redirect, trap, drop)
--------------------------------------

The DPAA2 switch is able to offload flow-based redirection of packets making
use of ACL tables. Shared filter blocks are supported by sharing a single ACL
table between multiple ports.

The following flow keys are supported:

 * Ethernet: dst_mac/src_mac
 * IPv4: dst_ip/src_ip/ip_proto/tos
 * VLAN: vlan_id/vlan_prio/vlan_tpid/vlan_dei
 * L4: dst_port/src_port

Also, the matchall filter can be used to redirect the entire traffic received
on a port.

As per flow actions, the following are supported:

 * drop
 * mirred egress redirect
 * trap

Each ACL entry (filter) can be setup with only one of the listed
actions.

Example 1: send frames received on eth4 with a SA of 00:01:02:03:04:05 to the
CPU::

        $ tc qdisc add dev eth4 clsact
        $ tc filter add dev eth4 ingress flower src_mac 00:01:02:03:04:05 skip_sw action trap

Example 2: drop frames received on eth4 with VID 100 and PCP of 3::

        $ tc filter add dev eth4 ingress protocol 802.1q flower skip_sw vlan_id 100 vlan_prio 3 action drop

Example 3: redirect all frames received on eth4 to eth1::

        $ tc filter add dev eth4 ingress matchall action mirred egress redirect dev eth1

Example 4: Use a single shared filter block on both eth5 and eth6::

        $ tc qdisc add dev eth5 ingress_block 1 clsact
        $ tc qdisc add dev eth6 ingress_block 1 clsact
        $ tc filter add block 1 ingress flower dst_mac 00:01:02:03:04:04 skip_sw \
                action trap
        $ tc filter add block 1 ingress protocol ipv4 flower src_ip 192.168.1.1 skip_sw \
                action mirred egress redirect dev eth3

port·VLAN mirroring 제약

Mirroring

DPAA2 switch는 port별 mirroring과 VLAN별 mirroring만 지원합니다. shared block에 mirroring filter를 추가하는 것도 지원합니다.

`802.1q` protocol과 함께 `tc-flower` classifier를 사용할 때는 `vlan_id` key만 허용합니다. 다른 802.1q field를 기준으로 mirroring하려 하면 거부됩니다.

tc qdisc add dev eth8 ingress_block 1 clsact
tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_prio 3 action mirred egress mirror dev eth6
Error: fsl_dpaa2_switch: Only matching on VLAN ID supported.
We have an error talking to the kernel

port에 mirroring VLAN filter를 요청하려면 해당 switch port에 VLAN이 설치되어 있어야 합니다. `bridge`로 설치하거나, switch port를 독립 인터페이스로 사용할 때 VLAN upper device를 만들 수 있습니다.

tc qdisc add dev eth8 ingress_block 1 clsact
tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 200 action mirred egress mirror dev eth6
Error: VLAN must be installed on the switch port.
We have an error talking to the kernel

bridge로 VLAN 200을 설치한 뒤 filter를 추가하는 예입니다.

bridge vlan add vid 200 dev eth8
tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 200 action mirred egress mirror dev eth6

VLAN upper device를 만든 뒤 filter를 추가하는 예입니다.

ip link add link eth8 name eth8.200 type vlan id 200
tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 200 action mirred egress mirror dev eth6

mirrored 트래픽도 다른 트래픽과 같은 egress 제한을 받습니다. packet의 VLAN이 mirror port에 설치되어 있지 않으면 port에 도달했을 때 drop됩니다.

DPAA2 switch는 mirroring destination을 하나만 지원합니다. 여러 mirror rule을 설치할 수 있지만 모두 같은 `to` port를 사용해야 합니다.

tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 200 action mirred egress mirror dev eth6
tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 100 action mirred egress mirror dev eth7
Error: fsl_dpaa2_switch: Multiple mirror ports not supported.
We have an error talking to the kernel

Mirroring
~~~~~~~~~

The DPAA2 switch supports only per port mirroring and per VLAN mirroring.
Adding mirroring filters in shared blocks is also supported.

When using the tc-flower classifier with the 802.1q protocol, only the
''vlan_id'' key will be accepted. Mirroring based on any other fields from the
802.1q protocol will be rejected::

        $ tc qdisc add dev eth8 ingress_block 1 clsact
        $ tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_prio 3 action mirred egress mirror dev eth6
        Error: fsl_dpaa2_switch: Only matching on VLAN ID supported.
        We have an error talking to the kernel

If a mirroring VLAN filter is requested on a port, the VLAN must to be
installed on the switch port in question either using ''bridge'' or by creating
a VLAN upper device if the switch port is used as a standalone interface::

        $ tc qdisc add dev eth8 ingress_block 1 clsact
        $ tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 200 action mirred egress mirror dev eth6
        Error: VLAN must be installed on the switch port.
        We have an error talking to the kernel

        $ bridge vlan add vid 200 dev eth8
        $ tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 200 action mirred egress mirror dev eth6

        $ ip link add link eth8 name eth8.200 type vlan id 200
        $ tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 200 action mirred egress mirror dev eth6

Also, it should be noted that the mirrored traffic will be subject to the same
egress restrictions as any other traffic. This means that when a mirrored
packet will reach the mirror port, if the VLAN found in the packet is not
installed on the port it will get dropped.

The DPAA2 switch supports only a single mirroring destination, thus multiple
mirror rules can be installed but their ''to'' port has to be the same::

        $ tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 200 action mirred egress mirror dev eth6
        $ tc filter add block 1 ingress protocol 802.1q flower skip_sw vlan_id 100 action mirred egress mirror dev eth7
        Error: fsl_dpaa2_switch: Multiple mirror ports not supported.
        We have an error talking to the kernel