XFRM device - offloading the IPsec computations

문서 정보

이 문서는 Shannon Nelson과 Leon Romanovsky가 작성한 XFRM device IPsec computation offload 설명서입니다.

.. SPDX-License-Identifier: GPL-2.0
.. _xfrm_device:

===============================================
XFRM device - offloading the IPsec computations
===============================================

Shannon Nelson <[email protected]>
Leon Romanovsky <[email protected]>


Overview

IPsec hardware offload 유형

IPsec는 network traffic 보호에 유용하지만 계산 비용이 큽니다. Traffic과 link 구성에 따라 10Gbps link가 1Gbps 미만으로 떨어질 수 있습니다. Hardware IPsec offload NIC는 throughput을 크게 높이고 CPU 사용량을 줄이며 XFRM Device interface가 driver의 offload를 stack에 공개합니다.

Kernel은 두 종류의 hardware offload를 지원합니다. IPsec crypto offload에서는 NIC가 encrypt/decrypt만 하고 kernel이 나머지를 수행합니다. IPsec packet offload에서는 NIC가 encrypt/decrypt와 encapsulation을 수행하고 kernel과 NIC가 SA·policy를 동기화하며 NIC가 그 state를 처리하고 kernel은 key manager와 통신합니다.

Userspace는 보통 libreswan이나 KAME/raccoon을 통해 offload에 접근하고 실험에는 iproute2의 `ip xfrm` command를 사용할 수 있습니다.

Crypto offload 예제는 ESP transport SA에 AES-GCM, selector, `offload dev eth4 dir in`을 설정합니다. Packet offload 예제는 `offload packet`을 SA와 policy에 지정하고 template으로 ESP reqid와 transport mode를 연결합니다. 복잡한 명령은 shell script나 libreswan이 대신 구성할 수 있습니다.

========

IPsec is a useful feature for securing network traffic, but the
computational cost is high: a 10Gbps link can easily be brought down
to under 1Gbps, depending on the traffic and link configuration.
Luckily, there are NICs that offer a hardware based IPsec offload which
can radically increase throughput and decrease CPU utilization.  The XFRM
Device interface allows NIC drivers to offer to the stack access to the
hardware offload.

Right now, there are two types of hardware offload that kernel supports.
 * IPsec crypto offload:
   * NIC performs encrypt/decrypt
   * Kernel does everything else
 * IPsec packet offload:
   * NIC performs encrypt/decrypt
   * NIC does encapsulation
   * Kernel and NIC have SA and policy in-sync
   * NIC handles the SA and policies states
   * The Kernel talks to the keymanager

Userland access to the offload is typically through a system such as
libreswan or KAME/raccoon, but the iproute2 'ip xfrm' command set can
be handy when experimenting.  An example command might look something
like this for crypto offload:

  ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \
     reqid 0x07 replay-window 32 \
     aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \
     sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp \
     offload dev eth4 dir in

and for packet offload

  ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \
     reqid 0x07 replay-window 32 \
     aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \
     sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp \
     offload packet dev eth4 dir in

  ip x p add src 14.0.0.70 dst 14.0.0.52 offload packet dev eth4 dir in
  tmpl src 14.0.0.70 dst 14.0.0.52 proto esp reqid 10000 mode transport

Yes, that's ugly, but that's what shell scripts and/or libreswan are for.

xfrmdev_ops callback

`include/linux/netdevice.h`의 `struct xfrmdev_ops`가 driver callback을 정의합니다.

Crypto와 packet offload 공통 callback은 `xdo_dev_state_add`, `xdo_dev_state_delete`, `xdo_dev_state_free`, `xdo_dev_offload_ok`, `xdo_dev_state_advance_esn`, `xdo_dev_state_update_stats`입니다.

Packet offload 전용 callback은 `xdo_dev_policy_add`, `xdo_dev_policy_delete`, `xdo_dev_policy_free`입니다.

IPsec offload NIC driver는 지원 mode에 필요한 callback을 구현해 XFRM subsystem에 기능을 제공합니다. `NETIF_F_HW_ESP`와 `NETIF_F_HW_ESP_TX_CSUM` feature bit가 offload availability를 알립니다.

Callbacks to implement
======================

::

  /* from include/linux/netdevice.h */
  struct xfrmdev_ops {
        /* Crypto and Packet offload callbacks */
        int        (*xdo_dev_state_add)(struct net_device *dev,
                                     struct xfrm_state *x,
                                     struct netlink_ext_ack *extack);
        void        (*xdo_dev_state_delete)(struct net_device *dev,
                                        struct xfrm_state *x);
        void        (*xdo_dev_state_free)(struct net_device *dev,
                                      struct xfrm_state *x);
        bool        (*xdo_dev_offload_ok) (struct sk_buff *skb,
                                       struct xfrm_state *x);
        void    (*xdo_dev_state_advance_esn) (struct xfrm_state *x);
        void    (*xdo_dev_state_update_stats) (struct xfrm_state *x);

        /* Solely packet offload callbacks */
        int        (*xdo_dev_policy_add) (struct xfrm_policy *x, struct netlink_ext_ack *extack);
        void        (*xdo_dev_policy_delete) (struct xfrm_policy *x);
        void        (*xdo_dev_policy_free) (struct xfrm_policy *x);
  };

The NIC driver offering ipsec offload will need to implement callbacks
relevant to supported offload to make the offload available to the network
stack's XFRM subsystem. Additionally, the feature bits NETIF_F_HW_ESP and
NETIF_F_HW_ESP_TX_CSUM will signal the availability of the offload.

Probe와 SA offload 설치

Probe 시점에 `register_netdev()`를 호출하기 전 driver는 local data structure와 XFRM callback을 설정하고 feature bit를 켭니다. XFRM listener가 `NETDEV_REGISTER` event에서 setup을 마칩니다.

예제는 netdev의 `xfrmdev_ops`를 `ixgbe_xfrmdev_ops`로 지정하고 `features`와 `hw_enc_features`에 `NETIF_F_HW_ESP`를 추가합니다.

새 SA가 offload를 요청하면 `xdo_dev_state_add()`가 SA와 RX/TX 방향을 받습니다. Driver는 algorithm 지원 여부를 검사하고 key, salt, target IP, protocol 등의 SA 정보를 저장하고 hardware SA offload를 활성화한 뒤 status를 반환합니다.

Driver는 SA의 `xs->xso.offload_handle`에 opaque context pointer를 저장해 fast path offload request로 전달할 수 있습니다.

Flow
====

At probe time and before the call to register_netdev(), the driver should
set up local data structures and XFRM callbacks, and set the feature bits.
The XFRM code's listener will finish the setup on NETDEV_REGISTER.

::

                adapter->netdev->xfrmdev_ops = &ixgbe_xfrmdev_ops;
                adapter->netdev->features |= NETIF_F_HW_ESP;
                adapter->netdev->hw_enc_features |= NETIF_F_HW_ESP;

When new SAs are set up with a request for "offload" feature, the
driver's xdo_dev_state_add() will be given the new SA to be offloaded
and an indication of whether it is for Rx or Tx.  The driver should

        - verify the algorithm is supported for offloads
        - store the SA information (key, salt, target-ip, protocol, etc)
        - enable the HW offload of the SA
        - return status value:

                ===========   ===================================
                0             success
                -EOPNETSUPP   offload not supported, try SW IPsec,
                              not applicable for packet offload mode
                other         fail the request
                ===========   ===================================

The driver can also set an offload_handle in the SA, an opaque void pointer
that can be used to convey context into the fast-path offload requests::

                xs->xso.offload_handle = context;

Crypto offload TX/RX flow

Network stack이 offloaded SA의 IPsec packet을 준비할 때 먼저 `xdo_dev_offload_ok(skb, x)`로 driver에 packet 처리 가능 여부를 묻습니다. Driver는 IPv4/IPv6, IPv4 option 부재 같은 packet 조건을 확인해 true/false를 반환합니다. Callback이 없으면 stack의 합리적인 default를 사용합니다.

Crypto offload TX에서 driver는 packet의 offload state와 opaque context를 읽어 hardware send를 설정합니다. Stack이 IPsec header를 이미 삽입했으므로 NIC는 encryption과 header value 보정만 수행합니다.

RX에서 hardware가 decryption offload 성공을 알리면 driver는 사용한 decoded SA reference를 `skb`에 추가합니다. 이때 payload는 복호화됐지만 IPsec header는 남아 있고 stack 상위의 `xfrm_input()`이 나중에 제거합니다.

Packet header에서 SPI, protocol, destination IP를 읽어 SA를 찾고 `xfrm_state_hold(xs)`로 reference를 잡습니다. `secpath_set(skb)`로 security path를 만든 뒤 `xvec`에 SA를 추가하고 length와 original length를 증가시킵니다.

`xfrm_offload(skb)`의 flag를 `CRYPTO_DONE`, status를 hardware crypto result로 설정하고 평소처럼 packet을 `napi_gro_receive()`에 전달합니다.

When the network stack is preparing an IPsec packet for an SA that has
been setup for offload, it first calls into xdo_dev_offload_ok() with
the skb and the intended offload state to ask the driver if the offload
will serviceable.  This can check the packet information to be sure the
offload can be supported (e.g. IPv4 or IPv6, no IPv4 options, etc) and
return true or false to signify its support. In case driver doesn't implement
this callback, the stack provides reasonable defaults.

Crypto offload mode:
When ready to send, the driver needs to inspect the Tx packet for the
offload information, including the opaque context, and set up the packet
send accordingly::

                xs = xfrm_input_state(skb);
                context = xs->xso.offload_handle;
                set up HW for send

The stack has already inserted the appropriate IPsec headers in the
packet data, the offload just needs to do the encryption and fix up the
header values.


When a packet is received and the HW has indicated that it offloaded a
decryption, the driver needs to add a reference to the decoded SA into
the packet's skb.  At this point the data should be decrypted but the
IPsec headers are still in the packet data; they are removed later up
the stack in xfrm_input().

        find and hold the SA that was used to the Rx skb::

                get spi, protocol, and destination IP from packet headers
                xs = find xs from (spi, protocol, dest_IP)
                xfrm_state_hold(xs);

        store the state information into the skb::

                sp = secpath_set(skb);
                if (!sp) return;
                sp->xvec[sp->len++] = xs;
                sp->olen++;

        indicate the success and/or error status of the offload::

                xo = xfrm_offload(skb);
                xo->flags = CRYPTO_DONE;
                xo->status = crypto_status;

        hand the packet to napi_gro_receive() as usual

In ESN mode, xdo_dev_state_advance_esn() is called from

ESN, packet offload와 resource lifecycle

ESN mode에서 RX는 `xfrm_replay_advance_esn()`, TX는 `xfrm_replay_overflow_offload_esn()`에서 `xdo_dev_state_advance_esn()`을 호출합니다. Driver는 packet sequence number를 검사해 필요하면 hardware ESN state machine을 갱신합니다.

Packet offload mode에서는 hardware가 XFRM header를 추가하고 삭제합니다. RX에서 hardware가 성공을 보고하면 XFRM stack을 우회하고, TX packet은 추가 header나 encryption 없이 kernel을 떠나며 hardware가 두 작업을 책임집니다.

User가 SA를 제거하면 `xdo_dev_state_delete()`와 `xdo_dev_policy_delete()`가 offload를 disable합니다. 모든 state·policy reference가 사라진 뒤 garbage collection에서 `xdo_dev_state_free()`와 `xdo_dev_policy_free()`를 호출해 남은 resource를 정리합니다. 구체적인 사용법은 hardware 요구에 달려 있습니다.

Netdev가 DOWN이 되면 XFRM listener가 남은 offloaded state에 state/policy delete와 free callback을 모두 호출합니다.

Hardware가 packet을 처리하면 XFRM core는 hard·soft limit를 정확히 셀 수 없습니다. Hardware와 driver가 이를 계산하고 `xdo_dev_state_update_stats()` 호출 때 정확한 값을 제공해야 합니다. Limit에 도달하면 driver가 `xfrm_state_check_expire()`를 호출해 XFRM rekey sequence를 시작하게 해야 합니다.

xfrm_replay_advance_esn() for RX, and xfrm_replay_overflow_offload_esn for TX.
Driver will check packet seq number and update HW ESN state machine if needed.

Packet offload mode:
HW adds and deletes XFRM headers. So in RX path, XFRM stack is bypassed if HW
reported success. In TX path, the packet lefts kernel without extra header
and not encrypted, the HW is responsible to perform it.

When the SA is removed by the user, the driver's xdo_dev_state_delete()
and xdo_dev_policy_delete() are asked to disable the offload.  Later,
xdo_dev_state_free() and xdo_dev_policy_free() are called from a garbage
collection routine after all reference counts to the state and policy
have been removed and any remaining resources can be cleared for the
offload state.  How these are used by the driver will depend on specific
hardware needs.

As a netdev is set to DOWN the XFRM stack's netdev listener will call
xdo_dev_state_delete(), xdo_dev_policy_delete(), xdo_dev_state_free() and
xdo_dev_policy_free() on any remaining offloaded states.

Outcome of HW handling packets, the XFRM core can't count hard, soft limits.
The HW/driver are responsible to perform it and provide accurate data when
xdo_dev_state_update_stats() is called. In case of one of these limits
occuried, the driver needs to call to xfrm_state_check_expire() to make sure
that XFRM performs rekeying sequence.