L2TP

개요와 control/data plane 분리

Layer 2 Tunneling Protocol(L2TP)은 IP 네트워크를 통해 L2 프레임을 터널링합니다. 이 문서는 애플리케이션 개발자를 위한 Linux 커널 L2TP API와 커널 개발자·유지관리자에게 필요한 내부 구현을 함께 설명합니다.

커널 L2TP 하위 시스템은 L2TPv2와 L2TPv3의 datapath를 구현합니다. L2TPv2는 항상 UDP를 사용하고, L2TPv3는 UDP 또는 IP protocol 115를 직접 사용할 수 있습니다. RFC가 정의하는 control packet은 사용자 공간이 처리하고 커널은 data packet만 처리합니다.

한 tunnel은 하나 이상의 session을 운반하고 하나의 socket과 연결됩니다. session은 `pppN`, `l2tpethN` 같은 가상 netdevice와 연결되어 L2 frame이 드나듭니다. Linux API로 tunnel과 session을 만든다는 것은 data path context를 설치한다는 뜻이며 control protocol의 모든 상태와 메시지 교환은 사용자 공간 책임입니다.

설정 순서는 tunnel socket 생성과 peer control 교환, 그 결과로 kernel tunnel context 생성, session control 교환, 그 결과로 kernel session context 생성입니다.

.. SPDX-License-Identifier: GPL-2.0

====
L2TP
====

Layer 2 Tunneling Protocol (L2TP) allows L2 frames to be tunneled over
an IP network.

This document covers the kernel's L2TP subsystem. It documents kernel
APIs for application developers who want to use the L2TP subsystem and
it provides some technical details about the internal implementation
which may be useful to kernel developers and maintainers.

Overview
========

The kernel's L2TP subsystem implements the datapath for L2TPv2 and
L2TPv3. L2TPv2 is carried over UDP. L2TPv3 is carried over UDP or
directly over IP (protocol 115).

The L2TP RFCs define two basic kinds of L2TP packets: control packets
(the "control plane"), and data packets (the "data plane"). The kernel
deals only with data packets. The more complex control packets are
handled by user space.

An L2TP tunnel carries one or more L2TP sessions. Each tunnel is
associated with a socket. Each session is associated with a virtual
netdevice, e.g. ``pppN``, ``l2tpethN``, through which data frames pass
to/from L2TP. Fields in the L2TP header identify the tunnel or session
and whether it is a control or data packet. When tunnels and sessions
are set up using the Linux kernel API, we're just setting up the L2TP
data path. All aspects of the control protocol are to be handled by
user space.

This split in responsibilities leads to a natural sequence of
operations when establishing tunnels and sessions. The procedure looks
like this:

    1) Create a tunnel socket. Exchange L2TP control protocol messages
       with the peer over that socket in order to establish a tunnel.

    2) Create a tunnel context in the kernel, using information
       obtained from the peer using the control protocol messages.

    3) Exchange L2TP control protocol messages with the peer over the
       tunnel socket in order to establish a session.

    4) Create a session context in the kernel using information
       obtained from the peer using the control protocol messages.

Tunnel socket API

L2TPv2 tunnel은 UDP를 사용하고 L2TPv3 tunnel은 UDP 또는 IP encapsulation을 선택할 수 있습니다. IPv4 UDP 예는 `socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`, IPv6 L2TP/IP 예는 `socket(AF_INET6, SOCK_DGRAM, IPPROTO_L2TP)`입니다.

`IPPROTO_L2TP`는 커널 L2TP 하위 시스템이 구현하는 IP protocol type입니다. `include/uapi/linux/l2tp.h`의 `sockaddr_l2tpip`과 `sockaddr_l2tpip6`에는 tunnel(connection) ID가 포함됩니다. L2TPv3 애플리케이션은 로컬 tunnel ID로 socket을 bind하고 peer tunnel ID와 IP 주소를 알게 되면 connect해야 합니다.

L2TP/IP tunnel 설정 요청을 수신하려면 요청 전용 L2TPIP socket을 열고 tunnel ID 0으로 bind합니다. 설정 요청이 ID 0으로 전달되기 때문입니다. tunnel socket을 닫으면 해당 tunnel과 모든 session이 자동으로 닫힙니다.

L2TP APIs
=========

This section documents each userspace API of the L2TP subsystem.

Tunnel Sockets
--------------

L2TPv2 always uses UDP. L2TPv3 may use UDP or IP encapsulation.

To create a tunnel socket for use by L2TP, the standard POSIX
socket API is used.

For example, for a tunnel using IPv4 addresses and UDP encapsulation::

    int sockfd = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);

Or for a tunnel using IPv6 addresses and IP encapsulation::

    int sockfd = socket(AF_INET6, SOCK_DGRAM, IPPROTO_L2TP);

UDP socket programming doesn't need to be covered here.

IPPROTO_L2TP is an IP protocol type implemented by the kernel's L2TP
subsystem. The L2TPIP socket address is defined in struct
sockaddr_l2tpip and struct sockaddr_l2tpip6 at
`include/uapi/linux/l2tp.h`_. The address includes the L2TP tunnel
(connection) id. To use L2TP IP encapsulation, an L2TPv3 application
should bind the L2TPIP socket using the locally assigned
tunnel id. When the peer's tunnel id and IP address is known, a
connect must be done.

If the L2TP application needs to handle L2TPv3 tunnel setup requests
from peers using L2TPIP, it must open a dedicated L2TPIP
socket to listen for those requests and bind the socket using tunnel
id 0 since tunnel setup requests are addressed to tunnel id 0.

An L2TP tunnel and all of its sessions are automatically closed when
its tunnel socket is closed.

Generic Netlink 관리 모델

애플리케이션은 `include/uapi/linux/l2tp.h`에 정의된 Generic Netlink API로 커널의 tunnel과 session instance를 관리합니다. tunnel과 session마다 Create, Delete, Modify, Get 명령이 있으며 예를 들어 tunnel 생성은 `L2TP_CMD_TUNNEL_CREATE`입니다.

각 instance는 로컬에서 고유한 32비트 ID로 식별됩니다. tunnel은 `L2TP_ATTR_CONN_ID`와 `L2TP_ATTR_PEER_CONN_ID`, session은 `L2TP_ATTR_SESSION_ID`와 `L2TP_ATTR_PEER_SESSION_ID`로 로컬·peer ID를 전달합니다. L2TPv2의 16비트 ID는 이 속성에서 32비트 값으로 확장됩니다.

tunnel 생성의 `L2TP_ATTR_FD`는 이미 bind와 connect를 마친 tunnel socket fd를 커널에 전달합니다. 생략하면 커널이 IP 주소와 UDP port 속성으로 socket을 직접 만들며, 이 방식은 iproute2의 `ip l2tp`가 구성하는 unmanaged L2TPv3 tunnel에 사용됩니다.

Netlink API
-----------

L2TP applications use netlink to manage L2TP tunnel and session
instances in the kernel. The L2TP netlink API is defined in
`include/uapi/linux/l2tp.h`_.

L2TP uses `Generic Netlink`_ (GENL). Several commands are defined:
Create, Delete, Modify and Get for tunnel and session
instances, e.g. ``L2TP_CMD_TUNNEL_CREATE``. The API header lists the
netlink attribute types that can be used with each command.

Tunnel and session instances are identified by a locally unique
32-bit id.  L2TP tunnel ids are given by ``L2TP_ATTR_CONN_ID`` and
``L2TP_ATTR_PEER_CONN_ID`` attributes and L2TP session ids are given
by ``L2TP_ATTR_SESSION_ID`` and ``L2TP_ATTR_PEER_SESSION_ID``
attributes. If netlink is used to manage L2TPv2 tunnel and session
instances, the L2TPv2 16-bit tunnel/session id is cast to a 32-bit
value in these attributes.

In the ``L2TP_CMD_TUNNEL_CREATE`` command, ``L2TP_ATTR_FD`` tells the
kernel the tunnel socket fd being used. If not specified, the kernel
creates a kernel socket for the tunnel, using IP parameters set in
``L2TP_ATTR_IP[6]_SADDR``, ``L2TP_ATTR_IP[6]_DADDR``,
``L2TP_ATTR_UDP_SPORT``, ``L2TP_ATTR_UDP_DPORT`` attributes. Kernel
sockets are used to implement unmanaged L2TPv3 tunnels (iproute2's "ip
l2tp" commands). If ``L2TP_ATTR_FD`` is given, it must be a socket fd
that is already bound and connected. There is more information about
unmanaged tunnels later in this document.

Tunnel·session 명령과 속성

`L2TP_CMD_TUNNEL_CREATE`에는 로컬·peer connection ID, protocol version 2 또는 3, UDP/IP encapsulation type이 필수입니다. FD는 선택 사항이며 FD가 없을 때만 IPv4/IPv6 source·destination, UDP source·destination port, IPv4 checksum, IPv6 zero-checksum 송수신 속성이 socket 생성에 사용됩니다. debug flag도 선택적으로 설정할 수 있습니다.

tunnel destroy와 modify에는 대상 `CONN_ID`가 필수이고 modify는 debug flag를 바꿀 수 있습니다. tunnel get은 단일 조회에 `CONN_ID`를 쓸 수 있지만 dump request에서는 무시됩니다.

`L2TP_CMD_SESSION_CREATE`에는 부모 tunnel ID, 로컬·peer session ID, pseudowire type이 필요합니다. 선택 속성으로 debug, 송수신 sequence number, LNS mode, reorder timeout을 설정하고 L2TPv3에서는 layer2-specific sublayer, local·peer cookie, interface name도 지정합니다. Ethernet session은 `l2tpeth` 가상 interface를 만들고 PPP session은 별도 PPPoL2TP socket을 열어 새 session과 연결해야 합니다.

session destroy·modify는 부모 `CONN_ID`와 `SESSION_ID`로 대상을 찾습니다. L2TPv3 Ethernet에서는 `IFNAME`을 주면 두 ID보다 우선합니다. modify는 debug, sequence, LNS mode, reorder timeout을 바꿀 수 있습니다. session get에서도 단일 조회에는 ID 또는 interface 이름을 사용할 수 있지만 dump request에서는 이 식별 속성을 무시합니다. 명령과 속성의 최종 정의는 `include/uapi/linux/l2tp.h`를 기준으로 해야 합니다.

``L2TP_CMD_TUNNEL_CREATE`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Sets the tunnel (connection) id.
PEER_CONN_ID       Y        Sets the peer tunnel (connection) id.
PROTO_VERSION      Y        Protocol version. 2 or 3.
ENCAP_TYPE         Y        Encapsulation type: UDP or IP.
FD                 N        Tunnel socket file descriptor.
UDP_CSUM           N        Enable IPv4 UDP checksums. Used only if FD is
                            not set.
UDP_ZERO_CSUM6_TX  N        Zero IPv6 UDP checksum on transmit. Used only
                            if FD is not set.
UDP_ZERO_CSUM6_RX  N        Zero IPv6 UDP checksum on receive. Used only if
                            FD is not set.
IP_SADDR           N        IPv4 source address. Used only if FD is not
                            set.
IP_DADDR           N        IPv4 destination address. Used only if FD is
                            not set.
UDP_SPORT          N        UDP source port. Used only if FD is not set.
UDP_DPORT          N        UDP destination port. Used only if FD is not
                            set.
IP6_SADDR          N        IPv6 source address. Used only if FD is not
                            set.
IP6_DADDR          N        IPv6 destination address. Used only if FD is
                            not set.
DEBUG              N        Debug flags.
================== ======== ===

``L2TP_CMD_TUNNEL_DESTROY`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Identifies the tunnel id to be destroyed.
================== ======== ===

``L2TP_CMD_TUNNEL_MODIFY`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Identifies the tunnel id to be modified.
DEBUG              N        Debug flags.
================== ======== ===

``L2TP_CMD_TUNNEL_GET`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            N        Identifies the tunnel id to be queried.
                            Ignored in DUMP requests.
================== ======== ===

``L2TP_CMD_SESSION_CREATE`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        The parent tunnel id.
SESSION_ID         Y        Sets the session id.
PEER_SESSION_ID    Y        Sets the parent session id.
PW_TYPE            Y        Sets the pseudowire type.
DEBUG              N        Debug flags.
RECV_SEQ           N        Enable rx data sequence numbers.
SEND_SEQ           N        Enable tx data sequence numbers.
LNS_MODE           N        Enable LNS mode (auto-enable data sequence
                            numbers).
RECV_TIMEOUT       N        Timeout to wait when reordering received
                            packets.
L2SPEC_TYPE        N        Sets layer2-specific-sublayer type (L2TPv3
                            only).
COOKIE             N        Sets optional cookie (L2TPv3 only).
PEER_COOKIE        N        Sets optional peer cookie (L2TPv3 only).
IFNAME             N        Sets interface name (L2TPv3 only).
================== ======== ===

For Ethernet session types, this will create an l2tpeth virtual
interface which can then be configured as required. For PPP session
types, a PPPoL2TP socket must also be opened and connected, mapping it
onto the new session. This is covered in "PPPoL2TP Sockets" later.

``L2TP_CMD_SESSION_DESTROY`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Identifies the parent tunnel id of the session
                            to be destroyed.
SESSION_ID         Y        Identifies the session id to be destroyed.
IFNAME             N        Identifies the session by interface name. If
                            set, this overrides any CONN_ID and SESSION_ID
                            attributes. Currently supported for L2TPv3
                            Ethernet sessions only.
================== ======== ===

``L2TP_CMD_SESSION_MODIFY`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            Y        Identifies the parent tunnel id of the session
                            to be modified.
SESSION_ID         Y        Identifies the session id to be modified.
IFNAME             N        Identifies the session by interface name. If
                            set, this overrides any CONN_ID and SESSION_ID
                            attributes. Currently supported for L2TPv3
                            Ethernet sessions only.
DEBUG              N        Debug flags.
RECV_SEQ           N        Enable rx data sequence numbers.
SEND_SEQ           N        Enable tx data sequence numbers.
LNS_MODE           N        Enable LNS mode (auto-enable data sequence
                            numbers).
RECV_TIMEOUT       N        Timeout to wait when reordering received
                            packets.
================== ======== ===

``L2TP_CMD_SESSION_GET`` attributes:-

================== ======== ===
Attribute          Required Use
================== ======== ===
CONN_ID            N        Identifies the tunnel id to be queried.
                            Ignored for DUMP requests.
SESSION_ID         N        Identifies the session id to be queried.
                            Ignored for DUMP requests.
IFNAME             N        Identifies the session by interface name.
                            If set, this overrides any CONN_ID and
                            SESSION_ID attributes. Ignored for DUMP
                            requests. Currently supported for L2TPv3
                            Ethernet sessions only.
================== ======== ===

Application developers should refer to `include/uapi/linux/l2tp.h`_ for
netlink command and attribute definitions.

libmnl Netlink 예제

예제는 netlink socket을 할당하고 Generic Netlink에 연결한 뒤 `L2TP_GENL_NAME`으로 family ID를 찾습니다. 각 요청은 netlink header에 family ID, `NLM_F_REQUEST | NLM_F_ACK`, sequence를 설정하고 Generic Netlink header에 명령과 `L2TP_GENL_VERSION`을 넣습니다.

tunnel 생성 요청은 `L2TP_CMD_TUNNEL_CREATE`와 tunnel socket FD, local·peer tunnel ID, protocol version, encapsulation type을 속성으로 넣습니다. session 생성은 `L2TP_CMD_SESSION_CREATE`와 tunnel·peer tunnel ID, local·peer session ID, pseudowire type을 넣으며 다른 session option도 `l2tp.h`의 속성으로 추가할 수 있습니다.

session 삭제는 `L2TP_CMD_SESSION_DELETE`에 tunnel ID와 session ID를 넣습니다. tunnel과 그 아래 모든 session을 한꺼번에 삭제하려면 `L2TP_CMD_TUNNEL_DELETE`에 tunnel ID를 넣어 요청합니다.

Sample userspace code using libmnl_:

  - Open L2TP netlink socket::

        struct nl_sock *nl_sock;
        int l2tp_nl_family_id;

        nl_sock = nl_socket_alloc();
        genl_connect(nl_sock);
        genl_id = genl_ctrl_resolve(nl_sock, L2TP_GENL_NAME);

  - Create a tunnel::

        struct nlmsghdr *nlh;
        struct genlmsghdr *gnlh;

        nlh = mnl_nlmsg_put_header(buf);
        nlh->nlmsg_type = genl_id; /* assigned to genl socket */
        nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK;
        nlh->nlmsg_seq = seq;

        gnlh = mnl_nlmsg_put_extra_header(nlh, sizeof(*gnlh));
        gnlh->cmd = L2TP_CMD_TUNNEL_CREATE;
        gnlh->version = L2TP_GENL_VERSION;
        gnlh->reserved = 0;

        mnl_attr_put_u32(nlh, L2TP_ATTR_FD, tunl_sock_fd);
        mnl_attr_put_u32(nlh, L2TP_ATTR_CONN_ID, tid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_PEER_CONN_ID, peer_tid);
        mnl_attr_put_u8(nlh, L2TP_ATTR_PROTO_VERSION, protocol_version);
        mnl_attr_put_u16(nlh, L2TP_ATTR_ENCAP_TYPE, encap);

  - Create a session::

        struct nlmsghdr *nlh;
        struct genlmsghdr *gnlh;

        nlh = mnl_nlmsg_put_header(buf);
        nlh->nlmsg_type = genl_id; /* assigned to genl socket */
        nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK;
        nlh->nlmsg_seq = seq;

        gnlh = mnl_nlmsg_put_extra_header(nlh, sizeof(*gnlh));
        gnlh->cmd = L2TP_CMD_SESSION_CREATE;
        gnlh->version = L2TP_GENL_VERSION;
        gnlh->reserved = 0;

        mnl_attr_put_u32(nlh, L2TP_ATTR_CONN_ID, tid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_PEER_CONN_ID, peer_tid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_SESSION_ID, sid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_PEER_SESSION_ID, peer_sid);
        mnl_attr_put_u16(nlh, L2TP_ATTR_PW_TYPE, pwtype);
        /* there are other session options which can be set using netlink
         * attributes during session creation -- see l2tp.h
         */

  - Delete a session::

        struct nlmsghdr *nlh;
        struct genlmsghdr *gnlh;

        nlh = mnl_nlmsg_put_header(buf);
        nlh->nlmsg_type = genl_id; /* assigned to genl socket */
        nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK;
        nlh->nlmsg_seq = seq;

        gnlh = mnl_nlmsg_put_extra_header(nlh, sizeof(*gnlh));
        gnlh->cmd = L2TP_CMD_SESSION_DELETE;
        gnlh->version = L2TP_GENL_VERSION;
        gnlh->reserved = 0;

        mnl_attr_put_u32(nlh, L2TP_ATTR_CONN_ID, tid);
        mnl_attr_put_u32(nlh, L2TP_ATTR_SESSION_ID, sid);

  - Delete a tunnel and all of its sessions (if any)::

        struct nlmsghdr *nlh;
        struct genlmsghdr *gnlh;

        nlh = mnl_nlmsg_put_header(buf);
        nlh->nlmsg_type = genl_id; /* assigned to genl socket */
        nlh->nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK;
        nlh->nlmsg_seq = seq;

        gnlh = mnl_nlmsg_put_extra_header(nlh, sizeof(*gnlh));
        gnlh->cmd = L2TP_CMD_TUNNEL_DELETE;
        gnlh->version = L2TP_GENL_VERSION;
        gnlh->reserved = 0;

        mnl_attr_put_u32(nlh, L2TP_ATTR_CONN_ID, tid);

PPPoL2TP session socket

PPP pseudowire를 사용하는 session은 PPPoL2TP socket을 열어 L2TP session에 연결해야 합니다. `connect()`에 UDP 또는 L2TPIP tunnel socket fd와 local·destination tunnel/session ID를 전달합니다. 역사적 이유로 L2TPv2/L2TPv3와 IPv4/IPv6 조합마다 주소 구조가 조금씩 다르므로 tunnel socket type에 맞는 구조체를 사용해야 합니다.

PPPoX socket의 `setsockopt`와 ioctl로 tunnel/session 동작을 제어합니다. `DEBUG`는 debug category bitmask, `SENDSEQ`는 송신 sequence number 사용 여부, `RECVSEQ`는 sequence number 없는 수신 packet 허용 여부, `LNSMODE`는 LAC/LNS 역할, `REORDERTO`는 millisecond 단위 reorder timeout입니다. `PPPIOCGL2TPSTATS`는 해당 PPPoX socket을 통해 tunnel 또는 session 통계를 얻습니다.

예제는 이미 local·peer 주소가 설정된 tunnel UDP socket을 받아 `socket(AF_PPPOX, SOCK_DGRAM, PX_PROTO_OL2TP)`으로 session socket을 만듭니다. `sockaddr_pppol2tp`에 tunnel fd, peer IPv4 주소·port, local·peer tunnel/session ID를 채워 connect합니다. 성공한 `session_fd`에서는 data path와 별도로 PPP control을 처리하며 L2TP control packet은 계속 `tunnel_fd`에서 읽을 수 있습니다.

PPPoL2TP Session Socket API
---------------------------

For PPP session types, a PPPoL2TP socket must be opened and connected
to the L2TP session.

When creating PPPoL2TP sockets, the application provides information
to the kernel about the tunnel and session in a socket connect()
call. Source and destination tunnel and session ids are provided, as
well as the file descriptor of a UDP or L2TPIP socket. See struct
pppol2tp_addr in `include/linux/if_pppol2tp.h`_. For historical reasons,
there are unfortunately slightly different address structures for
L2TPv2/L2TPv3 IPv4/IPv6 tunnels and userspace must use the appropriate
structure that matches the tunnel socket type.

Userspace may control behavior of the tunnel or session using
setsockopt and ioctl on the PPPoX socket. The following socket
options are supported:-

=========   ===========================================================
DEBUG       bitmask of debug message categories. See below.
SENDSEQ     - 0 => don't send packets with sequence numbers
            - 1 => send packets with sequence numbers
RECVSEQ     - 0 => receive packet sequence numbers are optional
            - 1 => drop receive packets without sequence numbers
LNSMODE     - 0 => act as LAC.
            - 1 => act as LNS.
REORDERTO   reorder timeout (in millisecs). If 0, don't try to reorder.
=========   ===========================================================

In addition to the standard PPP ioctls, a PPPIOCGL2TPSTATS is provided
to retrieve tunnel and session statistics from the kernel using the
PPPoX socket of the appropriate tunnel or session.

Sample userspace code:

  - Create session PPPoX data socket::

        /* Input: the L2TP tunnel UDP socket `tunnel_fd`, which needs to be
         * bound already (both sockname and peername), otherwise it will not be
         * ready.
         */

        struct sockaddr_pppol2tp sax;
        int session_fd;
        int ret;

        session_fd = socket(AF_PPPOX, SOCK_DGRAM, PX_PROTO_OL2TP);
        if (session_fd < 0)
                return -errno;

        sax.sa_family = AF_PPPOX;
        sax.sa_protocol = PX_PROTO_OL2TP;
        sax.pppol2tp.fd = tunnel_fd;
        sax.pppol2tp.addr.sin_addr.s_addr = addr->sin_addr.s_addr;
        sax.pppol2tp.addr.sin_port = addr->sin_port;
        sax.pppol2tp.addr.sin_family = AF_INET;
        sax.pppol2tp.s_tunnel  = tunnel_id;
        sax.pppol2tp.s_session = session_id;
        sax.pppol2tp.d_tunnel  = peer_tunnel_id;
        sax.pppol2tp.d_session = peer_session_id;

        /* session_fd is the fd of the session's PPPoL2TP socket.
         * tunnel_fd is the fd of the tunnel UDP / L2TPIP socket.
         */
        ret = connect(session_fd, (struct sockaddr *)&sax, sizeof(sax));
        if (ret < 0 ) {
                close(session_fd);
                return -errno;
        }

        return session_fd;

PPP channel, interface와 bridge

PPPoL2TP session socket에서 `PPPIOCGCHAN`으로 channel index를 얻고 `/dev/ppp`를 연 뒤 `PPPIOCATTCHAN`으로 channel을 붙이면 PPP channel fd가 만들어집니다. LCP PPP frame은 이 fd에서 읽을 수 있습니다.

PPP interface는 `/dev/ppp`를 다시 열어 `PPPIOCNEWUNIT`으로 unit을 만들고, channel fd에 `PPPIOCCONNECT`를 호출해 그 unit에 연결합니다. IPCP/IPv6CP frame은 interface fd로 전달됩니다. 생성된 `ppp<ifunit>`은 RTM_NEWLINK/ADDR/ROUTE, 기존 network ioctl 또는 `ip` 명령으로 일반 interface처럼 설정합니다.

PPP pseudowire session 두 개를 bridge하면 L2TP tunnel switching 또는 multihop을 구현할 수 있습니다. 두 session에서 channel index를 얻고 첫 channel을 `/dev/ppp` fd에 붙인 뒤 `PPPIOCBRIDGECHAN`으로 두 번째 channel을 연결합니다. 이때 PPP는 로컬에서 종료되지 않고 한 channel의 frame이 다른 channel로 곧바로 전달되므로 local PPP interface를 만들지 않습니다.

bridge 설정 뒤에는 임시 PPP channel fd를 계속 열어 둘 필요가 없고 두 session PPPoX data socket만 유지하면 됩니다. 같은 방법으로 PPPoL2TP channel을 PPPoE 같은 다른 PPP channel type과 bridge할 수도 있으며 PPP 측 세부 정보는 `ppp_generic.rst`를 참조합니다.

L2TP control packets will still be available for read on `tunnel_fd`.

  - Create PPP channel::

        /* Input: the session PPPoX data socket `session_fd` which was created
         * as described above.
         */

        int ppp_chan_fd;
        int chindx;
        int ret;

        ret = ioctl(session_fd, PPPIOCGCHAN, &chindx);
        if (ret < 0)
                return -errno;

        ppp_chan_fd = open("/dev/ppp", O_RDWR);
        if (ppp_chan_fd < 0)
                return -errno;

        ret = ioctl(ppp_chan_fd, PPPIOCATTCHAN, &chindx);
        if (ret < 0) {
                close(ppp_chan_fd);
                return -errno;
        }

        return ppp_chan_fd;

LCP PPP frames will be available for read on `ppp_chan_fd`.

  - Create PPP interface::

        /* Input: the PPP channel `ppp_chan_fd` which was created as described
         * above.
         */

        int ifunit = -1;
        int ppp_if_fd;
        int ret;

        ppp_if_fd = open("/dev/ppp", O_RDWR);
        if (ppp_if_fd < 0)
                return -errno;

        ret = ioctl(ppp_if_fd, PPPIOCNEWUNIT, &ifunit);
        if (ret < 0) {
                close(ppp_if_fd);
                return -errno;
        }

        ret = ioctl(ppp_chan_fd, PPPIOCCONNECT, &ifunit);
        if (ret < 0) {
                close(ppp_if_fd);
                return -errno;
        }

        return ppp_if_fd;

IPCP/IPv6CP PPP frames will be available for read on `ppp_if_fd`.

The ppp<ifunit> interface can then be configured as usual with netlink's
RTM_NEWLINK, RTM_NEWADDR, RTM_NEWROUTE, or ioctl's SIOCSIFMTU, SIOCSIFADDR,
SIOCSIFDSTADDR, SIOCSIFNETMASK, SIOCSIFFLAGS, or with the `ip` command.

  - Bridging L2TP sessions which have PPP pseudowire types (this is also called
    L2TP tunnel switching or L2TP multihop) is supported by bridging the PPP
    channels of the two L2TP sessions to be bridged::

        /* Input: the session PPPoX data sockets `session_fd1` and `session_fd2`
         * which were created as described further above.
         */

        int ppp_chan_fd;
        int chindx1;
        int chindx2;
        int ret;

        ret = ioctl(session_fd1, PPPIOCGCHAN, &chindx1);
        if (ret < 0)
                return -errno;

        ret = ioctl(session_fd2, PPPIOCGCHAN, &chindx2);
        if (ret < 0)
                return -errno;

        ppp_chan_fd = open("/dev/ppp", O_RDWR);
        if (ppp_chan_fd < 0)
                return -errno;

        ret = ioctl(ppp_chan_fd, PPPIOCATTCHAN, &chindx1);
        if (ret < 0) {
                close(ppp_chan_fd);
                return -errno;
        }

        ret = ioctl(ppp_chan_fd, PPPIOCBRIDGECHAN, &chindx2);
        close(ppp_chan_fd);
        if (ret < 0)
                return -errno;

        return 0;

It can be noted that when bridging PPP channels, the PPP session is not locally
terminated, and no local PPP interface is created.  PPP frames arriving on one
channel are directly passed to the other channel, and vice versa.

The PPP channel does not need to be kept open.  Only the session PPPoX data
sockets need to be kept open.

More generally, it is also possible in the same way to e.g. bridge a PPPoL2TP
PPP channel with other types of PPP channels, such as PPPoE.

See more details for the PPP side in ppp_generic.rst.

구형 API, unmanaged tunnel과 디버깅

Linux 2.6.23의 초기 L2TP 구현은 L2TPv2와 PPPoL2TP socket만 지원했습니다. session ID 0에 연결한 전용 PPPoL2TP 관리 socket으로 tunnel을 만들고 닫을 때 파괴했으며, 0이 아닌 session ID에 connect하면 session이 자동 생성되고 socket option으로 parameter를 설정했습니다. 이 API는 아직 지원하지만 새 애플리케이션은 Netlink로 tunnel/session을 먼저 만들고 PPPoL2TP session socket을 연결해야 합니다.

static 또는 unmanaged L2TPv3 tunnel은 사용자 공간 tunnel socket과 control message 교환 없이 양 끝을 수동 구성합니다. 모든 설정은 Netlink로 전달하고 커널이 `L2TP_CMD_TUNNEL_CREATE` 속성으로 tunnel socket을 만듭니다. 별도 L2TP daemon이 필요 없으며 iproute2의 `ip l2tp`로 관리합니다.

debugfs를 mount하면 `/debug/l2tp/tunnels`에서 현재 kernel tunnel/session context 요약을 볼 수 있습니다. 형식이 바뀔 수 있으므로 애플리케이션 상태 조회에는 Netlink를 사용하고 debugfs는 진단에만 써야 합니다. tracepoint는 `/debug/tracing/events/l2tp`에서 확인합니다. `/proc/net/pppol2tp`는 구형 코드 호환용으로 L2TPv2 정보만 제공하므로 사용을 권장하지 않습니다.

Old L2TPv2-only API
-------------------

When L2TP was first added to the Linux kernel in 2.6.23, it
implemented only L2TPv2 and did not include a netlink API. Instead,
tunnel and session instances in the kernel were managed directly using
only PPPoL2TP sockets. The PPPoL2TP socket is used as described in
section "PPPoL2TP Session Socket API" but tunnel and session instances
are automatically created on a connect() of the socket instead of
being created by a separate netlink request:

    - Tunnels are managed using a tunnel management socket which is a
      dedicated PPPoL2TP socket, connected to (invalid) session
      id 0. The L2TP tunnel instance is created when the PPPoL2TP
      tunnel management socket is connected and is destroyed when the
      socket is closed.

    - Session instances are created in the kernel when a PPPoL2TP
      socket is connected to a non-zero session id. Session parameters
      are set using setsockopt. The L2TP session instance is destroyed
      when the socket is closed.

This API is still supported but its use is discouraged. Instead, new
L2TPv2 applications should use netlink to first create the tunnel and
session, then create a PPPoL2TP socket for the session.

Unmanaged L2TPv3 tunnels
------------------------

The kernel L2TP subsystem also supports static (unmanaged) L2TPv3
tunnels. Unmanaged tunnels have no userspace tunnel socket, and
exchange no control messages with the peer to set up the tunnel; the
tunnel is configured manually at each end of the tunnel. All
configuration is done using netlink. There is no need for an L2TP
userspace application in this case -- the tunnel socket is created by
the kernel and configured using parameters sent in the
``L2TP_CMD_TUNNEL_CREATE`` netlink request. The ``ip`` utility of
``iproute2`` has commands for managing static L2TPv3 tunnels; do ``ip
l2tp help`` for more information.

Debugging
---------

The L2TP subsystem offers a range of debugging interfaces through the
debugfs filesystem.

To access these interfaces, the debugfs filesystem must first be mounted::

    # mount -t debugfs debugfs /debug

Files under the l2tp directory can then be accessed, providing a summary
of the current population of tunnel and session contexts existing in the
kernel::

    # cat /debug/l2tp/tunnels

The debugfs files should not be used by applications to obtain L2TP
state information because the file format is subject to change. It is
implemented to provide extra debug information to help diagnose
problems. Applications should instead use the netlink API.

In addition the L2TP subsystem implements tracepoints using the standard
kernel event tracing API.  The available L2TP events can be reviewed as
follows::

    # find /debug/tracing/events/l2tp

Finally, /proc/net/pppol2tp is also provided for backwards compatibility
with the original pppol2tp code. It lists information about L2TPv2
tunnels and sessions only. Its use is discouraged.

내부 socket과 tunnel 수명

UDP socket 자체는 networking core가 구현합니다. L2TP tunnel에 사용하면 UDP socket의 `encap_rcv`와 `encap_destroy` callback을 설정합니다. 수신 시 `l2tp_udp_encap_recv`, 사용자 공간이 socket을 닫을 때 `l2tp_udp_encap_destroy`가 호출됩니다. L2TPIP socket 구현은 `net/l2tp/l2tp_ip.c`와 `net/l2tp/l2tp_ip6.c`에 있습니다.

tunnel마다 `struct l2tp_tunnel` context가 있고 UDP 또는 L2TP/IP socket과 연결되며 session 목록을 보관합니다. L2TP core에 처음 등록할 때 socket reference count를 올려 L2TP 구조체가 참조하는 동안 socket이 제거되지 않게 합니다. tunnel ID는 v2에서 16비트, v3에서 32비트지만 내부에는 32비트로 저장되고 v2/v3가 공유하는 per-net namespace 목록에서 ID로 색인됩니다.

tunnel socket close는 참조 수명 때문에 까다롭습니다. context가 socket 참조를 보유하므로 tunnel이 `sock_put`하기 전에는 `sk_destruct`가 실행되지 않습니다. UDP는 `encap_destroy`, L2TPIP는 socket close handler가 tunnel 종료를 시작합니다. 먼저 모든 session을 닫아 각 session이 tunnel 참조를 놓고, tunnel 참조가 0이 되면 tunnel이 socket 참조를 놓습니다.

Internal Implementation
=======================

This section is for kernel developers and maintainers.

Sockets
-------

UDP sockets are implemented by the networking core. When an L2TP
tunnel is created using a UDP socket, the socket is set up as an
encapsulated UDP socket by setting encap_rcv and encap_destroy
callbacks on the UDP socket. l2tp_udp_encap_recv is called when
packets are received on the socket. l2tp_udp_encap_destroy is called
when userspace closes the socket.

L2TPIP sockets are implemented in `net/l2tp/l2tp_ip.c`_ and
`net/l2tp/l2tp_ip6.c`_.

Tunnels
-------

The kernel keeps a struct l2tp_tunnel context per L2TP tunnel. The
l2tp_tunnel is always associated with a UDP or L2TP/IP socket and
keeps a list of sessions in the tunnel. When a tunnel is first
registered with L2TP core, the reference count on the socket is
increased. This ensures that the socket cannot be removed while L2TP's
data structures reference it.

Tunnels are identified by a unique tunnel id. The id is 16-bit for
L2TPv2 and 32-bit for L2TPv3. Internally, the id is stored as a 32-bit
value.

Tunnels are kept in a per-net list, indexed by tunnel id. The
tunnel id namespace is shared by L2TPv2 and L2TPv3.

Handling tunnel socket close is perhaps the most tricky part of the
L2TP implementation. If userspace closes a tunnel socket, the L2TP
tunnel and all of its sessions must be closed and destroyed. Since the
tunnel context holds a ref on the tunnel socket, the socket's
sk_destruct won't be called until the tunnel sock_put's its
socket. For UDP sockets, when userspace closes the tunnel socket, the
socket's encap_destroy handler is invoked, which L2TP uses to initiate
its tunnel close actions. For L2TPIP sockets, the socket's close
handler initiates the same tunnel close actions. All sessions are
first closed. Each session drops its tunnel ref. When the tunnel ref
reaches zero, the tunnel drops its socket ref.

Session, PPP와 Ethernet 구현

session마다 `struct l2tp_session` context와 session type별 private data가 있습니다. L2TPv2 session은 항상 PPP를 운반합니다. L2TPv3는 Ethernet, PPP, ATM, HDLC, Frame Relay 등을 정의하지만 Linux는 현재 Ethernet과 PPP만 구현합니다. PPP pseudowire에는 socket이 있고 Ethernet pseudowire에는 없습니다.

session ID도 v2는 16비트, v3는 32비트이며 내부에는 32비트로 저장됩니다. session은 부모 tunnel 참조를 잡아 자신이 살아 있는 동안 tunnel이 유지되게 합니다. v2와 v3 session은 별도 per-net 목록에 저장되며 v2 key는 16비트 tunnel ID와 16비트 session ID를 합친 32비트 값, v3 key는 전역적으로 고유한 32비트 session ID입니다.

RFC와 달리 Linux는 역사적으로 서로 다른 tunnel에서 같은 L2TPv3 session ID 충돌을 허용했습니다. 이를 지원하기 위해 socket과 session ID를 key로 하는 per-net hash table을 사용하며, 같은 ID의 후보가 여러 개면 지정한 tunnel socket과 일치하는 session을 선택합니다.

`net/l2tp/l2tp_ppp.c`는 PPPoL2TP socket family를 구현합니다. `sk_user_data`가 `l2tp_session`을 참조하고 PPP control frame만 socket을 통과합니다. PPP data packet은 kernel PPP channel을 통해 L2TP session과 `pppN` netdev 사이에서 완전히 커널 내부로 처리됩니다. close는 Netlink create/destroy와 재연결 경쟁을 고려해 session과 socket 참조를 안전하게 정리해야 합니다.

`net/l2tp/l2tp_eth.c`는 L2TPv3 Ethernet pseudowire와 session별 netdev를 관리합니다. Ethernet session은 Netlink로 생성·삭제되거나 부모 tunnel 제거 시 삭제되며 PPP session과 달리 연결된 socket이 없습니다.

Sessions
--------

The kernel keeps a struct l2tp_session context for each session.  Each
session has private data which is used for data specific to the
session type. With L2TPv2, the session always carries PPP
traffic. With L2TPv3, the session can carry Ethernet frames (Ethernet
pseudowire) or other data types such as PPP, ATM, HDLC or Frame
Relay. Linux currently implements only Ethernet and PPP session types.

Some L2TP session types also have a socket (PPP pseudowires) while
others do not (Ethernet pseudowires).

Like tunnels, L2TP sessions are identified by a unique
session id. Just as with tunnel ids, the session id is 16-bit for
L2TPv2 and 32-bit for L2TPv3. Internally, the id is stored as a 32-bit
value.

Sessions hold a ref on their parent tunnel to ensure that the tunnel
stays extant while one or more sessions references it.

Sessions are kept in a per-net list. L2TPv2 sessions and L2TPv3
sessions are stored in separate lists. L2TPv2 sessions are keyed
by a 32-bit key made up of the 16-bit tunnel ID and 16-bit
session ID. L2TPv3 sessions are keyed by the 32-bit session ID, since
L2TPv3 session ids are unique across all tunnels.

Although the L2TPv3 RFC specifies that L2TPv3 session ids are not
scoped by the tunnel, the Linux implementation has historically
allowed this. Such session id collisions are supported using a per-net
hash table keyed by sk and session ID. When looking up L2TPv3
sessions, the list entry may link to multiple sessions with that
session ID, in which case the session matching the given sk (tunnel)
is used.

PPP
---

`net/l2tp/l2tp_ppp.c`_ implements the PPPoL2TP socket family. Each PPP
session has a PPPoL2TP socket.

The PPPoL2TP socket's sk_user_data references the l2tp_session.

Userspace sends and receives PPP packets over L2TP using a PPPoL2TP
socket. Only PPP control frames pass over this socket: PPP data
packets are handled entirely by the kernel, passing between the L2TP
session and its associated ``pppN`` netdev through the PPP channel
interface of the kernel PPP subsystem.

The L2TP PPP implementation handles the closing of a PPPoL2TP socket
by closing its corresponding L2TP session. This is complicated because
it must consider racing with netlink session create/destroy requests
and pppol2tp_connect trying to reconnect with a session that is in the
process of being closed. PPP sessions hold a ref on their associated
socket in order that the socket remains extants while the session
references it.

Ethernet
--------

`net/l2tp/l2tp_eth.c`_ implements L2TPv3 Ethernet pseudowires. It
manages a netdev for each session.

L2TP Ethernet sessions are created and destroyed by netlink request,
or are destroyed when the tunnel is destroyed. Unlike PPP sessions,
Ethernet sessions do not have an associated socket.

RFC, 구현체, 한계와 테스트

커널 datapath는 L2TPv2 RFC 2661, L2TPv3 RFC 3931, L2TPv3 Ethernet RFC 4719의 기능을 구현합니다. 이를 사용하는 공개 소프트웨어로 iproute2, go-l2tp, tunneldigger, xl2tpd가 있습니다.

현재 openvswitch 연동은 구현되지 않아 OVS Ethernet/VLAN port를 L2TPv3 tunnel에 직접 mapping할 수 없습니다. VLAN pseudowire는 `l2tpethN`에 VLAN sub-interface를 추가해 구현하지만 한 pseudowire가 VLAN 하나만 운반하므로 session마다 netdevice 두 개를 쓰는 구조보다 단일 netdevice가 나을 수 있습니다. 이를 위한 `L2TP_ATTR_VLAN_ID`가 추가됐지만 실제 구현되지 않았습니다.

unmanaged L2TPv3 Ethernet 기능은 `tools/testing/selftests/net/l2tp.sh`의 kernel selftest로 검사합니다. 별도 `l2tp-ktest` suite는 모든 L2TP API와 tunnel/session type을 다루며 향후 built-in selftest에 통합될 수 있습니다. 마지막 link 정의는 Generic Netlink, libmnl, UAPI/header, L2TP 구현 source path와 두 test 위치를 연결합니다.

Miscellaneous
=============

RFCs
----

The kernel code implements the datapath features specified in the
following RFCs:

======= =============== ===================================
RFC2661 L2TPv2          https://tools.ietf.org/html/rfc2661
RFC3931 L2TPv3          https://tools.ietf.org/html/rfc3931
RFC4719 L2TPv3 Ethernet https://tools.ietf.org/html/rfc4719
======= =============== ===================================

Implementations
---------------

A number of open source applications use the L2TP kernel subsystem:

============ ==============================================
iproute2     https://github.com/shemminger/iproute2
go-l2tp      https://github.com/katalix/go-l2tp
tunneldigger https://github.com/wlanslovenija/tunneldigger
xl2tpd       https://github.com/xelerance/xl2tpd
============ ==============================================

Limitations
-----------

The current implementation has a number of limitations:

  1) Interfacing with openvswitch is not yet implemented. It may be
     useful to map OVS Ethernet and VLAN ports into L2TPv3 tunnels.

  2) VLAN pseudowires are implemented using an ``l2tpethN`` interface
     configured with a VLAN sub-interface. Since L2TPv3 VLAN
     pseudowires carry one and only one VLAN, it may be better to use
     a single netdevice rather than an ``l2tpethN`` and ``l2tpethN``:M
     pair per VLAN session. The netlink attribute
     ``L2TP_ATTR_VLAN_ID`` was added for this, but it was never
     implemented.

Testing
-------

Unmanaged L2TPv3 Ethernet features are tested by the kernel's built-in
selftests. See `tools/testing/selftests/net/l2tp.sh`_.

Another test suite, l2tp-ktest_, covers all
of the L2TP APIs and tunnel/session types. This may be integrated into
the kernel's built-in L2TP selftests in the future.

.. Links
.. _Generic Netlink: generic_netlink.html
.. _libmnl: https://www.netfilter.org/projects/libmnl
.. _include/uapi/linux/l2tp.h: ../../../include/uapi/linux/l2tp.h
.. _include/linux/if_pppol2tp.h: ../../../include/linux/if_pppol2tp.h
.. _net/l2tp/l2tp_ip.c: ../../../net/l2tp/l2tp_ip.c
.. _net/l2tp/l2tp_ip6.c: ../../../net/l2tp/l2tp_ip6.c
.. _net/l2tp/l2tp_ppp.c: ../../../net/l2tp/l2tp_ppp.c
.. _net/l2tp/l2tp_eth.c: ../../../net/l2tp/l2tp_eth.c
.. _tools/testing/selftests/net/l2tp.sh: ../../../tools/testing/selftests/net/l2tp.sh
.. _l2tp-ktest: https://github.com/katalix/l2tp-ktest