Kernel 내부 TLS handshake

kTLS record와 handshake service 경계

이 문서는 GPL-2.0 라이선스를 따르며 kernel 내부 TLS handshake API를 설명합니다.

TLS(Transport Layer Security)는 TCP 위에서 실행되는 Upper Layer Protocol(ULP)입니다. Peer authentication과 더불어 end-to-end data integrity와 confidentiality를 제공합니다.

Kernel의 kTLS 구현은 TLS record subprotocol을 처리하지만 TLS session을 수립하는 handshake subprotocol은 처리하지 않습니다. Kernel consumer는 이 문서의 API로 TLS session 수립을 요청할 수 있습니다.

Kernel에서 handshake service를 제공하는 구현 방법은 여러 가지일 수 있습니다. 이 API는 구현 상세를 숨겨 kernel TLS consumer가 실제 handshake 수행 방식을 알 필요가 없게 설계됐습니다.

.. SPDX-License-Identifier: GPL-2.0

=======================
In-Kernel TLS Handshake
=======================

Overview
========

Transport Layer Security (TLS) is a Upper Layer Protocol (ULP) that runs
over TCP. TLS provides end-to-end data integrity and confidentiality in
addition to peer authentication.

The kernel's kTLS implementation handles the TLS record subprotocol, but
does not handle the TLS handshake subprotocol which is used to establish
a TLS session. Kernel consumers can use the API described here to
request TLS session establishment.

There are several possible ways to provide a handshake service in the
kernel. The API described here is designed to hide the details of those
implementations so that in-kernel TLS consumers do not need to be
aware of how the handshake gets done.

Network namespace별 userspace agent

현재 Linux kernel 안에는 TLS handshake 구현이 없습니다. 따라서 kernel consumer가 handshake를 요구할 수 있는 각 network namespace에서 보통 userspace인 handshake agent를 실행합니다. Agent는 대기 중인 handshake request를 알리는 kernel event를 듣습니다.

Kernel은 netlink operation으로 open socket을 agent에 넘기며, 이때 agent의 file descriptor table에 socket descriptor가 만들어집니다. Handshake가 성공하면 agent는 socket을 TLS ULP로 승격하고 `SOL_TLS` socket option으로 session 정보를 설정합니다. 이어 두 번째 netlink operation으로 socket을 kernel에 돌려줍니다.

User handshake agent
====================

As of this writing, there is no TLS handshake implementation in the
Linux kernel. To provide a handshake service, a handshake agent
(typically in user space) is started in each network namespace where a
kernel consumer might require a TLS handshake. Handshake agents listen
for events sent from the kernel that indicate a handshake request is
waiting.

An open socket is passed to a handshake agent via a netlink operation,
which creates a socket descriptor in the agent's file descriptor table.
If the handshake completes successfully, the handshake agent promotes
the socket to use the TLS ULP and sets the session information using the
SOL_TLS socket options. The handshake agent returns the socket to the
kernel via a second netlink operation.

tls_handshake_args와 socket 수명

Kernel TLS consumer는 open socket에서 `tls_client_hello()` 계열 함수를 호출해 client-side handshake를 시작합니다. 먼저 원문 `struct tls_handshake_args`에 request parameter를 채웁니다.

`ta_sock`은 open되고 connected된 socket을 참조합니다. Handshake 도중 socket이 파괴되지 않도록 consumer가 reference를 유지해야 하며 `sock->file`에 `struct file`도 미리 instance화해야 합니다.

Kernel Handshake API
====================

A kernel TLS consumer initiates a client-side TLS handshake on an open
socket by invoking one of the tls_client_hello() functions. First, it
fills in a structure that contains the parameters of the request:

.. code-block:: c

  struct tls_handshake_args {
        struct socket   *ta_sock;
        tls_done_func_t ta_done;
        void            *ta_data;
        const char      *ta_peername;
        unsigned int    ta_timeout_ms;
        key_serial_t    ta_keyring;
        key_serial_t    ta_my_cert;
        key_serial_t    ta_my_privkey;
        unsigned int    ta_num_peerids;
        key_serial_t    ta_my_peerids[5];
  };

The @ta_sock field references an open and connected socket. The consumer
must hold a reference on the socket to prevent it from being destroyed
while the handshake is in progress. The consumer must also have
instantiated a struct file in sock->file.

Callback, hostname, timeout과 X.509 material

`ta_done`은 handshake 완료 때 호출되는 callback이며 자세한 의미는 뒤의 Handshake Completion 절에서 설명합니다.

Consumer는 `ta_peername`에 NUL-terminated hostname을 넣어 ClientHello로 보낼 수 있습니다. 제공하지 않으면 server IP address에 대응하는 DNS hostname을 사용합니다.

`ta_timeout_ms`를 채우면 지정 millisecond 뒤 servicing agent를 종료시킬 수 있습니다. Kernel과 agent가 각각 endpoint를 닫은 뒤 socket이 완전히 close될 수 있게 합니다.

X.509 certificate, certificate private key, pre-shared key 같은 authentication material은 consumer가 request 전에 instance화한 key로 agent에 제공합니다. `ta_keyring`에 private keyring을 지정하면 agent process keyring에 연결되며 다른 subsystem이 key에 접근하지 못하게 합니다.

X.509-authenticated session은 certificate key serial을 `ta_my_cert`, private key serial을 `ta_my_privkey`에 넣고 `tls_client_hello_x509(args, gfp_flags)`를 호출합니다.

함수가 zero를 반환하면 request가 진행 중이며 해당 socket의 `ta_done` callback이 반드시 호출됩니다. Negative errno면 handshake를 시작하지 못했고 그 socket의 callback은 호출되지 않습니다.

@ta_done contains a callback function that is invoked when the handshake
has completed. Further explanation of this function is in the "Handshake
Completion" sesction below.

The consumer can provide a NUL-terminated hostname in the @ta_peername
field that is sent as part of ClientHello. If no peername is provided,
the DNS hostname associated with the server's IP address is used instead.

The consumer can fill in the @ta_timeout_ms field to force the servicing
handshake agent to exit after a number of milliseconds. This enables the
socket to be fully closed once both the kernel and the handshake agent
have closed their endpoints.

Authentication material such as x.509 certificates, private certificate
keys, and pre-shared keys are provided to the handshake agent in keys
that are instantiated by the consumer before making the handshake
request. The consumer can provide a private keyring that is linked into
the handshake agent's process keyring in the @ta_keyring field to prevent
access of those keys by other subsystems.

To request an x.509-authenticated TLS session, the consumer fills in
the @ta_my_cert and @ta_my_privkey fields with the serial numbers of
keys containing an x.509 certificate and the private key for that
certificate. Then, it invokes this function:

.. code-block:: c

  ret = tls_client_hello_x509(args, gfp_flags);

The function returns zero when the handshake request is under way. A
zero return guarantees the callback function @ta_done will be invoked
for this socket. The function returns a negative errno if the handshake
could not be started. A negative errno guarantees the callback function
@ta_done will not be invoked on this socket.

PSK, anonymous client와 server API

Pre-shared key client handshake는 `tls_client_hello_psk(args, gfp_flags)`를 사용합니다. `ta_my_peerids` array에 제안할 peer identity key serial을 넣고 `ta_num_peerids`에 채운 entry 수를 넣습니다. 다른 field는 앞과 같습니다.

Anonymous client handshake는 `tls_client_hello_anon(args, gfp_flags)`를 사용합니다. Agent는 remote에 peer identity 정보를 제시하지 않고 client가 server identity만 검증합니다. 따라서 established session은 encryption만 사용합니다.

Kernel 내부 server consumer는 X.509의 `tls_server_hello_x509(args, gfp_flags)` 또는 PSK의 `tls_server_hello_psk(args, gfp_flags)`를 사용하며 같은 argument structure를 채웁니다.

To initiate a client-side TLS handshake with a pre-shared key, use:

.. code-block:: c

  ret = tls_client_hello_psk(args, gfp_flags);

However, in this case, the consumer fills in the @ta_my_peerids array
with serial numbers of keys containing the peer identities it wishes
to offer, and the @ta_num_peerids field with the number of array
entries it has filled in. The other fields are filled in as above.


To initiate an anonymous client-side TLS handshake use:

.. code-block:: c

  ret = tls_client_hello_anon(args, gfp_flags);

The handshake agent presents no peer identity information to the remote
during this type of handshake. Only server authentication (ie the client
verifies the server's identity) is performed during the handshake. Thus
the established session uses encryption only.


Consumers that are in-kernel servers use:

.. code-block:: c

  ret = tls_server_hello_x509(args, gfp_flags);

or

.. code-block:: c

  ret = tls_server_hello_psk(args, gfp_flags);

The argument structure is filled in as above.

Handshake request 취소

Ctrl-C 같은 긴급 event 때문에 request를 취소해야 하면 `bool tls_handshake_cancel(sock)`을 호출합니다.

True면 `sock`의 handshake request가 취소됐고 consumer completion callback은 호출되지 않습니다. False면 completion callback이 이미 호출된 상태입니다.

If the consumer needs to cancel the handshake request, say, due to a ^C
or other exigent event, the consumer can invoke:

.. code-block:: c

  bool tls_handshake_cancel(sock);

This function returns true if the handshake request associated with
@sock has been canceled. The consumer's handshake completion callback
will not be invoked. If this function returns false, then the consumer's
completion callback has already been invoked.

Completion callback과 status

Handshake agent가 처리를 끝내면 socket을 다시 사용할 수 있다고 kernel에 알리고 `tls_handshake_args.ta_done`의 consumer completion callback을 호출합니다.

Callback type은 `tls_done_func_t(void *data, int status, key_serial_t peerid)`입니다. Consumer가 `ta_data`에 넣은 cookie가 `data`로 돌아오며, 이를 사용해 callback을 handshake 완료를 기다리던 thread와 대응시킵니다.

`peerid`는 remote peer identity를 담은 key serial이며 session이 authenticated되지 않았다면 `TLS_NO_PEERID`입니다.

Handshake가 실패하면 즉시 socket을 close하고 파괴하는 것이 권장됩니다.

Handshake Completion
====================

When the handshake agent has completed processing, it notifies the
kernel that the socket may be used by the consumer again. At this point,
the consumer's handshake completion callback, provided in the @ta_done
field in the tls_handshake_args structure, is invoked.

The synopsis of this function is:

.. code-block:: c

  typedef void        (*tls_done_func_t)(void *data, int status,
                                   key_serial_t peerid);

The consumer provides a cookie in the @ta_data field of the
tls_handshake_args structure that is returned in the @data parameter of
this callback. The consumer uses the cookie to match the callback to the
thread waiting for the handshake to complete.

The success status of the handshake is returned via the @status
parameter:

+------------+----------------------------------------------+
|  status    |  meaning                                     |
+============+==============================================+
|  0         |  TLS session established successfully        |
+------------+----------------------------------------------+
|  -EACCESS  |  Remote peer rejected the handshake or       |
|            |  authentication failed                       |
+------------+----------------------------------------------+
|  -ENOMEM   |  Temporary resource allocation failure       |
+------------+----------------------------------------------+
|  -EINVAL   |  Consumer provided an invalid argument       |
+------------+----------------------------------------------+
|  -ENOKEY   |  Missing authentication material             |
+------------+----------------------------------------------+
|  -EIO      |  An unexpected fault occurred                |
+------------+----------------------------------------------+

The @peerid parameter contains the serial number of a key containing the
remote peer's identity or the value TLS_NO_PEERID if the session is not
authenticated.

A best practice is to close and destroy the socket immediately if the
handshake failed.

Handshake 중 receive 차단과 이후 CMSG

Handshake가 진행되는 동안 kernel consumer는 socket의 `sk_data_ready` callback을 바꿔 모든 incoming data를 무시해야 합니다. Completion callback이 호출된 뒤 정상 receive를 재개할 수 있습니다.

TLS session 수립 뒤 consumer는 이후 모든 `sock_recvmsg()`에 포함되는 control message(CMSG)를 받을 buffer를 제공하고 내용을 검사해야 합니다. 각 CMSG는 receive message data가 TLS record data인지 session metadata인지 나타냅니다.

Socket이 TLS ULP로 승격된 뒤 kTLS consumer가 decrypted application data, alert와 handshake packet을 구분하는 방법은 `tls.rst`를 참조합니다.

Other considerations
--------------------

While a handshake is under way, the kernel consumer must alter the
socket's sk_data_ready callback function to ignore all incoming data.
Once the handshake completion callback function has been invoked, normal
receive operation can be resumed.

Once a TLS session is established, the consumer must provide a buffer
for and then examine the control message (CMSG) that is part of every
subsequent sock_recvmsg(). Each control message indicates whether the
received message data is TLS record data or session metadata.

See tls.rst for details on how a kTLS consumer recognizes incoming
(decrypted) application data, alerts, and handshake packets once the
socket has been promoted to use the TLS ULP.