Kernel Livepatching Core

동기와 Kprobes·Ftrace·Livepatching

Kernel livepatching은 system을 reboot하지 않고 critical function call을 새 구현으로 redirect합니다. 복잡한 과학 계산을 수행하거나 peak load를 처리하는 system처럼 reboot를 꺼리는 환경에서 uptime과 안정성·보안을 함께 유지하는 것이 목적입니다.

Linux kernel에는 code execution을 redirect하는 kernel probe, function tracing, livepatching이 있습니다. Kernel probe는 임의 instruction을 breakpoint instruction으로 바꾸는 가장 일반적인 방식입니다.

Function tracer는 function entry에 가까운 compiler-generated 위치에서 tracing code를 호출합니다. 이 위치는 GCC `-pg` option으로 생성됩니다.

Livepatch는 function parameter나 stack이 바뀌기 전인 function entry의 맨 앞에서 code를 redirect해야 합니다.

세 방식 모두 runtime에 기존 code를 수정하므로 서로 충돌하지 않게 조정해야 합니다. 대부분은 dynamic ftrace framework를 공통 기반으로 사용해 해결합니다.

Function entry를 probe하는 Kprobe는 `CONFIG_KPROBES_ON_FTRACE`일 때 ftrace handler로 등록됩니다. Livepatch의 alternative function도 custom ftrace handler를 통해 호출됩니다. 다만 뒤의 limitations 절에서 설명하는 제약이 있습니다.

=========
Livepatch
=========

This document outlines basic information about kernel livepatching.

.. Table of Contents:

.. contents:: :local:


1. Motivation
=============

There are many situations where users are reluctant to reboot a system. It may
be because their system is performing complex scientific computations or under
heavy load during peak usage. In addition to keeping systems up and running,
users want to also have a stable and secure system. Livepatching gives users
both by allowing for function calls to be redirected; thus, fixing critical
functions without a system reboot.


2. Kprobes, Ftrace, Livepatching
================================

There are multiple mechanisms in the Linux kernel that are directly related
to redirection of code execution; namely: kernel probes, function tracing,
and livepatching:

  - The kernel probes are the most generic. The code can be redirected by
    putting a breakpoint instruction instead of any instruction.

  - The function tracer calls the code from a predefined location that is
    close to the function entry point. This location is generated by the
    compiler using the '-pg' gcc option.

  - Livepatching typically needs to redirect the code at the very beginning
    of the function entry before the function parameters or the stack
    are in any way modified.

All three approaches need to modify the existing code at runtime. Therefore
they need to be aware of each other and not step over each other's toes.
Most of these problems are solved by using the dynamic ftrace framework as
a base. A Kprobe is registered as a ftrace handler when the function entry
is probed, see CONFIG_KPROBES_ON_FTRACE. Also an alternative function from
a live patch is called with the help of a custom ftrace handler. But there are
some limitations, see below.

Per-task consistency model과 안전 전환

Function은 input parameter, lock acquire·release, data read·write·processing, return value를 포함하는 정의된 semantic을 가집니다. NULL pointer check, boundary check, memory barrier 추가, critical section locking처럼 function 외부에서 보이는 semantic을 바꾸지 않는 self-contained fix는 function별로 독립 update할 수 있습니다.

그러나 여러 function의 lock ordering을 동시에 바꾸거나 temporary structure의 의미와 관련 function 전체를 함께 바꾸는 복잡한 fix는 해당 unit이 모든 새 function version을 동시에 사용해야 합니다. 전환은 관련 lock이 해제되고 수정 structure에 data가 남아 있지 않은 안전한 시점에 일어나야 합니다.

Consistency model은 system consistency를 유지하면서 새 구현을 사용할 수 있는 조건을 정의합니다. Livepatch model은 kGraft의 per-task consistency와 syscall barrier switching, kpatch의 stack-trace switching을 결합하고 여러 fallback을 제공합니다.

Patch는 task별로 안전하다고 판정될 때 적용됩니다. Enable하면 task들이 patched state로 수렴하는 transition에 들어가며 보통 몇 초 안에 끝납니다. Disable할 때도 같은 절차로 patched state에서 unpatched state로 수렴합니다.

Interrupt handler는 interrupt된 task의 patch state를 상속합니다. Fork된 child task도 parent의 patch state를 상속합니다.

첫째이자 가장 효과적인 판정은 sleeping task의 stack 검사입니다. 해당 task stack에 affected function이 없으면 patched state로 전환하고, 있으면 주기적으로 다시 검사합니다. 이 방식은 architecture가 `HAVE_RELIABLE_STACKTRACE`를 제공할 때만 사용할 수 있습니다.

둘째는 kernel exit switching입니다. System call, userspace IRQ, signal 처리 뒤 userspace로 돌아갈 때 task를 전환합니다. Affected function에서 sleep하는 I/O-bound task는 `SIGSTOP`과 `SIGCONT`로 kernel을 빠져나오게 할 수 있습니다. CPU-bound userspace task는 다음 IRQ interrupt 때 전환됩니다.

셋째는 kernel을 떠나지 않는 idle `swapper` task를 위한 방법입니다. Idle loop의 `klp_update_patch_state()`가 CPU idle 진입 전에 state를 바꿉니다. 일반 kthread에는 아직 동등한 공통 방법이 없습니다.

Reliable stacktrace가 없는 architecture는 둘째 방식에만 의존하므로 old function에서 반환하지 않은 task가 남을 가능성이 큽니다. 특히 깨워지지 않는 kthread는 강제로 wake해야 합니다. Kthread를 patch할 다른 방법이 없다면 이런 architecture는 kernel livepatching에서 완전 지원으로 보지 않습니다.

`/sys/kernel/livepatch/<patch>/transition`은 patch가 transition 중인지 표시합니다. 동시에 transition할 수 있는 patch는 하나뿐이며 initial patch state에 stuck된 task가 있으면 무기한 지속될 수 있습니다.

진행 중 transition에서 `/sys/kernel/livepatch/<patch>/enabled`에 반대 값을 쓰면 transition을 뒤집어 취소할 수 있고 모든 task가 원래 state로 다시 수렴합니다.

`/proc/<pid>/patch_state`는 transition을 막는 task를 찾는 데 사용합니다. Transition 중에는 unpatched 0, patched 1을 표시하고 transition이 없으면 -1을 표시합니다.

Blocking task에 `SIGSTOP`·`SIGCONT`를 보내 전환을 강제할 수 있지만 system에 해로울 수 있습니다. 더 나은 대안은 pending signal data 없이 task를 interrupt하거나 wake하는 fake signal이며 남은 blocking task에 15초마다 자동 전송됩니다.

Administrator는 `/sys/kernel/livepatch/<patch>/force`에 1을 써 모든 task의 `TIF_PATCH_PENDING`을 지우고 patched state를 강제할 수 있습니다. 이는 blocking task 때문에 장시간 stuck된 경우에만 쓰는 최후 수단입니다.

Force 전에 blocking task의 stack trace 등 필요한 data를 수집하고 patch distributor의 승인을 받아야 합니다. `/proc/<pid>/stack`이 어떤 unpatched function에서 sleep하는지 찾는 데 도움을 줍니다. 무단 사용은 patch 성격과 task 위치에 따라 system을 손상시킬 수 있습니다.

Force를 사용하면 patch module의 `rmmod`가 영구히 금지됩니다. 해당 module 안에서 sleep하는 task가 없음을 보장할 수 없고 disable·enable 반복 시 reference count가 무한히 증가할 수 있기 때문입니다.

Force는 이후 livepatch 적용에도 악영향을 줄 수 있습니다. 먼저 transition 취소를 고려하고, force를 썼다면 reboot를 계획하며 추가 livepatch를 적용하지 않아야 합니다.

3. Consistency model
====================

Functions are there for a reason. They take some input parameters, acquire or
release locks, read, process, and even write some data in a defined way,
have return values. In other words, each function has a defined semantic.

Many fixes do not change the semantic of the modified functions. For
example, they add a NULL pointer or a boundary check, fix a race by adding
a missing memory barrier, or add some locking around a critical section.
Most of these changes are self contained and the function presents itself
the same way to the rest of the system. In this case, the functions might
be updated independently one by one.

But there are more complex fixes. For example, a patch might change
ordering of locking in multiple functions at the same time. Or a patch
might exchange meaning of some temporary structures and update
all the relevant functions. In this case, the affected unit
(thread, whole kernel) need to start using all new versions of
the functions at the same time. Also the switch must happen only
when it is safe to do so, e.g. when the affected locks are released
or no data are stored in the modified structures at the moment.

The theory about how to apply functions a safe way is rather complex.
The aim is to define a so-called consistency model. It attempts to define
conditions when the new implementation could be used so that the system
stays consistent.

Livepatch has a consistency model which is a hybrid of kGraft and
kpatch:  it uses kGraft's per-task consistency and syscall barrier
switching combined with kpatch's stack trace switching.  There are also
a number of fallback options which make it quite flexible.

Patches are applied on a per-task basis, when the task is deemed safe to
switch over.  When a patch is enabled, livepatch enters into a
transition state where tasks are converging to the patched state.
Usually this transition state can complete in a few seconds.  The same
sequence occurs when a patch is disabled, except the tasks converge from
the patched state to the unpatched state.

An interrupt handler inherits the patched state of the task it
interrupts.  The same is true for forked tasks: the child inherits the
patched state of the parent.

Livepatch uses several complementary approaches to determine when it's
safe to patch tasks:

1. The first and most effective approach is stack checking of sleeping
   tasks.  If no affected functions are on the stack of a given task,
   the task is patched.  In most cases this will patch most or all of
   the tasks on the first try.  Otherwise it'll keep trying
   periodically.  This option is only available if the architecture has
   reliable stacks (HAVE_RELIABLE_STACKTRACE).

2. The second approach, if needed, is kernel exit switching.  A
   task is switched when it returns to user space from a system call, a
   user space IRQ, or a signal.  It's useful in the following cases:

   a) Patching I/O-bound user tasks which are sleeping on an affected
      function.  In this case you have to send SIGSTOP and SIGCONT to
      force it to exit the kernel and be patched.
   b) Patching CPU-bound user tasks.  If the task is highly CPU-bound
      then it will get patched the next time it gets interrupted by an
      IRQ.

3. For idle "swapper" tasks, since they don't ever exit the kernel, they
   instead have a klp_update_patch_state() call in the idle loop which
   allows them to be patched before the CPU enters the idle state.

   (Note there's not yet such an approach for kthreads.)

Architectures which don't have HAVE_RELIABLE_STACKTRACE solely rely on
the second approach. It's highly likely that some tasks may still be
running with an old version of the function, until that function
returns. In this case you would have to signal the tasks. This
especially applies to kthreads. They may not be woken up and would need
to be forced. See below for more information.

Unless we can come up with another way to patch kthreads, architectures
without HAVE_RELIABLE_STACKTRACE are not considered fully supported by
the kernel livepatching.

The /sys/kernel/livepatch/<patch>/transition file shows whether a patch
is in transition.  Only a single patch can be in transition at a given
time.  A patch can remain in transition indefinitely, if any of the tasks
are stuck in the initial patch state.

A transition can be reversed and effectively canceled by writing the
opposite value to the /sys/kernel/livepatch/<patch>/enabled file while
the transition is in progress.  Then all the tasks will attempt to
converge back to the original patch state.

There's also a /proc/<pid>/patch_state file which can be used to
determine which tasks are blocking completion of a patching operation.
If a patch is in transition, this file shows 0 to indicate the task is
unpatched and 1 to indicate it's patched.  Otherwise, if no patch is in
transition, it shows -1.  Any tasks which are blocking the transition
can be signaled with SIGSTOP and SIGCONT to force them to change their
patched state. This may be harmful to the system though. Sending a fake signal
to all remaining blocking tasks is a better alternative. No proper signal is
actually delivered (there is no data in signal pending structures). Tasks are
interrupted or woken up, and forced to change their patched state. The fake
signal is automatically sent every 15 seconds.

Administrator can also affect a transition through
/sys/kernel/livepatch/<patch>/force attribute. Writing 1 there clears
TIF_PATCH_PENDING flag of all tasks and thus forces the tasks to the patched
state. Important note! The force attribute is intended for cases when the
transition gets stuck for a long time because of a blocking task. Administrator
is expected to collect all necessary data (namely stack traces of such blocking
tasks) and request a clearance from a patch distributor to force the transition.
Unauthorized usage may cause harm to the system. It depends on the nature of the
patch, which functions are (un)patched, and which functions the blocking tasks
are sleeping in (/proc/<pid>/stack may help here). Removal (rmmod) of patch
modules is permanently disabled when the force feature is used. It cannot be
guaranteed there is no task sleeping in such module. It implies unbounded
reference count if a patch module is disabled and enabled in a loop.

Moreover, the usage of force may also affect future applications of live
patches and cause even more harm to the system. Administrator should first
consider to simply cancel a transition (see above). If force is used, reboot
should be planned and no more live patches applied.

새 architecture의 consistency 지원

새 architecture에 consistency model을 추가하는 첫 번째 선택은 `CONFIG_HAVE_RELIABLE_STACKTRACE`를 제공하는 것입니다. 이를 위해 objtool을 port하고, DWARF가 아닌 unwinder에서는 stack trace code가 stack 위 interrupt를 감지할 방법도 보장해야 합니다.

대안은 모든 kthread의 안전한 위치에 `klp_update_patch_state()`를 넣는 것입니다. Kthread의 반복 loop에서 lock이 잡혀 있지 않고 data structure가 잘 정의된 지점을 선택해야 합니다.

Workqueue나 kthread worker API는 generic loop에서 독립 action을 처리하므로 안전 지점이 명확합니다. Custom loop를 가진 kthread는 case-by-case로 매우 신중하게 선택해야 합니다.

이 대안을 쓰면 reliable stacktrace가 없는 architecture도 kernel·userspace boundary를 지나는 user task와 지정 patch point에 도달한 kthread·idle task를 전환할 수 있습니다.

다만 user task에 signal을 보내고 kthread를 깨워야 하므로 reliable stacktrace보다 불편합니다. 아직 reliable stack trace가 없는 architecture에는 유용한 backup이 될 수 있습니다.

3.1 Adding consistency model support to new architectures
---------------------------------------------------------

For adding consistency model support to new architectures, there are a
few options:

1) Add CONFIG_HAVE_RELIABLE_STACKTRACE.  This means porting objtool, and
   for non-DWARF unwinders, also making sure there's a way for the stack
   tracing code to detect interrupts on the stack.

2) Alternatively, ensure that every kthread has a call to
   klp_update_patch_state() in a safe location.  Kthreads are typically
   in an infinite loop which does some action repeatedly.  The safe
   location to switch the kthread's patch state would be at a designated
   point in the loop where there are no locks taken and all data
   structures are in a well-defined state.

   The location is clear when using workqueues or the kthread worker
   API.  These kthreads process independent actions in a generic loop.

   It's much more complicated with kthreads which have a custom loop.
   There the safe location must be carefully selected on a case-by-case
   basis.

   In that case, arches without HAVE_RELIABLE_STACKTRACE would still be
   able to use the non-stack-checking parts of the consistency model:

   a) patching user tasks when they cross the kernel/user space
      boundary; and

   b) patching kthreads and idle tasks at their designated patch points.

   This option isn't as good as option 1 because it requires signaling
   user tasks and waking kthreads to patch them.  But it could still be
   a good backup option for those architectures which don't have
   reliable stack traces yet.

Livepatch module과 새 function

Livepatch는 kernel module로 배포하며 예제는 `samples/livepatch/livepatch-sample.c`입니다.

Module은 교체할 function의 새 구현, 원본과 새 구현 관계를 설명하는 metadata 구조체, module load 때 새 code 사용을 시작하는 code, module 제거 전 cleanup code를 포함합니다.

새 function version은 보통 원본 source에서 복사합니다. Backtrace 등에서 원본과 구분하도록 이름에 prefix를 붙이는 것이 좋고, 직접 호출되지 않아 global visibility가 필요 없으므로 `static`으로 선언할 수 있습니다.

Patch에는 실제로 수정된 function만 넣습니다. 새 function이 원본 source file의 local function이나 data에 접근해야 한다면 generated livepatch module의 특별 relocation section으로 해결할 수 있습니다. 자세한 형식은 `Documentation/livepatch/module-elf-format.rst`에 있습니다.

4. Livepatch module
===================

Livepatches are distributed using kernel modules, see
samples/livepatch/livepatch-sample.c.

The module includes a new implementation of functions that we want
to replace. In addition, it defines some structures describing the
relation between the original and the new implementation. Then there
is code that makes the kernel start using the new code when the livepatch
module is loaded. Also there is code that cleans up before the
livepatch module is removed. All this is explained in more details in
the next sections.


4.1. New functions
------------------

New versions of functions are typically just copied from the original
sources. A good practice is to add a prefix to the names so that they
can be distinguished from the original ones, e.g. in a backtrace. Also
they can be declared as static because they are not called directly
and do not need the global visibility.

The patch contains only functions that are really modified. But they
might want to access functions or data from the original source file
that may only be locally accessible. This can be solved by a special
relocation section in the generated livepatch module, see
Documentation/livepatch/module-elf-format.rst for more details.

klp_func·klp_object·klp_patch metadata

Patch metadata는 세 계층으로 나뉩니다. `struct klp_func`는 patch할 function마다 하나씩 존재하며 원본 function과 새 구현의 관계를 설명합니다.

`klp_func`는 원본 function 이름을 string으로 보관하고 runtime에 kallsyms로 주소를 찾습니다. 새 function 주소는 보통 같은 source file에 정의한 function pointer를 직접 대입합니다.

같은 이름의 function이 여러 개면 optional symbol position으로 구분할 수 있습니다. 이 값은 kallsyms database의 절대 위치가 아니라 특정 object인 `vmlinux` 또는 kernel module 안에서 그 이름이 발견된 순서입니다. Kallsyms는 object name을 기준으로 symbol을 검색할 수 있습니다.

`struct klp_object`는 같은 object에 속한 `klp_func` 배열을 정의합니다. Object가 `vmlinux`이면 이름은 `NULL`, module이면 module name입니다.

이 계층은 object별 function을 함께 관리합니다. Target module이 patch보다 나중에 load될 수 있으며 관련 function은 object가 사용 가능해졌을 때 patch됩니다.

`struct klp_patch`는 `klp_object` 배열을 정의합니다. 모든 patched function을 일관되고 최종적으로 동기화해 적용하며, 모든 symbol을 찾았을 때 전체 patch를 적용합니다. 아직 load되지 않은 kernel module object의 symbol만 예외입니다.

4.2. Metadata
-------------

The patch is described by several structures that split the information
into three levels:

  - struct klp_func is defined for each patched function. It describes
    the relation between the original and the new implementation of a
    particular function.

    The structure includes the name, as a string, of the original function.
    The function address is found via kallsyms at runtime.

    Then it includes the address of the new function. It is defined
    directly by assigning the function pointer. Note that the new
    function is typically defined in the same source file.

    As an optional parameter, the symbol position in the kallsyms database can
    be used to disambiguate functions of the same name. This is not the
    absolute position in the database, but rather the order it has been found
    only for a particular object ( vmlinux or a kernel module ). Note that
    kallsyms allows for searching symbols according to the object name.

  - struct klp_object defines an array of patched functions (struct
    klp_func) in the same object. Where the object is either vmlinux
    (NULL) or a module name.

    The structure helps to group and handle functions for each object
    together. Note that patched modules might be loaded later than
    the patch itself and the relevant functions might be patched
    only when they are available.


  - struct klp_patch defines an array of patched objects (struct
    klp_object).

    This structure handles all patched functions consistently and eventually,
    synchronously. The whole patch is applied only when all patched
    symbols are found. The only exception are symbols from objects
    (kernel modules) that have not been loaded yet.

    For more details on how the patch is applied on a per-task basis,
    see the "Consistency model" section.

Life-cycle, loading, enabling

Livepatch lifecycle은 loading, enabling, replacing, disabling, removing의 다섯 기본 operation으로 설명합니다. Replacing과 disabling은 특정 patch에는 같은 결과를 내지만 system 전체에는 다르므로 상호 배타적입니다.

합리적인 loading 방식은 livepatch kernel module load 중 patch를 enable하는 것입니다. `module_init()` callback에서 `klp_enable_patch()`를 호출합니다.

Module만 관련 `struct klp_patch`에 쉽게 접근할 수 있고, enable 실패 error code로 module load 자체를 거부할 수 있다는 두 이유가 있습니다.

Enable 첫 단계에서는 patched function 이름으로 주소를 찾고 special relocation을 적용하며 `/sys/kernel/livepatch/<name>` 아래 entry를 만듭니다. 하나라도 실패하면 patch를 거부합니다.

둘째로 task가 patched state로 수렴하는 transition에 들어갑니다. 원본 function을 처음 patch하는 경우 function별 `struct klp_ops`를 만들고 universal ftrace handler를 등록합니다. 이 단계는 `transition` 값 1로 표시됩니다.

모든 task가 patched state에 도달하면 `transition`은 0으로 바뀝니다.

같은 function을 여러 번 patch해도 ftrace handler는 function당 한 번만 등록됩니다. 후속 patch는 `klp_ops.func_stack` list에 entry를 추가하고 handler가 consistency model에 따라 올바른 구현을 선택합니다.

모든 변경의 consistency를 유지하려면 cumulative livepatch 사용을 강하게 권장합니다. 이 경우 같은 function이 두 번 patch되는 기간은 transition 동안으로 제한됩니다.

5. Livepatch life-cycle
=======================

Livepatching can be described by five basic operations:
loading, enabling, replacing, disabling, removing.

Where the replacing and the disabling operations are mutually
exclusive. They have the same result for the given patch but
not for the system.


5.1. Loading
------------

The only reasonable way is to enable the patch when the livepatch kernel
module is being loaded. For this, klp_enable_patch() has to be called
in the module_init() callback. There are two main reasons:

First, only the module has an easy access to the related struct klp_patch.

Second, the error code might be used to refuse loading the module when
the patch cannot get enabled.


5.2. Enabling
-------------

The livepatch gets enabled by calling klp_enable_patch() from
the module_init() callback. The system will start using the new
implementation of the patched functions at this stage.

First, the addresses of the patched functions are found according to their
names. The special relocations, mentioned in the section "New functions",
are applied. The relevant entries are created under
/sys/kernel/livepatch/<name>. The patch is rejected when any above
operation fails.

Second, livepatch enters into a transition state where tasks are converging
to the patched state. If an original function is patched for the first
time, a function specific struct klp_ops is created and an universal
ftrace handler is registered\ [#]_. This stage is indicated by a value of '1'
in /sys/kernel/livepatch/<name>/transition. For more information about
this process, see the "Consistency model" section.

Finally, once all tasks have been patched, the 'transition' value changes
to '0'.

.. [#]

    Note that functions might be patched multiple times. The ftrace handler
    is registered only once for a given function. Further patches just add
    an entry to the list (see field `func_stack`) of the struct klp_ops.
    The right implementation is selected by the ftrace handler, see
    the "Consistency model" section.

    That said, it is highly recommended to use cumulative livepatches
    because they help keeping the consistency of all changes. In this case,
    functions might be patched two times only during the transition period.

Replacing·disabling·removing과 sysfs

`.replace` flag가 설정된 cumulative patch는 enable된 모든 patch를 대체할 수 있습니다. 새 patch의 transition이 끝나면 교체된 patch의 모든 `klp_func`가 대응 `klp_ops`에서 제거됩니다.

새 patch가 더 이상 수정하지 않는 function은 `func_stack`이 비게 되므로 ftrace handler를 unregister하고 `klp_ops`를 해제합니다. 자세한 내용은 `Documentation/livepatch/cumulative-patches.rst`에 있습니다.

Enabled patch를 disable하려면 `/sys/kernel/livepatch/<name>/enabled`에 0을 씁니다. Task는 unpatched state로 수렴하며 system은 이전에 enable된 patch code 또는 원본 code를 사용하기 시작합니다. Transition 중에는 `transition = 1`입니다.

모든 task가 unpatched되면 `transition = 0`이 되고 disable 대상 patch의 `klp_func`를 대응 `klp_ops`에서 제거합니다. `func_stack`이 비면 ftrace handler와 `klp_ops`도 제거하고 마지막으로 sysfs interface를 파괴합니다.

Module 제거는 module이 제공하는 function 사용자가 없을 때만 안전합니다. Forced transition은 old code에서 sleep하거나 실행하는 task가 없음을 보장하지 못하므로 force 기능을 한 번이라도 사용하면 module 제거가 영구히 disable됩니다.

Registered patch 정보와 enable·disable control은 `/sys/kernel/livepatch` 아래에 있습니다. `<patch>/force`는 administrator가 patching operation에 개입하는 attribute이며 자세한 ABI는 `Documentation/ABI/testing/sysfs-kernel-livepatch`에 있습니다.

5.3. Replacing
--------------

All enabled patches might get replaced by a cumulative patch that
has the .replace flag set.

Once the new patch is enabled and the 'transition' finishes then
all the functions (struct klp_func) associated with the replaced
patches are removed from the corresponding struct klp_ops. Also
the ftrace handler is unregistered and the struct klp_ops is
freed when the related function is not modified by the new patch
and func_stack list becomes empty.

See Documentation/livepatch/cumulative-patches.rst for more details.


5.4. Disabling
--------------

Enabled patches might get disabled by writing '0' to
/sys/kernel/livepatch/<name>/enabled.

First, livepatch enters into a transition state where tasks are converging
to the unpatched state. The system starts using either the code from
the previously enabled patch or even the original one. This stage is
indicated by a value of '1' in /sys/kernel/livepatch/<name>/transition.
For more information about this process, see the "Consistency model"
section.

Second, once all tasks have been unpatched, the 'transition' value changes
to '0'. All the functions (struct klp_func) associated with the to-be-disabled
patch are removed from the corresponding struct klp_ops. The ftrace handler
is unregistered and the struct klp_ops is freed when the func_stack list
becomes empty.

Third, the sysfs interface is destroyed.


5.5. Removing
-------------

Module removal is only safe when there are no users of functions provided
by the module. This is the reason why the force feature permanently
disables the removal. Only when the system is successfully transitioned
to a new patch state (patched/unpatched) without being forced it is
guaranteed that no task sleeps or runs in the old code.


6. Sysfs
========

Information about the registered patches can be found under
/sys/kernel/livepatch. The patches could be enabled and disabled
by writing there.

/sys/kernel/livepatch/<patch>/force attributes allow administrator to affect a
patching operation.

See Documentation/ABI/testing/sysfs-kernel-livepatch for more details.

현재 구현의 제한

첫째, trace 가능한 function만 patch할 수 있습니다. Livepatch가 dynamic ftrace를 기반으로 하므로 ftrace 자체나 livepatch ftrace handler를 구현하는 function을 patch하면 무한 loop가 생길 수 있습니다. 이런 위험 function은 `notrace`로 표시해 실수를 막습니다.

둘째, dynamic ftrace 위치가 function 맨 앞에 있어야 안정적으로 동작합니다. Stack이나 parameter가 조금이라도 바뀌기 전에 redirect해야 하며 x86_64에서는 GCC `-fentry` option이 필요합니다.

PPC port는 relative addressing과 TOC를 사용하므로 예외입니다. 각 function이 ftrace handler 호출 전에 TOC를 처리하고 LR을 저장한 뒤 return 때 되돌려야 합니다. Generic ftrace도 같은 문제가 있어 ftrace level에서 처리합니다.

셋째, ftrace framework를 사용하는 kretprobe와 patched function이 충돌합니다. 둘 다 return address를 바꾸는 ftrace handler를 사용하므로 먼저 등록한 쪽이 이기며, 다른 쪽 handler가 이미 사용 중이면 probe 또는 patch가 거부됩니다.

넷째, 원본 function의 Kprobe는 code가 새 구현으로 redirect된 뒤 무시됩니다. 이 상황에 대한 warning을 추가하는 작업이 진행 중입니다.

7. Limitations
==============

The current Livepatch implementation has several limitations:

  - Only functions that can be traced could be patched.

    Livepatch is based on the dynamic ftrace. In particular, functions
    implementing ftrace or the livepatch ftrace handler could not be
    patched. Otherwise, the code would end up in an infinite loop. A
    potential mistake is prevented by marking the problematic functions
    by "notrace".



  - Livepatch works reliably only when the dynamic ftrace is located at
    the very beginning of the function.

    The function need to be redirected before the stack or the function
    parameters are modified in any way. For example, livepatch requires
    using -fentry gcc compiler option on x86_64.

    One exception is the PPC port. It uses relative addressing and TOC.
    Each function has to handle TOC and save LR before it could call
    the ftrace handler. This operation has to be reverted on return.
    Fortunately, the generic ftrace code has the same problem and all
    this is handled on the ftrace level.


  - Kretprobes using the ftrace framework conflict with the patched
    functions.

    Both kretprobes and livepatches use a ftrace handler that modifies
    the return address. The first user wins. Either the probe or the patch
    is rejected when the handler is already in use by the other.


  - Kprobes in the original function are ignored when the code is
    redirected to the new implementation.

    There is a work in progress to add warnings about this situation.