The x86 KVM shadow MMU

MMU 책임과 설계 요구사항

`arch/x86/kvm`의 `mmu.[ch]`와 `paging_tmpl.h`는 guest에 표준 x86 MMU를 제시하면서 guest physical address를 host physical address로 변환합니다.

.. SPDX-License-Identifier: GPL-2.0

======================
The x86 kvm shadow mmu
======================

The mmu (in arch/x86/kvm, files mmu.[ch] and paging_tmpl.h) is responsible
for presenting a standard x86 mmu to the guest, while translating guest
physical addresses to host physical addresses.

The mmu code attempts to satisfy the following requirements:

- correctness:
	       the guest should not be able to determine that it is running
               on an emulated mmu except for timing (we attempt to comply
               with the specification, not emulate the characteristics of
               a particular implementation such as tlb size)
- security:
	       the guest must not be able to touch host memory not assigned
               to it
- performance:
               minimize the performance penalty imposed by the mmu
- scaling:
               need to scale to large memory and large vcpu guests
- hardware:
               support the full range of x86 virtualization hardware
- integration:
               Linux memory management code must be in control of guest memory
               so that swapping, page migration, page merging, transparent
               hugepages, and similar features work without change
- dirty tracking:
               report writes to guest memory to enable live migration
               and framebuffer-based displays
- footprint:
               keep the amount of pinned kernel memory low (most memory
               should be shrinkable)
- reliability:
               avoid multipage or GFP_ATOMIC allocations

Acronyms

주소와 page-table 용어

========

====  ====================================================================
pfn   host page frame number
hpa   host physical address
hva   host virtual address
gfn   guest frame number
gpa   guest physical address
gva   guest virtual address
ngpa  nested guest physical address
ngva  nested guest virtual address
pte   page table entry (used also to refer generically to paging structure
      entries)
gpte  guest pte (referring to gfns)
spte  shadow pte (referring to pfns)
tdp   two dimensional paging (vendor neutral term for NPT and EPT)
====  ====================================================================

Virtual and real hardware supported

지원하는 가상·실제 hardware

MMU는 guest 진입 시 paging mode와 CR3를 원자적으로 전환하는 1세대 hardware 및 AMD NPT·Intel EPT의 TDP를 지원합니다.

guest에는 global page, PAE, PSE, PSE36, `CR0.WP`, 1GB page를 갖춘 전통적인 2·3·4-level x86 MMU를 에뮬레이션합니다. NPT 가능 host에서는 NPT 가능 가상 hardware도 노출할 수 있습니다.

===================================

The mmu supports first-generation mmu hardware, which allows an atomic switch
of the current paging mode and cr3 during guest entry, as well as
two-dimensional paging (AMD's NPT and Intel's EPT).  The emulated hardware
it exposes is the traditional 2/3/4 level x86 mmu, with support for global
pages, pae, pse, pse36, cr0.wp, and 1GB pages. Emulated hardware also
able to expose NPT capable hardware on NPT capable hosts.

Translation

주소 변환 조합

hardware는 traditional paging에서 1단계, TDP에서 2단계만 encode할 수 있는데 KVM은 최대 3단계를 표현해야 합니다. 필요한 변환 수가 hardware와 맞으면 direct mode, 맞지 않으면 shadow mode로 동작합니다.

===========

The primary job of the mmu is to program the processor's mmu to translate
addresses for the guest.  Different translations are required at different
times:

- when guest paging is disabled, we translate guest physical addresses to
  host physical addresses (gpa->hpa)
- when guest paging is enabled, we translate guest virtual addresses, to
  guest physical addresses, to host physical addresses (gva->gpa->hpa)
- when the guest launches a guest of its own, we translate nested guest
  virtual addresses, to nested guest physical addresses, to guest physical
  addresses, to host physical addresses (ngva->ngpa->gpa->hpa)

The primary challenge is to encode between 1 and 3 translations into hardware
that support only 1 (traditional) and 2 (tdp) translations.  When the
number of required translations matches the hardware, the mmu operates in
direct mode; otherwise it operates in shadow mode (see below).

Memory

guest memory backing

GPA memory는 KVM을 사용하는 process의 user address space 일부입니다. userspace가 GPA에서 HVA로의 변환을 정의합니다. 서로 다른 두 GPA가 같은 HVA를 alias할 수 있지만 하나의 GPA가 여러 HVA를 가리킬 수는 없습니다.

HVA는 anonymous, file-backed, device memory 등 host가 지원하는 방식으로 backing할 수 있고 host는 언제든 page out할 수 있습니다.

======

Guest memory (gpa) is part of the user address space of the process that is
using kvm.  Userspace defines the translation between guest addresses and user
addresses (gpa->hva); note that two gpas may alias to the same hva, but not
vice versa.

These hvas may be backed using any method available to the host: anonymous
memory, file backed memory, and device memory.  Memory might be paged by the
host at any time.

Events

MMU를 구동하는 event

======

The mmu is driven by events, some from the guest, some from the host.

Guest generated events:

- writes to control registers (especially cr3)
- invlpg/invlpga instruction execution
- access to missing or protected translations

Host generated events:

- changes in the gpa->hpa translation (either through gpa->hva changes or
  through hva->hpa changes)
- memory pressure (the shrinker)

Shadow pages

shadow page 구조와 role

핵심 자료구조 `struct kvm_mmu_page`는 leaf 또는 nonleaf SPTE 512개를 담고 한 page에서 둘을 섞을 수 있습니다. nonleaf SPTE는 다른 shadow page를 가리켜 hardware MMU가 leaf에 도달하게 하며 직접 translation은 나타내지 않습니다. leaf SPTE는 하나 또는 두 주소 변환을 한 entry에 encode하고 guest page를 가리킵니다.

`role.direct`일 때 linear range는 `gfn << PAGE_SHIFT`에서 시작하며 level에 따라 2MB, 1GB, 0.5TB, 256TB 크기입니다. 32-bit GPTE guest를 64-bit SPTE host가 shadow하면 level에 따라 quadrant 0-1 또는 0-3을 사용하고 각 quadrant는 1GB virtual address space를 매핑합니다.

`spt` 배열은 shadow page가 node, guest page가 leaf인 DAG를 이룹니다. `sp1->spt[n]`이 `sp2->spt`의 physical address를 가리킬 수 있고 `sp2`는 `parent_pte`로 `sp1`을 역참조합니다.

MMU generation mismatch는 모든 기존 shadow MMU page를 빠르게 obsolete로 만들어 vCPU가 guest 재진입 전에 새 root를 load하게 합니다. TDP MMU는 non-root page가 owning root에서만 reachable하므로 root의 `role.invalid`만으로 전체를 무효화합니다.

`write_flooding_count`는 guest가 page table을 반복 write해 emulation을 과도하게 만드는 경우를 다룹니다. leaf는 unsync할 수 있지만 nonleaf는 불가능하므로 마지막 실제 사용 뒤 emulation 횟수가 너무 많으면 page를 unmap해 이후 비용을 줄입니다.

============

The principal data structure is the shadow page, 'struct kvm_mmu_page'.  A
shadow page contains 512 sptes, which can be either leaf or nonleaf sptes.  A
shadow page may contain a mix of leaf and nonleaf sptes.

A nonleaf spte allows the hardware mmu to reach the leaf pages and
is not related to a translation directly.  It points to other shadow pages.

A leaf spte corresponds to either one or two translations encoded into
one paging structure entry.  These are always the lowest level of the
translation stack, with optional higher level translations left to NPT/EPT.
Leaf ptes point at guest pages.

The following table shows translations encoded by leaf ptes, with higher-level
translations in parentheses:

 Non-nested guests::

  nonpaging:     gpa->hpa
  paging:        gva->gpa->hpa
  paging, tdp:   (gva->)gpa->hpa

 Nested guests::

  non-tdp:       ngva->gpa->hpa  (*)
  tdp:           (ngva->)ngpa->gpa->hpa

  (*) the guest hypervisor will encode the ngva->gpa translation into its page
      tables if npt is not present

Shadow pages contain the following information:
  role.level:
    The level in the shadow paging hierarchy that this shadow page belongs to.
    1=4k sptes, 2=2M sptes, 3=1G sptes, etc.
  role.direct:
    If set, leaf sptes reachable from this page are for a linear range.
    Examples include real mode translation, large guest pages backed by small
    host pages, and gpa->hpa translations when NPT or EPT is active.
    The linear range starts at (gfn << PAGE_SHIFT) and its size is determined
    by role.level (2MB for first level, 1GB for second level, 0.5TB for third
    level, 256TB for fourth level)
    If clear, this page corresponds to a guest page table denoted by the gfn
    field.
  role.quadrant:
    When role.has_4_byte_gpte=1, the guest uses 32-bit gptes while the host uses 64-bit
    sptes.  That means a guest page table contains more ptes than the host,
    so multiple shadow pages are needed to shadow one guest page.
    For first-level shadow pages, role.quadrant can be 0 or 1 and denotes the
    first or second 512-gpte block in the guest page table.  For second-level
    page tables, each 32-bit gpte is converted to two 64-bit sptes
    (since each first-level guest page is shadowed by two first-level
    shadow pages) so role.quadrant takes values in the range 0..3.  Each
    quadrant maps 1GB virtual address space.
  role.access:
    Inherited guest access permissions from the parent ptes in the form uwx.
    Note execute permission is positive, not negative.
  role.invalid:
    The page is invalid and should not be used.  It is a root page that is
    currently pinned (by a cpu hardware register pointing to it); once it is
    unpinned it will be destroyed.
  role.has_4_byte_gpte:
    Reflects the size of the guest PTE for which the page is valid, i.e. '0'
    if direct map or 64-bit gptes are in use, '1' if 32-bit gptes are in use.
  role.efer_nx:
    Contains the value of efer.nx for which the page is valid.
  role.cr0_wp:
    Contains the value of cr0.wp for which the page is valid.
  role.smep_andnot_wp:
    Contains the value of cr4.smep && !cr0.wp for which the page is valid
    (pages for which this is true are different from other pages; see the
    treatment of cr0.wp=0 below).
  role.smap_andnot_wp:
    Contains the value of cr4.smap && !cr0.wp for which the page is valid
    (pages for which this is true are different from other pages; see the
    treatment of cr0.wp=0 below).
  role.smm:
    Is 1 if the page is valid in system management mode.  This field
    determines which of the kvm_memslots array was used to build this
    shadow page; it is also used to go back from a struct kvm_mmu_page
    to a memslot, through the kvm_memslots_for_spte_role macro and
    __gfn_to_memslot.
  role.ad_disabled:
    Is 1 if the MMU instance cannot use A/D bits.  EPT did not have A/D
    bits before Haswell; shadow EPT page tables also cannot use A/D bits
    if the L1 hypervisor does not enable them.
  role.guest_mode:
    Indicates the shadow page is created for a nested guest.
  role.passthrough:
    The page is not backed by a guest page table, but its first entry
    points to one.  This is set if NPT uses 5-level page tables (host
    CR4.LA57=1) and is shadowing L1's 4-level NPT (L1 CR4.LA57=0).
  mmu_valid_gen:
    The MMU generation of this page, used to fast zap of all MMU pages within a
    VM without blocking vCPUs too long. Specifically, KVM updates the per-VM
    valid MMU generation which causes the mismatch of mmu_valid_gen for each mmu
    page. This makes all existing MMU pages obsolete. Obsolete pages can't be
    used. Therefore, vCPUs must load a new, valid root before re-entering the
    guest. The MMU generation is only ever '0' or '1'. Note, the TDP MMU doesn't
    use this field as non-root TDP MMU pages are reachable only from their
    owning root. Thus it suffices for TDP MMU to use role.invalid in root pages
    to invalidate all MMU pages.
  gfn:
    Either the guest page table containing the translations shadowed by this
    page, or the base page frame for linear translations.  See role.direct.
  spt:
    A pageful of 64-bit sptes containing the translations for this page.
    Accessed by both kvm and hardware.
    The page pointed to by spt will have its page->private pointing back
    at the shadow page structure.
    sptes in spt point either at guest pages, or at lower-level shadow pages.
    Specifically, if sp1 and sp2 are shadow pages, then sp1->spt[n] may point
    at __pa(sp2->spt).  sp2 will point back at sp1 through parent_pte.
    The spt array forms a DAG structure with the shadow page as a node, and
    guest pages as leaves.
  shadowed_translation:
    An array of 512 shadow translation entries, one for each present pte. Used
    to perform a reverse map from a pte to a gfn as well as its access
    permission. When role.direct is set, the shadow_translation array is not
    allocated. This is because the gfn contained in any element of this array
    can be calculated from the gfn field when used.  In addition, when
    role.direct is set, KVM does not track access permission for each of the
    gfn. See role.direct and gfn.
  root_count / tdp_mmu_root_count:
     root_count is a reference counter for root shadow pages in Shadow MMU.
     vCPUs elevate the refcount when getting a shadow page that will be used as
     a root page, i.e. page that will be loaded into hardware directly (CR3,
     PDPTRs, nCR3 EPTP). Root pages cannot be destroyed while their refcount is
     non-zero. See role.invalid. tdp_mmu_root_count is similar but exclusively
     used in TDP MMU as an atomic refcount.
  parent_ptes:
    The reverse mapping for the pte/ptes pointing at this page's spt. If
    parent_ptes bit 0 is zero, only one spte points at this page and
    parent_ptes points at this single spte, otherwise, there exists multiple
    sptes pointing at this page and (parent_ptes & ~0x1) points at a data
    structure with a list of parent sptes.
  ptep:
    The kernel virtual address of the SPTE that points at this shadow page.
    Used exclusively by the TDP MMU, this field is a union with parent_ptes.
  unsync:
    If true, then the translations in this page may not match the guest's
    translation.  This is equivalent to the state of the tlb when a pte is
    changed but before the tlb entry is flushed.  Accordingly, unsync ptes
    are synchronized when the guest executes invlpg or flushes its tlb by
    other means.  Valid for leaf pages.
  unsync_children:
    How many sptes in the page point at pages that are unsync (or have
    unsynchronized children).
  unsync_child_bitmap:
    A bitmap indicating which sptes in spt point (directly or indirectly) at
    pages that may be unsynchronized.  Used to quickly locate all unsynchronized
    pages reachable from a given page.
  clear_spte_count:
    Only present on 32-bit hosts, where a 64-bit spte cannot be written
    atomically.  The reader uses this while running out of the MMU lock
    to detect in-progress updates and retry them until the writer has
    finished the write.
  write_flooding_count:
    A guest may write to a page table many times, causing a lot of
    emulations if the page needs to be write-protected (see "Synchronized
    and unsynchronized pages" below).  Leaf pages can be unsynchronized
    so that they do not trigger frequent emulation, but this is not
    possible for non-leafs.  This field counts the number of emulations
    since the last time the page table was actually used; if emulation
    is triggered too frequently on this page, KVM will unmap the page
    to avoid emulation in the future.
  tdp_mmu_page:
    Is 1 if the shadow page is a TDP MMU page. This variable is used to
    bifurcate the control flows for KVM when walking any data structure that
    may contain pages from both TDP MMU and shadow MMU.

reverse map

MMU는 GFN에서 그 page를 매핑하는 모든 PTE에 도달하는 reverse mapping을 유지합니다. host가 page를 swap out할 때처럼 해당 translation을 찾아 제거하거나 갱신하는 데 사용합니다.

Reverse map
===========

The mmu maintains a reverse mapping whereby all ptes mapping a page can be
reached given its gfn.  This is used, for example, when swapping out a page.

Synchronized and unsynchronized pages

synchronized와 unsynchronized page

guest는 TLB flush와 `invlpg` page invalidation으로 TLB와 page table을 동기화합니다. TLB flush는 guest CR3에서 reachable한 모든 SPTE를 동기화해야 하므로 비쌉니다. 일반적으로 guest page table을 write-protect하고 GPTE write 때 SPTE를 맞춥니다.

현재 guest CR3에서 reachable한 guest page table은 guest가 translation 사용 전에 INVLPG를 실행할 의무가 있습니다. KVM은 이를 이용해 write protection을 제거하고 guest가 여러 GPTE를 자유롭게 바꾸게 한 뒤 INVLPG 시 수정분을 동기화합니다.

이 최적화는 여러 GPTE 수정과 page-table page를 일반 data로 재사용할 때 emulation을 줄이지만 TLB flush에서는 reachable한 모든 unsynchronized shadow page를 다시 동기화해야 합니다.

=====================================

The guest uses two events to synchronize its tlb and page tables: tlb flushes
and page invalidations (invlpg).

A tlb flush means that we need to synchronize all sptes reachable from the
guest's cr3.  This is expensive, so we keep all guest page tables write
protected, and synchronize sptes to gptes when a gpte is written.

A special case is when a guest page table is reachable from the current
guest cr3.  In this case, the guest is obliged to issue an invlpg instruction
before using the translation.  We take advantage of that by removing write
protection from the guest page, and allowing the guest to modify it freely.
We synchronize modified gptes when the guest invokes invlpg.  This reduces
the amount of emulation we have to do when the guest modifies multiple gptes,
or when the a guest page is no longer used as a page table and is used for
random guest data.

As a side effect we have to resynchronize all reachable unsynchronized shadow
pages on a tlb flush.


Reaction to events

page fault와 갱신 event 처리

guest page fault, NPT fault, EPT violation은 true guest fault, missing translation, dirty logging 또는 synchronized page의 write protection, MMIO처럼 변환 불가능한 memory 때문에 발생할 수 있습니다. true fault와 synchronized-page 보호는 direct mode에는 적용되지 않습니다.

MMIO 정보는 `vcpu->arch.mmio_gva`, `mmio_access`, `mmio_gfn`에 cache하며 SPTE cache mask는 `kvm_mmu_set_mmio_spte_mask` 호출 경로를 따릅니다. fast page fault의 locking 규칙은 `Documentation/virt/kvm/locking.rst`에 있습니다.

==================

- guest page fault (or npt page fault, or ept violation)

This is the most complicated event.  The cause of a page fault can be:

  - a true guest fault (the guest translation won't allow the access) (*)
  - access to a missing translation
  - access to a protected translation
    - when logging dirty pages, memory is write protected
    - synchronized shadow pages are write protected (*)
  - access to untranslatable memory (mmio)

  (*) not applicable in direct mode

Handling a page fault is performed as follows:

 - if the RSV bit of the error code is set, the page fault is caused by guest
   accessing MMIO and cached MMIO information is available.

   - walk shadow page table
   - check for valid generation number in the spte (see "Fast invalidation of
     MMIO sptes" below)
   - cache the information to vcpu->arch.mmio_gva, vcpu->arch.mmio_access and
     vcpu->arch.mmio_gfn, and call the emulator

 - If both P bit and R/W bit of error code are set, this could possibly
   be handled as a "fast page fault" (fixed without taking the MMU lock).  See
   the description in Documentation/virt/kvm/locking.rst.

 - if needed, walk the guest page tables to determine the guest translation
   (gva->gpa or ngpa->gpa)

   - if permissions are insufficient, reflect the fault back to the guest

 - determine the host page

   - if this is an mmio request, there is no host page; cache the info to
     vcpu->arch.mmio_gva, vcpu->arch.mmio_access and vcpu->arch.mmio_gfn

 - walk the shadow page table to find the spte for the translation,
   instantiating missing intermediate page tables as necessary

   - If this is an mmio request, cache the mmio info to the spte and set some
     reserved bit on the spte (see callers of kvm_mmu_set_mmio_spte_mask)

 - try to unsynchronize the page

   - if successful, we can let the guest continue and modify the gpte

 - emulate the instruction

   - if failed, unshadow the page and let the guest continue

 - update any translations that were modified by the instruction

invlpg handling:

  - walk the shadow page hierarchy and drop affected translations
  - try to reinstantiate the indicated translation in the hope that the
    guest will use it in the near future

Guest control register updates:

- mov to cr3

  - look up new shadow roots
  - synchronize newly reachable shadow pages

- mov to cr0/cr4/efer

  - set up mmu context for new paging mode
  - look up new shadow roots
  - synchronize newly reachable shadow pages

Host translation updates:

  - mmu notifier called with updated hva
  - look up affected sptes through reverse map
  - drop (or update) translations

Emulating cr0.wp

CR0.WP 에뮬레이션

TDP가 없으면 host는 guest kernel write protection을 위해 `CR0.WP=1`을 유지해야 합니다. guest도 WP=1이면 문제가 없지만 guest WP=0에서는 `GPTE.U=1, GPTE.W=0`의 kernel full access와 user read-only 의미를 단일 SPTE permission으로 표현할 수 없습니다.

kernel page로 바꾼 SPTE에서 SMEP가 켜져 있으면 kernel execution을 막기 위해 `SPTE.NX`도 set합니다. user fetch/read fault가 오면 `SPTE.U=1`과 원래 `GPTE.NX`를 복원합니다. 이를 위해 shadow paging에서는 KVM이 `EFER.NX=1`을 강제합니다.

SMAP가 꺼진 상태에서 kernel page로 바꾼 SPTE는 SMAP를 켠 뒤 재사용할 수 없으므로 `CR4.SMAP && !CR0.WP`를 shadow page role에 포함합니다. SMAP가 이미 켜져 권한 검사가 실패하면 KVM은 guest에 직접 #PF를 주입합니다.

WP=0에서 만든 kernel-writable SPTE가 WP=1 뒤 재사용되지 않게 `CR0.WP`도 page role에 포함합니다. 같은 이유로 `!CR0.WP && CR4.SMEP` 값도 role의 일부여야 합니다.

================

If tdp is not enabled, the host must keep cr0.wp=1 so page write protection
works for the guest kernel, not guest userspace.  When the guest
cr0.wp=1, this does not present a problem.  However when the guest cr0.wp=0,
we cannot map the permissions for gpte.u=1, gpte.w=0 to any spte (the
semantics require allowing any guest kernel access plus user read access).

We handle this by mapping the permissions to two possible sptes, depending
on fault type:

- kernel write fault: spte.u=0, spte.w=1 (allows full kernel access,
  disallows user access)
- read fault: spte.u=1, spte.w=0 (allows full read access, disallows kernel
  write access)

(user write faults generate a #PF)

In the first case there are two additional complications:

- if CR4.SMEP is enabled: since we've turned the page into a kernel page,
  the kernel may now execute it.  We handle this by also setting spte.nx.
  If we get a user fetch or read fault, we'll change spte.u=1 and
  spte.nx=gpte.nx back.  For this to work, KVM forces EFER.NX to 1 when
  shadow paging is in use.
- if CR4.SMAP is disabled: since the page has been changed to a kernel
  page, it can not be reused when CR4.SMAP is enabled. We set
  CR4.SMAP && !CR0.WP into shadow page's role to avoid this case. Note,
  here we do not care the case that CR4.SMAP is enabled since KVM will
  directly inject #PF to guest due to failed permission check.

To prevent an spte that was converted into a kernel page with cr0.wp=0
from being written by the kernel after cr0.wp has changed to 1, we make
the value of cr0.wp part of the page role.  This means that an spte created
with one value of cr0.wp cannot be used when cr0.wp has a different value -
it will simply be missed by the shadow page lookup code.  A similar issue
exists when an spte created with cr0.wp=0 and cr4.smep=0 is used after
changing cr4.smep to 1.  To avoid this, the value of !cr0.wp && cr4.smep
is also made a part of the page role.

Large pages

large page

MMU는 guest와 host의 large·small page 모든 조합을 지원하며 4K, 2M, 4M, 1G 크기를 다룹니다. 항상 PAE paging을 쓰므로 4M page는 guest와 host 모두 두 개의 2M page로 취급합니다.

각 memslot과 large page size에 `disallow_lpage` 배열을 유지합니다. write-protected page마다 값을 증가시켜 large SPTE를 막고, 정렬되지 않은 memslot 끝 frame은 인위적으로 값을 높여 절대 large SPTE가 되지 않게 합니다.

===========

The mmu supports all combinations of large and small guest and host pages.
Supported page sizes include 4k, 2M, 4M, and 1G.  4M pages are treated as
two separate 2M pages, on both guest and host, since the mmu always uses PAE
paging.

To instantiate a large spte, four constraints must be satisfied:

- the spte must point to a large host page
- the guest pte must be a large pte of at least equivalent size (if tdp is
  enabled, there is no guest pte and this condition is satisfied)
- if the spte will be writeable, the large page frame may not overlap any
  write-protected pages
- the guest page must be wholly contained by a single memory slot

To check the last two conditions, the mmu maintains a ->disallow_lpage set of
arrays for each memory slot and large page size.  Every write protected page
causes its disallow_lpage to be incremented, thus preventing instantiation of
a large spte.  The frames at the end of an unaligned memory slot have
artificially inflated ->disallow_lpages so they can never be instantiated.

Fast invalidation of MMIO sptes

MMIO SPTE 빠른 무효화

KVM은 leaf SPTE에 MMIO 정보를 cache합니다. memslot 추가·변경으로 cache가 stale해지면 무효화해야 하지만 모든 shadow page를 MMU lock 아래 walk하는 비용을 피하기 위해 generation을 사용합니다.

MMIO SPTE의 spare bit 18개에 generation을 저장하고 전역값은 `kvm_memslots(kvm)->generation`에 둡니다. guest memory 정보가 바뀌면 전역 generation을 증가시키고 SPTE와 다르면 cache를 무시해 slow page-fault path로 처리합니다. 18-bit overflow 시 모든 page를 zap합니다.

한 memory access가 `kvm_memslots()`를 여러 번 읽으면 오래된 정보로 SPTE를 만들면서 마지막에는 새 generation을 저장할 위험이 있습니다. 이를 막기 위해 `synchronize_srcu` 반환 뒤 generation을 다시 증가시킵니다.

memslot update 중에는 generation bit 63이 1인 odd 값이며 일부 SRCU reader가 이전 copy를 사용할 수 있습니다. update-in-progress bit는 MMIO SPTE에 저장하지 않아 추출 시 암묵적으로 0입니다. update 중 생성된 SPTE는 update window 안에서는 in-progress flag 차이로, 이후에는 더 높은 generation 차이로 반드시 cache miss가 됩니다.

===============================

As mentioned in "Reaction to events" above, kvm will cache MMIO
information in leaf sptes.  When a new memslot is added or an existing
memslot is changed, this information may become stale and needs to be
invalidated.  This also needs to hold the MMU lock while walking all
shadow pages, and is made more scalable with a similar technique.

MMIO sptes have a few spare bits, which are used to store a
generation number.  The global generation number is stored in
kvm_memslots(kvm)->generation, and increased whenever guest memory info
changes.

When KVM finds an MMIO spte, it checks the generation number of the spte.
If the generation number of the spte does not equal the global generation
number, it will ignore the cached MMIO information and handle the page
fault through the slow path.

Since only 18 bits are used to store generation-number on mmio spte, all
pages are zapped when there is an overflow.

Unfortunately, a single memory access might access kvm_memslots(kvm) multiple
times, the last one happening when the generation number is retrieved and
stored into the MMIO spte.  Thus, the MMIO spte might be created based on
out-of-date information, but with an up-to-date generation number.

To avoid this, the generation number is incremented again after synchronize_srcu
returns; thus, bit 63 of kvm_memslots(kvm)->generation set to 1 only during a
memslot update, while some SRCU readers might be using the old copy.  We do not
want to use an MMIO sptes created with an odd generation number, and we can do
this without losing a bit in the MMIO spte.  The "update in-progress" bit of the
generation is not stored in MMIO spte, and is so is implicitly zero when the
generation is extracted out of the spte.  If KVM is unlucky and creates an MMIO
spte while an update is in-progress, the next access to the spte will always be
a cache miss.  For example, a subsequent access during the update window will
miss due to the in-progress flag diverging, while an access after the update
window closes will have a higher generation number (as compared to the spte).


Further reading

추가 자료

KVM Forum 2008의 NPT 발표 자료가 shadow MMU와 two-dimensional paging의 추가 배경을 제공합니다.

===============

- NPT presentation from KVM Forum 2008
  https://www.linux-kvm.org/images/c/c8/KvmForum2008%24kdf2008_21.pdf