PPC KVM paravirtual interface

실행 원리

PowerPC KVM은 커널 공간 코드를 사용자 공간을 뜻하는 `PR=1` 상태에서 실행해 모든 특권 명령을 trap하고 에뮬레이션합니다. 그러나 직접 처리할 수 있는 명령까지 하이퍼바이저로 돌아가면 비용이 커집니다.

PPC PV 인터페이스는 하이퍼바이저의 도움으로 특권 명령을 비특권 명령으로 바꾸어 불필요한 trap을 줄입니다. 문서의 벤치마크에서는 가상화 비용이 약 50% 감소했으며 구현은 `arch/powerpc/kernel/kvm*`에 있습니다.

.. SPDX-License-Identifier: GPL-2.0

=================================
The PPC KVM paravirtual interface
=================================

The basic execution principle by which KVM on PowerPC works is to run all kernel
space code in PR=1 which is user space. This way we trap all privileged
instructions and can emulate them accordingly.

Unfortunately that is also the downfall. There are quite some privileged
instructions that needlessly return us to the hypervisor even though they
could be handled differently.

This is what the PPC PV interface helps with. It takes privileged instructions
and transforms them into unprivileged ones with some help from the hypervisor.
This cuts down virtualization costs by about 50% on some of my benchmarks.

The code for that interface can be found in arch/powerpc/kernel/kvm*

Querying for existence

KVM 실행 환경 탐지

게스트는 device tree의 `/hypervisor` 노드를 확인합니다. KVM에서 실행 중이면 이 노드가 존재하고 `compatible` 속성 값이 `linux,kvm`입니다. PV 지원 KVM임을 확인한 뒤에만 하이퍼콜을 사용합니다.

======================

To find out if we're running on KVM or not, we leverage the device tree. When
Linux is running on KVM, a node /hypervisor exists. That node contains a
compatible property with the value "linux,kvm".

Once you determined you're running under a PV capable KVM, you can now use
hypercalls as described below.

KVM hypercalls

하이퍼콜 ABI

`/hypervisor` 노드의 `hypercall-instructions` 속성에는 하이퍼콜을 구성하는 최대 4개 opcode가 들어 있습니다. 게스트는 이 명령을 그대로 호출합니다.

하이퍼콜 정의는 공통 코드에서 공유되므로 x86과 PowerPC가 같은 번호를 사용합니다. PowerPC에서는 각 번호에 KVM vendor code `(42 << 16)`을 OR해야 합니다.

==============

Inside the device tree's /hypervisor node there's a property called
'hypercall-instructions'. This property contains at most 4 opcodes that make
up the hypercall. To call a hypercall, just call these instructions.

The parameters are as follows:

        ========	================	================
	Register	IN			OUT
        ========	================	================
	r0		-			volatile
	r3		1st parameter		Return code
	r4		2nd parameter		1st output value
	r5		3rd parameter		2nd output value
	r6		4th parameter		3rd output value
	r7		5th parameter		4th output value
	r8		6th parameter		5th output value
	r9		7th parameter		6th output value
	r10		8th parameter		7th output value
	r11		hypercall number	8th output value
	r12		-			volatile
        ========	================	================

Hypercall definitions are shared in generic code, so the same hypercall numbers
apply for x86 and powerpc alike with the exception that each KVM hypercall
also needs to be ORed with the KVM vendor code which is (42 << 16).

Return codes can be as follows:

	====		=========================
	Code		Meaning
	====		=========================
	0		Success
	12		Hypercall not implemented
	<0		Error
	====		=========================

The magic page

magic page

magic page는 하이퍼바이저와 게스트가 supervisor-visible 레지스터 상태 일부를 공유하는 페이지입니다. 게스트는 `KVM_HC_PPC_MAP_MAGIC_PAGE` 하이퍼콜로 원하는 위치에 매핑합니다.

첫 인자는 MMU가 켜졌을 때의 effective address이고 두 번째 인자는 대상에 적용되는 real-mode 주소입니다. 현재는 항상 `-4096`에 매핑하므로 `ld rX, -4096(0)`처럼 absolute load/store로 접근할 수 있습니다.

인터페이스는 확장 가능하지만 필드를 추가할 때는 호스트가 새 레지스터를 제공한다는 feature도 새로 정의해야 합니다. 게스트는 해당 feature 지원을 확인한 경우에만 추가 필드를 사용해야 합니다. 레이아웃은 `arch/powerpc/include/uapi/asm/kvm_para.h`의 `struct kvm_vcpu_arch_shared`에 정의됩니다.

==============

To enable communication between the hypervisor and guest there is a new shared
page that contains parts of supervisor visible register state. The guest can
map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE.

With this hypercall issued the guest always gets the magic page mapped at the
desired location. The first parameter indicates the effective address when the
MMU is enabled. The second parameter indicates the address in real mode, if
applicable to the target. For now, we always map the page to -4096. This way we
can access it using absolute load and store functions. The following
instruction reads the first field of the magic page::

	ld	rX, -4096(0)

The interface is designed to be extensible should there be need later to add
additional registers to the magic page. If you add fields to the magic page,
also define a new hypercall feature to indicate that the host can give you more
registers. Only if the host supports the additional features, make use of them.

The magic page layout is described by struct kvm_vcpu_arch_shared
in arch/powerpc/include/uapi/asm/kvm_para.h.

magic page feature

magic page 매핑 하이퍼콜의 두 번째 반환값은 사용 가능한 기능 비트맵입니다. 게스트는 향상된 필드를 사용하기 전에 반드시 해당 비트가 있는지 확인해야 합니다.

Magic page features
===================

When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE,
a second return value is passed to the guest. This second return value contains
a bitmap of available features inside the magic page.

The following enhancements to the magic page are currently available:

  ============================  =======================================
  KVM_MAGIC_FEAT_SR		Maps SR registers r/w in the magic page
  KVM_MAGIC_FEAT_MAS0_TO_SPRG7	Maps MASn, ESR, PIR and high SPRGs
  ============================  =======================================

For enhanced features in the magic page, please check for the existence of the
feature before using them!

magic page flag

feature가 호스트 능력을 게스트에 알리는 채널이라면 flag는 게스트 능력을 호스트에 알리는 채널입니다. flag는 effective address의 하위 12비트로 전달합니다.

Magic page flags
================

In addition to features that indicate whether a host is capable of a particular
feature we also have a channel for a guest to tell the host whether it's capable
of something. This is what we call "flags".

Flags are passed to the host in the low 12 bits of the Effective Address.

The following flags are currently available for a guest to expose:

  MAGIC_PAGE_FLAG_NOT_MAPPED_NX Guest handles NX bits correctly wrt magic page

MSR 비트

MSR에는 하이퍼바이저 개입이 필요한 비트와 게스트 진입 시에만 해석되거나 하이퍼바이저 동작에 영향을 주지 않는 비트가 함께 있습니다. 게스트 안에서 직접 안전하게 설정할 수 있는 비트는 `MSR_EE`와 `MSR_RI`입니다.

이 둘 이외의 MSR 비트를 바꿀 때는 여전히 `mtmsr(d)`를 사용해야 합니다.

MSR bits
========

The MSR contains bits that require hypervisor intervention and bits that do
not require direct hypervisor intervention because they only get interpreted
when entering the guest or don't have any impact on the hypervisor's behavior.

The following bits are safe to be set inside the guest:

  - MSR_EE
  - MSR_RI

If any other bit changes in the MSR, please still use mtmsr(d).

게스트 명령 패치

32비트 시스템에서는 big-endian 배치를 맞추기 위해 `ld`와 `std`를 각각 offset 4를 더한 `lwz`와 `stw`로 바꿉니다. Linux 게스트는 MSR, SPRG, SRR, DAR, DSISR 접근을 magic page load/store로 치환하고 `tlbsync`는 `nop`으로 바꿀 수 있습니다.

이 치환은 선택 사항입니다. 특권 명령 trap도 공유 페이지에 작용하므로 패치하지 않은 기존 명령 역시 계속 동작합니다.

단순 load/store로 처리할 수 없는 명령은 별도 RAM에 실시간 번역 코드를 만듭니다. 예를 들어 EE를 1로 바꿀 때 pending interrupt 확인 같은 추가 로직을 삽입할 수 있습니다.

Patched instructions
====================

The "ld" and "std" instructions are transformed to "lwz" and "stw" instructions
respectively on 32-bit systems with an added offset of 4 to accommodate for big
endianness.

The following is a list of mapping the Linux kernel performs when running as
guest. Implementing any of those mappings is optional, as the instruction traps
also act on the shared page. So calling privileged instructions still works as
before.

======================= ================================
From			To
======================= ================================
mfmsr	rX		ld	rX, magic_page->msr
mfsprg	rX, 0		ld	rX, magic_page->sprg0
mfsprg	rX, 1		ld	rX, magic_page->sprg1
mfsprg	rX, 2		ld	rX, magic_page->sprg2
mfsprg	rX, 3		ld	rX, magic_page->sprg3
mfsrr0	rX		ld	rX, magic_page->srr0
mfsrr1	rX		ld	rX, magic_page->srr1
mfdar	rX		ld	rX, magic_page->dar
mfdsisr	rX		lwz	rX, magic_page->dsisr

mtmsr	rX		std	rX, magic_page->msr
mtsprg	0, rX		std	rX, magic_page->sprg0
mtsprg	1, rX		std	rX, magic_page->sprg1
mtsprg	2, rX		std	rX, magic_page->sprg2
mtsprg	3, rX		std	rX, magic_page->sprg3
mtsrr0	rX		std	rX, magic_page->srr0
mtsrr1	rX		std	rX, magic_page->srr1
mtdar	rX		std	rX, magic_page->dar
mtdsisr	rX		stw	rX, magic_page->dsisr

tlbsync			nop

mtmsrd	rX, 0		b	<special mtmsr section>
mtmsr	rX		b	<special mtmsr section>

mtmsrd	rX, 1		b	<special mtmsrd section>

[Book3S only]
mtsrin	rX, rY		b	<special mtsrin section>

[BookE only]
wrteei	[0|1]		b	<special wrteei section>
======================= ================================

Some instructions require more logic to determine what's going on than a load
or store instruction can deliver. To enable patching of those, we keep some
RAM around where we can live translate instructions to. What happens is the
following:

	1) copy emulation code to memory
	2) patch that code to fit the emulated instruction
	3) patch that code to return to the original pc + 4
	4) patch the original instruction to branch to the new code

That way we can inject an arbitrary amount of code as replacement for a single
instruction. This allows us to check for pending interrupts when setting EE=1
for example.

PowerPC KVM 하이퍼콜 계열

PAPR 하이퍼콜은 일부를 커널이, 나머지를 userspace가 처리합니다. OSI 하이퍼콜은 호환성을 위해 모두 userspace로 전달하며 `book3s_64`에서도 사용할 수 있습니다.

Hypercall ABIs in KVM on PowerPC
=================================

1) KVM hypercalls (ePAPR)

These are ePAPR compliant hypercall implementation (mentioned above). Even
generic hypercalls are implemented here, like the ePAPR idle hcall. These are
available on all targets.

2) PAPR hypercalls

PAPR hypercalls are needed to run server PowerPC PAPR guests (-M pseries in QEMU).
These are the same hypercalls that pHyp, the POWER hypervisor, implements. Some of
them are handled in the kernel, some are handled in user space. This is only
available on book3s_64.

3) OSI hypercalls

Mac-on-Linux is another user of KVM on PowerPC, which has its own hypercall (long
before KVM). This is supported to maintain compatibility. All these hypercalls get
forwarded to user space. This is only useful on book3s_32, but can be used with
book3s_64 as well.