요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
VMBus
=====
VMBus is a software construct provided by Hyper-V to guest VMs. It
consists of a control path and common facilities used by synthetic
devices that Hyper-V presents to guest VMs. The control path is
used to offer synthetic devices to the guest VM and, in some cases,
to rescind those devices. The common facilities include software
channels for communicating between the device driver in the guest VM
and the synthetic device implementation that is part of Hyper-V, and
signaling primitives to allow Hyper-V and the guest to interrupt
each other.
VMBus is modeled in Linux as a bus, with the expected /sys/bus/vmbus
entry in a running Linux guest. The VMBus driver (drivers/hv/vmbus_drv.c)
establishes the VMBus control path with the Hyper-V host, then
registers itself as a Linux bus driver. It implements the standard
bus functions for adding and removing devices to/from the bus.
Most synthetic devices offered by Hyper-V have a corresponding Linux
device driver. These devices include:
* SCSI controller
* NIC
* Graphics frame buffer
* Keyboard
* Mouse
* PCI device pass-thru
* Heartbeat
* Time Sync
* Shutdown
* Memory balloon
* Key/Value Pair (KVP) exchange with Hyper-V
* Hyper-V online backup (a.k.a. VSS)
Guest VMs may have multiple instances of the synthetic SCSI
controller, synthetic NIC, and PCI pass-thru devices. Other
synthetic devices are limited to a single instance per VM. Not
listed above are a small number of synthetic devices offered by
Hyper-V that are used only by Windows guests and for which Linux
does not have a driver.
Hyper-V uses the terms "VSP" and "VSC" in describing synthetic
devices. "VSP" refers to the Hyper-V code that implements a
particular synthetic device, while "VSC" refers to the driver for
the device in the guest VM. For example, the Linux driver for the
synthetic NIC is referred to as "netvsc" and the Linux driver for
the synthetic SCSI controller is "storvsc". These drivers contain
functions with names like "storvsc_connect_to_vsp".
VMBus channels
--------------
An instance of a synthetic device uses VMBus channels to communicate
between the VSP and the VSC. Channels are bi-directional and used
for passing messages. Most synthetic devices use a single channel,
but the synthetic SCSI controller and synthetic NIC may use multiple
channels to achieve higher performance and greater parallelism.
Each channel consists of two ring buffers. These are classic ring
buffers from a university data structures textbook. If the read
and writes pointers are equal, the ring buffer is considered to be
empty, so a full ring buffer always has at least one byte unused.
The "in" ring buffer is for messages from the Hyper-V host to the
guest, and the "out" ring buffer is for messages from the guest to
the Hyper-V host. In Linux, the "in" and "out" designations are as
viewed by the guest side. The ring buffers are memory that is
shared between the guest and the host, and they follow the standard
paradigm where the memory is allocated by the guest, with the list
of GPAs that make up the ring buffer communicated to the host. Each
ring buffer consists of a header page (4 Kbytes) with the read and
write indices and some control flags, followed by the memory for the
actual ring. The size of the ring is determined by the VSC in the
guest and is specific to each synthetic device. The list of GPAs
making up the ring is communicated to the Hyper-V host over the
VMBus control path as a GPA Descriptor List (GPADL). See function
vmbus_establish_gpadl().
Each ring buffer is mapped into contiguous Linux kernel virtual
space in three parts: 1) the 4 Kbyte header page, 2) the memory
that makes up the ring itself, and 3) a second mapping of the memory
that makes up the ring itself. Because (2) and (3) are contiguous
in kernel virtual space, the code that copies data to and from the
ring buffer need not be concerned with ring buffer wrap-around.
Once a copy operation has completed, the read or write index may
need to be reset to point back into the first mapping, but the
actual data copy does not need to be broken into two parts. This
approach also allows complex data structures to be easily accessed
directly in the ring without handling wrap-around.
On arm64 with page sizes > 4 Kbytes, the header page must still be
passed to Hyper-V as a 4 Kbyte area. But the memory for the actual
ring must be aligned to PAGE_SIZE and have a size that is a multiple
of PAGE_SIZE so that the duplicate mapping trick can be done. Hence
a portion of the header page is unused and not communicated to
Hyper-V. This case is handled by vmbus_establish_gpadl().
Hyper-V enforces a limit on the aggregate amount of guest memory
that can be shared with the host via GPADLs. This limit ensures
that a rogue guest can't force the consumption of excessive host
resources. For Windows Server 2019 and later, this limit is
approximately 1280 Mbytes. For versions prior to Windows Server
2019, the limit is approximately 384 Mbytes.
VMBus channel messages
----------------------
All messages sent in a VMBus channel have a standard header that includes
the message length, the offset of the message payload, some flags, and a
transactionID. The portion of the message after the header is
unique to each VSP/VSC pair.
Messages follow one of two patterns:
* Unidirectional: Either side sends a message and does not
expect a response message
* Request/response: One side (usually the guest) sends a message
and expects a response
The transactionID (a.k.a. "requestID") is for matching requests &
responses. Some synthetic devices allow multiple requests to be in-
flight simultaneously, so the guest specifies a transactionID when
sending a request. Hyper-V sends back the same transactionID in the
matching response.
Messages passed between the VSP and VSC are control messages. For
example, a message sent from the storvsc driver might be "execute
this SCSI command". If a message also implies some data transfer
between the guest and the Hyper-V host, the actual data to be
transferred may be embedded with the control message, or it may be
specified as a separate data buffer that the Hyper-V host will
access as a DMA operation. The former case is used when the size of
the data is small and the cost of copying the data to and from the
ring buffer is minimal. For example, time sync messages from the
Hyper-V host to the guest contain the actual time value. When the
data is larger, a separate data buffer is used. In this case, the
control message contains a list of GPAs that describe the data
buffer. For example, the storvsc driver uses this approach to
specify the data buffers to/from which disk I/O is done.
Three functions exist to send VMBus channel messages:
1. vmbus_sendpacket(): Control-only messages and messages with
embedded data -- no GPAs
2. vmbus_sendpacket_pagebuffer(): Message with list of GPAs
identifying data to transfer. An offset and length is
associated with each GPA so that multiple discontinuous areas
of guest memory can be targeted.
3. vmbus_sendpacket_mpb_desc(): Message with list of GPAs
identifying data to transfer. A single offset and length is
associated with a list of GPAs. The GPAs must describe a
single logical area of guest memory to be targeted.
Historically, Linux guests have trusted Hyper-V to send well-formed
and valid messages, and Linux drivers for synthetic devices did not
fully validate messages. With the introduction of processor
technologies that fully encrypt guest memory and that allow the
guest to not trust the hypervisor (AMD SEV-SNP, Intel TDX), trusting
the Hyper-V host is no longer a valid assumption. The drivers for
VMBus synthetic devices are being updated to fully validate any
values read from memory that is shared with Hyper-V, which includes
messages from VMBus devices. To facilitate such validation,
messages read by the guest from the "in" ring buffer are copied to a
temporary buffer that is not shared with Hyper-V. Validation is
performed in this temporary buffer without the risk of Hyper-V
maliciously modifying the message after it is validated but before
it is used.
Synthetic Interrupt Controller (synic)
--------------------------------------
Hyper-V provides each guest CPU with a synthetic interrupt controller
that is used by VMBus for host-guest communication. While each synic
defines 16 synthetic interrupts (SINT), Linux uses only one of the 16
(VMBUS_MESSAGE_SINT). All interrupts related to communication between
the Hyper-V host and a guest CPU use that SINT.
The SINT is mapped to a single per-CPU architectural interrupt (i.e,
an 8-bit x86/x64 interrupt vector, or an arm64 PPI INTID). Because
each CPU in the guest has a synic and may receive VMBus interrupts,
they are best modeled in Linux as per-CPU interrupts. This model works
well on arm64 where a single per-CPU Linux IRQ is allocated for
VMBUS_MESSAGE_SINT. This IRQ appears in /proc/interrupts as an IRQ labelled
"Hyper-V VMbus". Since x86/x64 lacks support for per-CPU IRQs, an x86
interrupt vector is statically allocated (HYPERVISOR_CALLBACK_VECTOR)
across all CPUs and explicitly coded to call vmbus_isr(). In this case,
there's no Linux IRQ, and the interrupts are visible in aggregate in
/proc/interrupts on the "HYP" line.
The synic provides the means to demultiplex the architectural interrupt into
one or more logical interrupts and route the logical interrupt to the proper
VMBus handler in Linux. This demultiplexing is done by vmbus_isr() and
related functions that access synic data structures.
The synic is not modeled in Linux as an irq chip or irq domain,
and the demultiplexed logical interrupts are not Linux IRQs. As such,
they don't appear in /proc/interrupts or /proc/irq. The CPU
affinity for one of these logical interrupts is controlled via an
entry under /sys/bus/vmbus as described below.
VMBus interrupts
----------------
VMBus provides a mechanism for the guest to interrupt the host when
the guest has queued new messages in a ring buffer. The host
expects that the guest will send an interrupt only when an "out"
ring buffer transitions from empty to non-empty. If the guest sends
interrupts at other times, the host deems such interrupts to be
unnecessary. If a guest sends an excessive number of unnecessary
interrupts, the host may throttle that guest by suspending its
execution for a few seconds to prevent a denial-of-service attack.
Similarly, the host will interrupt the guest via the synic when
it sends a new message on the VMBus control path, or when a VMBus
channel "in" ring buffer transitions from empty to non-empty due to
the host inserting a new VMBus channel message. The control message stream
and each VMBus channel "in" ring buffer are separate logical interrupts
that are demultiplexed by vmbus_isr(). It demultiplexes by first checking
for channel interrupts by calling vmbus_chan_sched(), which looks at a synic
bitmap to determine which channels have pending interrupts on this CPU.
If multiple channels have pending interrupts for this CPU, they are
processed sequentially. When all channel interrupts have been processed,
vmbus_isr() checks for and processes any messages received on the VMBus
control path.
The guest CPU that a VMBus channel will interrupt is selected by the
guest when the channel is created, and the host is informed of that
selection. VMBus devices are broadly grouped into two categories:
1. "Slow" devices that need only one VMBus channel. The devices
(such as keyboard, mouse, heartbeat, and timesync) generate
relatively few interrupts. Their VMBus channels are all
assigned to interrupt the VMBUS_CONNECT_CPU, which is always
CPU 0.
2. "High speed" devices that may use multiple VMBus channels for
higher parallelism and performance. These devices include the
synthetic SCSI controller and synthetic NIC. Their VMBus
channels interrupts are assigned to CPUs that are spread out
among the available CPUs in the VM so that interrupts on
multiple channels can be processed in parallel.
The assignment of VMBus channel interrupts to CPUs is done in the
function init_vp_index(). This assignment is done outside of the
normal Linux interrupt affinity mechanism, so the interrupts are
neither "unmanaged" nor "managed" interrupts.
The CPU that a VMBus channel will interrupt can be seen in
/sys/bus/vmbus/devices/<deviceGUID>/ channels/<channelRelID>/cpu.
When running on later versions of Hyper-V, the CPU can be changed
by writing a new value to this sysfs entry. Because VMBus channel
interrupts are not Linux IRQs, there are no entries in /proc/interrupts
or /proc/irq corresponding to individual VMBus channel interrupts.
An online CPU in a Linux guest may not be taken offline if it has
VMBus channel interrupts assigned to it. Starting in kernel v6.15,
any such interrupts are automatically reassigned to some other CPU
at the time of offlining. The "other" CPU is chosen by the
implementation and is not load balanced or otherwise intelligently
determined. If the CPU is onlined again, channel interrupts previously
assigned to it are not moved back. As a result, after multiple CPUs
have been offlined, and perhaps onlined again, the interrupt-to-CPU
mapping may be scrambled and non-optimal. In such a case, optimal
assignments must be re-established manually. For kernels v6.14 and
earlier, any conflicting channel interrupts must first be manually
reassigned to another CPU as described above. Then when no channel
interrupts are assigned to the CPU, it can be taken offline.
The VMBus channel interrupt handling code is designed to work
correctly even if an interrupt is received on a CPU other than the
CPU assigned to the channel. Specifically, the code does not use
CPU-based exclusion for correctness. In normal operation, Hyper-V
will interrupt the assigned CPU. But when the CPU assigned to a
channel is being changed via sysfs, the guest doesn't know exactly
when Hyper-V will make the transition. The code must work correctly
even if there is a time lag before Hyper-V starts interrupting the
new CPU. See comments in target_cpu_store().
VMBus device creation/deletion
------------------------------
Hyper-V and the Linux guest have a separate message-passing path
that is used for synthetic device creation and deletion. This
path does not use a VMBus channel. See vmbus_post_msg() and
vmbus_on_msg_dpc().
The first step is for the guest to connect to the generic
Hyper-V VMBus mechanism. As part of establishing this connection,
the guest and Hyper-V agree on a VMBus protocol version they will
use. This negotiation allows newer Linux kernels to run on older
Hyper-V versions, and vice versa.
The guest then tells Hyper-V to "send offers". Hyper-V sends an
offer message to the guest for each synthetic device that the VM
is configured to have. Each VMBus device type has a fixed GUID
known as the "class ID", and each VMBus device instance is also
identified by a GUID. The offer message from Hyper-V contains
both GUIDs to uniquely (within the VM) identify the device.
There is one offer message for each device instance, so a VM with
two synthetic NICs will get two offers messages with the NIC
class ID. The ordering of offer messages can vary from boot-to-boot
and must not be assumed to be consistent in Linux code. Offer
messages may also arrive long after Linux has initially booted
because Hyper-V supports adding devices, such as synthetic NICs,
to running VMs. A new offer message is processed by
vmbus_process_offer(), which indirectly invokes vmbus_add_channel_work().
Upon receipt of an offer message, the guest identifies the device
type based on the class ID, and invokes the correct driver to set up
the device. Driver/device matching is performed using the standard
Linux mechanism.
The device driver probe function opens the primary VMBus channel to
the corresponding VSP. It allocates guest memory for the channel
ring buffers and shares the ring buffer with the Hyper-V host by
giving the host a list of GPAs for the ring buffer memory. See
vmbus_establish_gpadl().
Once the ring buffer is set up, the device driver and VSP exchange
setup messages via the primary channel. These messages may include
negotiating the device protocol version to be used between the Linux
VSC and the VSP on the Hyper-V host. The setup messages may also
include creating additional VMBus channels, which are somewhat
mis-named as "sub-channels" since they are functionally
equivalent to the primary channel once they are created.
Finally, the device driver may create entries in /dev as with
any device driver.
The Hyper-V host can send a "rescind" message to the guest to
remove a device that was previously offered. Linux drivers must
handle such a rescind message at any time. Rescinding a device
invokes the device driver "remove" function to cleanly shut
down the device and remove it. Once a synthetic device is
rescinded, neither Hyper-V nor Linux retains any state about
its previous existence. Such a device might be re-added later,
in which case it is treated as an entirely new device. See
vmbus_onoffer_rescind().
For some devices, such as the KVP device, Hyper-V automatically
sends a rescind message when the primary channel is closed,
likely as a result of unbinding the device from its driver.
The rescind causes Linux to remove the device. But then Hyper-V
immediately reoffers the device to the guest, causing a new
instance of the device to be created in Linux. For other
devices, such as the synthetic SCSI and NIC devices, closing the
primary channel does *not* result in Hyper-V sending a rescind
message. The device continues to exist in Linux on the VMBus,
but with no driver bound to it. The same driver or a new driver
can subsequently be bound to the existing instance of the device.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
VMBus 모델과 합성 장치
1-50VMBus는 Hyper-V가 guest VM에 제공하는 software construct입니다. Hyper-V가 guest에 제시하는 synthetic device가 공통으로 사용하는 제어 경로와 기반 기능으로 이루어집니다.
제어 경로는 synthetic device를 guest VM에 offer하고, 일부 경우에는 이미 offer한 장치를 rescind하는 데 사용됩니다. 공통 기능에는 guest의 device driver와 Hyper-V 내부의 synthetic device 구현 사이에서 통신하는 software channel, 그리고 Hyper-V와 guest가 서로 interrupt를 일으키는 signaling primitive가 포함됩니다.
Linux는 VMBus를 일반 bus로 모델링하므로 실행 중인 Linux guest에는 `/sys/bus/vmbus`가 나타납니다. VMBus driver인 `drivers/hv/vmbus_drv.c`는 Hyper-V host와 제어 경로를 만든 다음 자신을 Linux bus driver로 등록하고, 장치를 bus에 추가하거나 제거하는 표준 bus function을 구현합니다.
Hyper-V가 제공하는 synthetic device 대부분에는 대응하는 Linux device driver가 있습니다. Synthetic SCSI controller, NIC, graphics frame buffer, keyboard, mouse, PCI device pass-through, heartbeat, time synchronization, shutdown, memory balloon, Hyper-V와의 Key/Value Pair(KVP) 교환, Hyper-V online backup(VSS)이 여기에 포함됩니다.
원문에 열거된 Linux 지원 장치와 서비스입니다.
Guest VM은 synthetic SCSI controller, synthetic NIC, PCI pass-through device를 여러 instance로 가질 수 있습니다. 그 밖의 synthetic device는 VM당 하나로 제한됩니다. Hyper-V가 Windows guest에만 제공하여 Linux driver가 없는 소수의 synthetic device도 있지만 위 목록에는 포함하지 않았습니다.
Hyper-V 용어에서 VSP는 특정 synthetic device를 구현하는 Hyper-V 쪽 code이고, VSC는 guest VM 안의 해당 device driver입니다. 예를 들어 Linux synthetic NIC driver는 `netvsc`, synthetic SCSI controller driver는 `storvsc`라고 부르며 `storvsc_connect_to_vsp` 같은 함수 이름에도 이 구분이 드러납니다.
Synthetic device 양쪽 끝의 역할을 구분합니다.
.. SPDX-License-Identifier: GPL-2.0
VMBus
=====
VMBus is a software construct provided by Hyper-V to guest VMs. It
consists of a control path and common facilities used by synthetic
devices that Hyper-V presents to guest VMs. The control path is
used to offer synthetic devices to the guest VM and, in some cases,
to rescind those devices. The common facilities include software
channels for communicating between the device driver in the guest VM
and the synthetic device implementation that is part of Hyper-V, and
signaling primitives to allow Hyper-V and the guest to interrupt
each other.
VMBus is modeled in Linux as a bus, with the expected /sys/bus/vmbus
entry in a running Linux guest. The VMBus driver (drivers/hv/vmbus_drv.c)
establishes the VMBus control path with the Hyper-V host, then
registers itself as a Linux bus driver. It implements the standard
bus functions for adding and removing devices to/from the bus.
Most synthetic devices offered by Hyper-V have a corresponding Linux
device driver. These devices include:
* SCSI controller
* NIC
* Graphics frame buffer
* Keyboard
* Mouse
* PCI device pass-thru
* Heartbeat
* Time Sync
* Shutdown
* Memory balloon
* Key/Value Pair (KVP) exchange with Hyper-V
* Hyper-V online backup (a.k.a. VSS)
Guest VMs may have multiple instances of the synthetic SCSI
controller, synthetic NIC, and PCI pass-thru devices. Other
synthetic devices are limited to a single instance per VM. Not
listed above are a small number of synthetic devices offered by
Hyper-V that are used only by Windows guests and for which Linux
does not have a driver.
Hyper-V uses the terms "VSP" and "VSC" in describing synthetic
devices. "VSP" refers to the Hyper-V code that implements a
particular synthetic device, while "VSC" refers to the driver for
the device in the guest VM. For example, the Linux driver for the
synthetic NIC is referred to as "netvsc" and the Linux driver for
the synthetic SCSI controller is "storvsc". These drivers contain
functions with names like "storvsc_connect_to_vsp".VMBus channel과 ring buffer
51-103Synthetic device instance는 VSP와 VSC 사이의 통신에 VMBus channel을 사용합니다. Channel은 message를 양방향으로 전달합니다. 대부분의 synthetic device는 channel 하나를 사용하지만 synthetic SCSI controller와 synthetic NIC는 성능과 병렬성을 높이기 위해 여러 channel을 사용할 수 있습니다.
Channel 하나는 두 개의 고전적인 ring buffer로 구성됩니다. Read pointer와 write pointer가 같으면 비어 있다고 판단하므로, 가득 찬 ring buffer에도 최소 1 byte는 사용하지 않은 채 남습니다.
`in` ring buffer는 Hyper-V host에서 guest로 오는 message용이고 `out` ring buffer는 guest에서 Hyper-V host로 가는 message용입니다. Linux의 `in`과 `out` 명칭은 guest 관점입니다.
Ring buffer는 guest와 host가 공유하는 memory입니다. Guest가 memory를 할당하고 ring buffer를 이루는 GPA 목록을 host에 전달합니다. 각 ring buffer에는 read·write index와 제어 flag가 있는 4-Kbyte header page가 먼저 오고 실제 ring memory가 뒤따릅니다. Ring 크기는 guest VSC가 device별로 정합니다.
Ring을 구성하는 GPA 목록은 VMBus 제어 경로에서 GPA Descriptor List(GPADL)로 Hyper-V host에 전달됩니다. 관련 함수는 `vmbus_establish_gpadl()`입니다.
방향 표기는 Linux guest 기준입니다.
Linux kernel virtual address space에서는 각 ring buffer를 세 부분이 연속되도록 mapping합니다. 첫째는 4-Kbyte header page이고, 둘째는 실제 ring memory이며, 셋째는 같은 ring memory를 한 번 더 mapping한 영역입니다.
두 번째와 세 번째 mapping이 연속되므로 data copy code는 ring 끝의 wrap-around를 의식할 필요가 없습니다. Copy가 끝난 뒤 read 또는 write index를 첫 mapping 범위로 되돌려야 할 수는 있지만 실제 copy를 두 조각으로 나눌 필요가 없습니다. 복잡한 data structure도 wrap-around 처리를 하지 않고 ring 안에서 직접 접근할 수 있습니다.
실제 ring memory를 연속해서 두 번 mapping해 wrap-around copy를 단순화합니다.
Linux page size가 4 Kbyte보다 큰 arm64에서도 header page는 Hyper-V에 정확히 4-Kbyte 영역으로 전달해야 합니다. 하지만 중복 mapping을 만들려면 실제 ring memory는 `PAGE_SIZE`에 맞춰 정렬되고 크기도 `PAGE_SIZE`의 배수여야 합니다. 따라서 header page 일부는 사용되지 않고 Hyper-V에도 전달되지 않으며, `vmbus_establish_gpadl()`이 이 경우를 처리합니다.
Hyper-V는 guest가 GPADL로 host와 공유할 수 있는 memory 총량을 제한합니다. 악의적인 guest가 host resource를 과도하게 소비하지 못하게 하기 위한 제한입니다. Windows Server 2019 이상에서는 약 1280 Mbytes이고, 그 이전 version에서는 약 384 Mbytes입니다.
Hyper-V version 계열에 따른 대략적인 guest 공유 memory 총량입니다.
VMBus channels
--------------
An instance of a synthetic device uses VMBus channels to communicate
between the VSP and the VSC. Channels are bi-directional and used
for passing messages. Most synthetic devices use a single channel,
but the synthetic SCSI controller and synthetic NIC may use multiple
channels to achieve higher performance and greater parallelism.
Each channel consists of two ring buffers. These are classic ring
buffers from a university data structures textbook. If the read
and writes pointers are equal, the ring buffer is considered to be
empty, so a full ring buffer always has at least one byte unused.
The "in" ring buffer is for messages from the Hyper-V host to the
guest, and the "out" ring buffer is for messages from the guest to
the Hyper-V host. In Linux, the "in" and "out" designations are as
viewed by the guest side. The ring buffers are memory that is
shared between the guest and the host, and they follow the standard
paradigm where the memory is allocated by the guest, with the list
of GPAs that make up the ring buffer communicated to the host. Each
ring buffer consists of a header page (4 Kbytes) with the read and
write indices and some control flags, followed by the memory for the
actual ring. The size of the ring is determined by the VSC in the
guest and is specific to each synthetic device. The list of GPAs
making up the ring is communicated to the Hyper-V host over the
VMBus control path as a GPA Descriptor List (GPADL). See function
vmbus_establish_gpadl().
Each ring buffer is mapped into contiguous Linux kernel virtual
space in three parts: 1) the 4 Kbyte header page, 2) the memory
that makes up the ring itself, and 3) a second mapping of the memory
that makes up the ring itself. Because (2) and (3) are contiguous
in kernel virtual space, the code that copies data to and from the
ring buffer need not be concerned with ring buffer wrap-around.
Once a copy operation has completed, the read or write index may
need to be reset to point back into the first mapping, but the
actual data copy does not need to be broken into two parts. This
approach also allows complex data structures to be easily accessed
directly in the ring without handling wrap-around.
On arm64 with page sizes > 4 Kbytes, the header page must still be
passed to Hyper-V as a 4 Kbyte area. But the memory for the actual
ring must be aligned to PAGE_SIZE and have a size that is a multiple
of PAGE_SIZE so that the duplicate mapping trick can be done. Hence
a portion of the header page is unused and not communicated to
Hyper-V. This case is handled by vmbus_establish_gpadl().
Hyper-V enforces a limit on the aggregate amount of guest memory
that can be shared with the host via GPADLs. This limit ensures
that a rogue guest can't force the consumption of excessive host
resources. For Windows Server 2019 and later, this limit is
approximately 1280 Mbytes. For versions prior to Windows Server
2019, the limit is approximately 384 Mbytes.Channel message와 기밀 VM 검증
104-166VMBus channel에서 보내는 모든 message에는 message length, payload offset, 여러 flag, `transactionID`를 담은 표준 header가 있습니다. Header 뒤의 내용은 각 VSP/VSC pair마다 고유합니다.
Message pattern은 두 가지입니다. Unidirectional message는 어느 한쪽이 보내고 response를 기대하지 않습니다. Request/response pattern에서는 한쪽, 보통 guest가 request를 보내고 response를 기다립니다.
`transactionID`는 `requestID`라고도 하며 request와 response를 짝짓는 데 사용됩니다. 여러 request를 동시에 in-flight 상태로 둘 수 있는 synthetic device에서는 guest가 request를 보낼 때 transactionID를 지정하고 Hyper-V가 대응 response에 같은 값을 돌려줍니다.
응답 여부와 transactionID의 역할을 비교합니다.
VSP와 VSC 사이의 message는 control message입니다. 예를 들어 `storvsc`가 보내는 message는 특정 SCSI command 실행을 지시할 수 있습니다. Data transfer도 수반한다면 실제 data를 control message 안에 넣거나, Hyper-V host가 DMA operation으로 접근할 별도 data buffer를 지정합니다.
작은 data는 ring buffer copy 비용이 작으므로 message 안에 넣습니다. Host에서 guest로 보내는 time sync message에 실제 time value가 들어가는 것이 예입니다. 큰 data는 별도 buffer를 사용하고 control message에는 그 buffer를 설명하는 GPA 목록을 넣습니다. `storvsc`는 disk I/O를 수행할 data buffer를 이 방식으로 지정합니다.
VMBus channel message를 보내는 함수는 세 가지입니다. `vmbus_sendpacket()`은 GPA가 없는 control-only message 또는 embedded data message를 보냅니다. `vmbus_sendpacket_pagebuffer()`는 각 GPA마다 offset과 length를 붙여 서로 떨어진 guest memory 영역 여러 개를 대상으로 삼습니다. `vmbus_sendpacket_mpb_desc()`는 GPA 목록 전체에 하나의 offset과 length를 붙이며 GPA들은 하나의 논리적 guest memory 영역을 나타내야 합니다.
Payload와 GPA descriptor 형태에 따라 API를 선택합니다.
과거 Linux guest는 Hyper-V가 올바른 message만 보낸다고 신뢰했기 때문에 synthetic device driver가 message를 완전히 검증하지 않았습니다. 그러나 AMD SEV-SNP와 Intel TDX처럼 guest memory를 전면 암호화하고 hypervisor를 신뢰하지 않을 수 있게 하는 기술에서는 이 가정이 더 이상 유효하지 않습니다.
VMBus synthetic device driver는 Hyper-V와 공유하는 memory에서 읽는 모든 값을 완전하게 검증하도록 갱신되고 있으며 VMBus device message도 그 대상입니다. Guest는 `in` ring buffer에서 읽은 message를 Hyper-V와 공유하지 않는 temporary buffer로 복사한 뒤 그곳에서 검증합니다. 이 방식은 검증이 끝난 뒤 사용하기 전에 Hyper-V가 message를 악의적으로 바꾸는 위험을 제거합니다.
공유 memory를 직접 신뢰하지 않고 private temporary buffer에서 검사합니다.
VMBus channel messages
----------------------
All messages sent in a VMBus channel have a standard header that includes
the message length, the offset of the message payload, some flags, and a
transactionID. The portion of the message after the header is
unique to each VSP/VSC pair.
Messages follow one of two patterns:
* Unidirectional: Either side sends a message and does not
expect a response message
* Request/response: One side (usually the guest) sends a message
and expects a response
The transactionID (a.k.a. "requestID") is for matching requests &
responses. Some synthetic devices allow multiple requests to be in-
flight simultaneously, so the guest specifies a transactionID when
sending a request. Hyper-V sends back the same transactionID in the
matching response.
Messages passed between the VSP and VSC are control messages. For
example, a message sent from the storvsc driver might be "execute
this SCSI command". If a message also implies some data transfer
between the guest and the Hyper-V host, the actual data to be
transferred may be embedded with the control message, or it may be
specified as a separate data buffer that the Hyper-V host will
access as a DMA operation. The former case is used when the size of
the data is small and the cost of copying the data to and from the
ring buffer is minimal. For example, time sync messages from the
Hyper-V host to the guest contain the actual time value. When the
data is larger, a separate data buffer is used. In this case, the
control message contains a list of GPAs that describe the data
buffer. For example, the storvsc driver uses this approach to
specify the data buffers to/from which disk I/O is done.
Three functions exist to send VMBus channel messages:
1. vmbus_sendpacket(): Control-only messages and messages with
embedded data -- no GPAs
2. vmbus_sendpacket_pagebuffer(): Message with list of GPAs
identifying data to transfer. An offset and length is
associated with each GPA so that multiple discontinuous areas
of guest memory can be targeted.
3. vmbus_sendpacket_mpb_desc(): Message with list of GPAs
identifying data to transfer. A single offset and length is
associated with a list of GPAs. The GPAs must describe a
single logical area of guest memory to be targeted.
Historically, Linux guests have trusted Hyper-V to send well-formed
and valid messages, and Linux drivers for synthetic devices did not
fully validate messages. With the introduction of processor
technologies that fully encrypt guest memory and that allow the
guest to not trust the hypervisor (AMD SEV-SNP, Intel TDX), trusting
the Hyper-V host is no longer a valid assumption. The drivers for
VMBus synthetic devices are being updated to fully validate any
values read from memory that is shared with Hyper-V, which includes
messages from VMBus devices. To facilitate such validation,
messages read by the guest from the "in" ring buffer are copied to a
temporary buffer that is not shared with Hyper-V. Validation is
performed in this temporary buffer without the risk of Hyper-V
maliciously modifying the message after it is validated but before
it is used.Synthetic Interrupt Controller(SynIC)
167-197Hyper-V는 host와 guest 사이의 VMBus 통신에 쓰이는 synthetic interrupt controller, 즉 SynIC를 guest CPU마다 제공합니다. SynIC 하나는 16개의 synthetic interrupt(SINT)를 정의하지만 Linux는 그중 `VMBUS_MESSAGE_SINT` 하나만 사용합니다. Hyper-V host와 guest CPU 사이의 모든 통신 interrupt가 이 SINT를 통과합니다.
SINT는 CPU별 architecture interrupt 하나에 mapping됩니다. x86/x64에서는 8-bit interrupt vector이고 arm64에서는 PPI INTID입니다. Guest CPU마다 SynIC가 있고 VMBus interrupt를 받을 수 있으므로 Linux에서는 per-CPU interrupt로 모델링하는 것이 가장 알맞습니다.
Arm64에서는 `VMBUS_MESSAGE_SINT`에 per-CPU Linux IRQ 하나를 할당하며 `/proc/interrupts`에 `Hyper-V VMbus` label의 IRQ로 나타납니다.
x86/x64는 per-CPU IRQ를 지원하지 않으므로 모든 CPU에 걸쳐 x86 interrupt vector `HYPERVISOR_CALLBACK_VECTOR`를 정적으로 할당하고 `vmbus_isr()`를 직접 호출하도록 구성합니다. 이 경우 Linux IRQ는 없고 `/proc/interrupts`의 `HYP` 행에 interrupt 합계만 보입니다.
같은 SINT를 architecture별 Linux interrupt 표현으로 연결합니다.
SynIC는 architecture interrupt를 하나 이상의 logical interrupt로 demultiplex하고 올바른 Linux VMBus handler로 route합니다. `vmbus_isr()`와 SynIC data structure에 접근하는 관련 함수가 이 분배를 수행합니다.
Linux는 SynIC를 irq chip이나 irq domain으로 모델링하지 않으며, 분리된 logical interrupt도 Linux IRQ가 아닙니다. 따라서 개별 logical interrupt는 `/proc/interrupts`나 `/proc/irq`에 나타나지 않고 CPU affinity는 뒤에서 설명하는 `/sys/bus/vmbus` 항목으로 제어합니다.
Synthetic Interrupt Controller (synic)
--------------------------------------
Hyper-V provides each guest CPU with a synthetic interrupt controller
that is used by VMBus for host-guest communication. While each synic
defines 16 synthetic interrupts (SINT), Linux uses only one of the 16
(VMBUS_MESSAGE_SINT). All interrupts related to communication between
the Hyper-V host and a guest CPU use that SINT.
The SINT is mapped to a single per-CPU architectural interrupt (i.e,
an 8-bit x86/x64 interrupt vector, or an arm64 PPI INTID). Because
each CPU in the guest has a synic and may receive VMBus interrupts,
they are best modeled in Linux as per-CPU interrupts. This model works
well on arm64 where a single per-CPU Linux IRQ is allocated for
VMBUS_MESSAGE_SINT. This IRQ appears in /proc/interrupts as an IRQ labelled
"Hyper-V VMbus". Since x86/x64 lacks support for per-CPU IRQs, an x86
interrupt vector is statically allocated (HYPERVISOR_CALLBACK_VECTOR)
across all CPUs and explicitly coded to call vmbus_isr(). In this case,
there's no Linux IRQ, and the interrupts are visible in aggregate in
/proc/interrupts on the "HYP" line.
The synic provides the means to demultiplex the architectural interrupt into
one or more logical interrupts and route the logical interrupt to the proper
VMBus handler in Linux. This demultiplexing is done by vmbus_isr() and
related functions that access synic data structures.
The synic is not modeled in Linux as an irq chip or irq domain,
and the demultiplexed logical interrupts are not Linux IRQs. As such,
they don't appear in /proc/interrupts or /proc/irq. The CPU
affinity for one of these logical interrupts is controlled via an
entry under /sys/bus/vmbus as described below.VMBus interrupt 전달과 demultiplex
198-221Guest는 ring buffer에 새 message를 queue했음을 host에 interrupt로 알릴 수 있습니다. Host는 `out` ring buffer가 empty에서 non-empty로 바뀔 때만 guest가 interrupt를 보낼 것으로 기대합니다.
다른 시점의 interrupt는 불필요한 것으로 간주됩니다. Guest가 이런 interrupt를 지나치게 많이 보내면 Hyper-V host는 denial-of-service 공격을 막기 위해 해당 guest의 실행을 몇 초간 정지시켜 throttle할 수 있습니다.
반대 방향에서는 Hyper-V가 VMBus 제어 경로에 새 message를 보내거나, channel의 `in` ring buffer가 host의 새 message 삽입으로 empty에서 non-empty로 바뀔 때 SynIC를 통해 guest에 interrupt를 보냅니다.
제어 message stream과 각 VMBus channel의 `in` ring buffer는 서로 별개의 logical interrupt입니다. `vmbus_isr()`는 먼저 `vmbus_chan_sched()`를 호출하고 SynIC bitmap에서 현재 CPU에 pending인 channel을 찾아 channel interrupt부터 분리합니다. 여러 channel이 pending이면 차례대로 처리하며, 모두 끝난 뒤 VMBus 제어 경로의 수신 message를 확인하고 처리합니다.
SynIC의 한 architecture interrupt를 channel과 control logical interrupt로 나눕니다.
방향별 정상 signaling 조건입니다.
VMBus interrupts
----------------
VMBus provides a mechanism for the guest to interrupt the host when
the guest has queued new messages in a ring buffer. The host
expects that the guest will send an interrupt only when an "out"
ring buffer transitions from empty to non-empty. If the guest sends
interrupts at other times, the host deems such interrupts to be
unnecessary. If a guest sends an excessive number of unnecessary
interrupts, the host may throttle that guest by suspending its
execution for a few seconds to prevent a denial-of-service attack.
Similarly, the host will interrupt the guest via the synic when
it sends a new message on the VMBus control path, or when a VMBus
channel "in" ring buffer transitions from empty to non-empty due to
the host inserting a new VMBus channel message. The control message stream
and each VMBus channel "in" ring buffer are separate logical interrupts
that are demultiplexed by vmbus_isr(). It demultiplexes by first checking
for channel interrupts by calling vmbus_chan_sched(), which looks at a synic
bitmap to determine which channels have pending interrupts on this CPU.
If multiple channels have pending interrupts for this CPU, they are
processed sequentially. When all channel interrupts have been processed,
vmbus_isr() checks for and processes any messages received on the VMBus
control path.Channel interrupt의 CPU 할당
222-275VMBus channel을 만들 때 guest가 그 channel이 interrupt할 guest CPU를 선택하고 host에 알려 줍니다. 장치는 대체로 느린 단일-channel 장치와 고속 multi-channel 장치 두 부류로 나뉩니다.
Keyboard, mouse, heartbeat, timesync처럼 interrupt가 적은 느린 장치는 VMBus channel 하나만 필요합니다. 이 channel들은 언제나 CPU 0인 `VMBUS_CONNECT_CPU`를 interrupt하도록 모두 할당됩니다.
Synthetic SCSI controller와 synthetic NIC 같은 고속 장치는 병렬성과 성능을 위해 여러 VMBus channel을 사용할 수 있습니다. 여러 channel interrupt를 병렬 처리할 수 있도록 VM의 가용 CPU에 나누어 할당합니다.
장치 속성과 channel 수에 따른 기본 배치입니다.
VMBus channel interrupt의 CPU 할당은 `init_vp_index()`에서 수행합니다. 표준 Linux interrupt affinity mechanism 밖에서 이루어지므로 이 interrupt는 `unmanaged`도 `managed`도 아닙니다.
Channel이 interrupt할 CPU는 `/sys/bus/vmbus/devices/<deviceGUID>/channels/<channelRelID>/cpu`에서 확인할 수 있습니다. 비교적 최신 Hyper-V에서는 이 sysfs 항목에 새 값을 써서 CPU를 바꿀 수 있습니다. 개별 VMBus channel interrupt는 Linux IRQ가 아니므로 대응하는 `/proc/interrupts` 또는 `/proc/irq` 항목이 없습니다.
Linux kernel v6.15부터는 VMBus channel interrupt가 할당된 CPU를 offline할 때 해당 interrupt를 자동으로 다른 CPU에 재할당합니다. 새 CPU는 구현이 고를 뿐 load balancing 등 지능적인 기준으로 정하지 않습니다.
CPU를 다시 online해도 전에 그 CPU에 있던 channel interrupt는 돌아오지 않습니다. 여러 CPU를 offline한 뒤 다시 online하면 interrupt-to-CPU mapping이 흐트러져 비최적 상태가 될 수 있으며 최적 배치는 수동으로 복원해야 합니다.
Kernel v6.14 이하에서는 CPU를 offline하기 전에 충돌하는 channel interrupt를 위 sysfs 방식으로 다른 CPU에 수동 재할당해야 합니다. 대상 CPU에 할당된 channel interrupt가 없어져야 offline할 수 있습니다.
Kernel version에 따라 자동 재할당 지원이 다릅니다.
VMBus channel interrupt 처리 code는 channel에 지정되지 않은 CPU에서 interrupt가 들어와도 올바르게 동작하도록 설계됐습니다. Correctness를 위해 CPU 기반 배제를 사용하지 않습니다.
정상 상태에서는 Hyper-V가 지정 CPU를 interrupt하지만 sysfs로 CPU 할당을 바꾸는 동안 guest는 Hyper-V가 정확히 언제 전환할지 알 수 없습니다. 새 CPU로 interrupt를 보내기 시작할 때까지 지연이 있어도 code가 안전해야 합니다. 자세한 내용은 `target_cpu_store()`의 주석을 참조합니다.
The guest CPU that a VMBus channel will interrupt is selected by the
guest when the channel is created, and the host is informed of that
selection. VMBus devices are broadly grouped into two categories:
1. "Slow" devices that need only one VMBus channel. The devices
(such as keyboard, mouse, heartbeat, and timesync) generate
relatively few interrupts. Their VMBus channels are all
assigned to interrupt the VMBUS_CONNECT_CPU, which is always
CPU 0.
2. "High speed" devices that may use multiple VMBus channels for
higher parallelism and performance. These devices include the
synthetic SCSI controller and synthetic NIC. Their VMBus
channels interrupts are assigned to CPUs that are spread out
among the available CPUs in the VM so that interrupts on
multiple channels can be processed in parallel.
The assignment of VMBus channel interrupts to CPUs is done in the
function init_vp_index(). This assignment is done outside of the
normal Linux interrupt affinity mechanism, so the interrupts are
neither "unmanaged" nor "managed" interrupts.
The CPU that a VMBus channel will interrupt can be seen in
/sys/bus/vmbus/devices/<deviceGUID>/ channels/<channelRelID>/cpu.
When running on later versions of Hyper-V, the CPU can be changed
by writing a new value to this sysfs entry. Because VMBus channel
interrupts are not Linux IRQs, there are no entries in /proc/interrupts
or /proc/irq corresponding to individual VMBus channel interrupts.
An online CPU in a Linux guest may not be taken offline if it has
VMBus channel interrupts assigned to it. Starting in kernel v6.15,
any such interrupts are automatically reassigned to some other CPU
at the time of offlining. The "other" CPU is chosen by the
implementation and is not load balanced or otherwise intelligently
determined. If the CPU is onlined again, channel interrupts previously
assigned to it are not moved back. As a result, after multiple CPUs
have been offlined, and perhaps onlined again, the interrupt-to-CPU
mapping may be scrambled and non-optimal. In such a case, optimal
assignments must be re-established manually. For kernels v6.14 and
earlier, any conflicting channel interrupts must first be manually
reassigned to another CPU as described above. Then when no channel
interrupts are assigned to the CPU, it can be taken offline.
The VMBus channel interrupt handling code is designed to work
correctly even if an interrupt is received on a CPU other than the
CPU assigned to the channel. Specifically, the code does not use
CPU-based exclusion for correctness. In normal operation, Hyper-V
will interrupt the assigned CPU. But when the CPU assigned to a
channel is being changed via sysfs, the guest doesn't know exactly
when Hyper-V will make the transition. The code must work correctly
even if there is a time lag before Hyper-V starts interrupting the
new CPU. See comments in target_cpu_store().
VMBus device 생성과 channel 설정
276-324Hyper-V와 Linux guest는 synthetic device 생성·삭제에 별도의 message-passing 경로를 사용하며 이 경로는 VMBus channel을 사용하지 않습니다. 관련 함수는 `vmbus_post_msg()`와 `vmbus_on_msg_dpc()`입니다.
먼저 guest가 일반 Hyper-V VMBus mechanism에 연결합니다. 연결을 만드는 과정에서 guest와 Hyper-V는 사용할 VMBus protocol version에 합의합니다. 이 협상 덕분에 새 Linux kernel이 오래된 Hyper-V에서, 오래된 kernel이 새 Hyper-V에서 동작할 수 있습니다.
그다음 guest가 Hyper-V에 `send offers`를 요청하면 Hyper-V는 VM에 구성된 synthetic device instance마다 offer message 하나를 보냅니다. 각 VMBus device type에는 `class ID`라는 고정 GUID가 있고 각 instance도 GUID로 식별됩니다. Offer에는 두 GUID가 모두 들어가 VM 안에서 device를 고유하게 식별합니다.
Synthetic NIC가 두 개인 VM은 같은 NIC class ID를 가진 offer 두 개를 받습니다. Offer 순서는 boot할 때마다 달라질 수 있으므로 Linux code가 일정한 순서를 가정해서는 안 됩니다. Hyper-V는 실행 중인 VM에도 synthetic NIC 같은 device를 추가할 수 있어 initial boot보다 훨씬 뒤에 offer가 올 수도 있습니다.
새 offer는 `vmbus_process_offer()`가 처리하고 이 함수는 간접적으로 `vmbus_add_channel_work()`를 호출합니다. Guest는 class ID로 device type을 식별하고 표준 Linux driver/device matching mechanism으로 올바른 driver의 설정 과정을 시작합니다.
Device driver의 probe 함수는 대응 VSP로 향하는 primary VMBus channel을 엽니다. Guest memory에 channel ring buffer를 할당하고 그 memory의 GPA 목록을 host에 제공해 Hyper-V와 공유합니다. 이때 `vmbus_establish_gpadl()`을 사용합니다.
Ring buffer가 준비되면 device driver와 VSP가 primary channel에서 설정 message를 교환합니다. Linux VSC와 Hyper-V host의 VSP가 사용할 device protocol version을 협상할 수 있고 추가 VMBus channel을 만들 수도 있습니다.
추가 channel은 관례상 `sub-channel`이라고 부르지만 생성된 뒤에는 기능적으로 primary channel과 동등하므로 이름이 다소 부정확합니다. 마지막으로 일반 device driver와 마찬가지로 driver가 `/dev` entry를 만들 수 있습니다.
VMBus 연결에서 Linux device node까지의 수명 주기입니다.
Bus 연결 협상과 개별 장치 협상은 서로 다른 계층입니다.
VMBus device creation/deletion
------------------------------
Hyper-V and the Linux guest have a separate message-passing path
that is used for synthetic device creation and deletion. This
path does not use a VMBus channel. See vmbus_post_msg() and
vmbus_on_msg_dpc().
The first step is for the guest to connect to the generic
Hyper-V VMBus mechanism. As part of establishing this connection,
the guest and Hyper-V agree on a VMBus protocol version they will
use. This negotiation allows newer Linux kernels to run on older
Hyper-V versions, and vice versa.
The guest then tells Hyper-V to "send offers". Hyper-V sends an
offer message to the guest for each synthetic device that the VM
is configured to have. Each VMBus device type has a fixed GUID
known as the "class ID", and each VMBus device instance is also
identified by a GUID. The offer message from Hyper-V contains
both GUIDs to uniquely (within the VM) identify the device.
There is one offer message for each device instance, so a VM with
two synthetic NICs will get two offers messages with the NIC
class ID. The ordering of offer messages can vary from boot-to-boot
and must not be assumed to be consistent in Linux code. Offer
messages may also arrive long after Linux has initially booted
because Hyper-V supports adding devices, such as synthetic NICs,
to running VMs. A new offer message is processed by
vmbus_process_offer(), which indirectly invokes vmbus_add_channel_work().
Upon receipt of an offer message, the guest identifies the device
type based on the class ID, and invokes the correct driver to set up
the device. Driver/device matching is performed using the standard
Linux mechanism.
The device driver probe function opens the primary VMBus channel to
the corresponding VSP. It allocates guest memory for the channel
ring buffers and shares the ring buffer with the Hyper-V host by
giving the host a list of GPAs for the ring buffer memory. See
vmbus_establish_gpadl().
Once the ring buffer is set up, the device driver and VSP exchange
setup messages via the primary channel. These messages may include
negotiating the device protocol version to be used between the Linux
VSC and the VSP on the Hyper-V host. The setup messages may also
include creating additional VMBus channels, which are somewhat
mis-named as "sub-channels" since they are functionally
equivalent to the primary channel once they are created.
Finally, the device driver may create entries in /dev as with
any device driver.Device rescind와 재제공
325-346Hyper-V host는 이전에 offer한 device를 제거하기 위해 언제든 guest에 `rescind` message를 보낼 수 있습니다. Linux driver는 어느 시점의 rescind도 처리해야 합니다.
Rescind는 device driver의 `remove` 함수를 호출해 device를 정상적으로 종료하고 제거합니다. 제거된 뒤 Hyper-V와 Linux 어느 쪽도 이전 존재에 관한 상태를 보존하지 않습니다. 나중에 같은 종류의 device가 다시 추가되더라도 완전히 새로운 device로 취급합니다. 관련 함수는 `vmbus_onoffer_rescind()`입니다.
KVP device 같은 일부 장치는 driver unbind 등으로 primary channel을 닫으면 Hyper-V가 자동으로 rescind를 보냅니다. Linux가 device를 제거하면 Hyper-V가 즉시 다시 offer하여 Linux에 새 instance가 만들어집니다.
Synthetic SCSI와 NIC 같은 다른 장치는 primary channel을 닫아도 Hyper-V가 rescind를 보내지 않습니다. Device는 driver가 bind되지 않은 상태로 Linux VMBus에 계속 존재하며, 나중에 같은 driver나 새 driver를 기존 instance에 다시 bind할 수 있습니다.
Device 종류에 따라 rescind와 instance 수명이 달라집니다.
Offer된 device를 완전히 제거하는 경로입니다.
The Hyper-V host can send a "rescind" message to the guest to
remove a device that was previously offered. Linux drivers must
handle such a rescind message at any time. Rescinding a device
invokes the device driver "remove" function to cleanly shut
down the device and remove it. Once a synthetic device is
rescinded, neither Hyper-V nor Linux retains any state about
its previous existence. Such a device might be re-added later,
in which case it is treated as an entirely new device. See
vmbus_onoffer_rescind().
For some devices, such as the KVP device, Hyper-V automatically
sends a rescind message when the primary channel is closed,
likely as a result of unbinding the device from its driver.
The rescind causes Linux to remove the device. But then Hyper-V
immediately reoffers the device to the guest, causing a new
instance of the device to be created in Linux. For other
devices, such as the synthetic SCSI and NIC devices, closing the
primary channel does *not* result in Hyper-V sending a rescind
message. The device continues to exist in Linux on the VMBus,
but with no driver bound to it. The same driver or a new driver
can subsequently be bound to the existing instance of the device.
요약·해설
vmbus.rst:1-346VMBus는 Hyper-V synthetic device를 Linux bus model에 연결하는 제어·data plane입니다. Channel마다 host→guest와 guest→host ring을 두고 GPADL로 guest memory를 공유하며, 같은 ring memory를 두 번 연속 mapping해 wrap-around copy를 단순화합니다.
Message 계층에서는 transactionID로 request와 response를 연결하고 data 크기에 따라 embedded payload 또는 GPA 기반 buffer를 선택합니다. SEV-SNP·TDX 같은 기밀 VM에서는 shared ring의 값을 신뢰하지 않고 private temporary buffer로 복사해 검증해야 합니다.
Interrupt는 CPU별 SynIC의 `VMBUS_MESSAGE_SINT`로 들어오지만 개별 channel logical interrupt는 Linux IRQ가 아닙니다. CPU affinity는 VMBus sysfs에서 관리하며 v6.15부터 CPU offline 때 자동 재할당되지만 최적 배치를 보장하거나 online 뒤 원복하지는 않습니다.
Device 수명 주기는 VMBus protocol 협상, offer의 class·instance GUID, primary channel과 GPADL 설정, 선택적 sub-channel 생성, rescind로 이어집니다. Offer 순서는 안정적이지 않으며 device별 primary-channel close 동작도 다르므로 driver는 GUID와 message 기반 상태 전이를 따라야 합니다.