Nova-core Task List

Task list 형식

이 문서는 `nova-core` 개발에 필요한 task list입니다. 원문 SPDX 라이선스는 `(GPL-2.0+ OR MIT)`입니다.

각 task의 `Complexity`는 필요한 Rust 지식 또는 관련 kernel API·subsystem 친숙도를 나타냅니다. 단계는 `Beginner`, `Intermediate`, `Advanced`, `Expert` 네 가지입니다.

`Reference`는 다른 task를 가리키고, `Link`는 외부 자료, `Contact`는 추가 정보를 문의할 담당자를 나타냅니다.

Task 이름 뒤의 `[ABCD]` code는 source tree에서 해당 task와 관련된 `TODO` entry를 grep할 때 사용하는 식별자입니다.

.. SPDX-License-Identifier: (GPL-2.0+ OR MIT)

=========
Task List
=========

Tasks may have the following fields:

- ``Complexity``: Describes the required familiarity with Rust and / or the
  corresponding kernel APIs or subsystems. There are four different complexities,
  ``Beginner``, ``Intermediate``, ``Advanced`` and ``Expert``.
- ``Reference``: References to other tasks.
- ``Link``: Links to external resources.
- ``Contact``: The person that can be contacted for further information about
  the task.

A task might have `[ABCD]` code after its name. This code can be used to grep
into the code for `TODO` entries related to it.

Rust enablement: conversion·allocation·register

`Enablement (Rust)`는 `nova-core` 자체 기능은 아니지만 필요한 kernel Rust API를 마련하는 선행 작업입니다.

`FromPrimitive API [FPRI]`는 number를 enum 또는 structure 값으로 변환하는 일반 API를 요구합니다. 예를 들어 GPU probe에서 register 값 `0x192`를 읽으면 `Chipset::AD102`를 얻어야 합니다. 현재 `Chipset::from_u32`라는 자체 구현을 사용하지만, num crate의 `FromPrimitive` trait 같은 일반화가 바람직합니다. 이 일반화는 number와 value 사이 mapping을 자동 생성하는 generic macro에도 도움이 됩니다. 난이도는 Beginner입니다.

`Conversion from byte slices for types implementing FromBytes [TRSM]`는 BIOS나 loaded firmware의 byte stream에서 structure를 복원하는 문제입니다. 현재 byte slice를 올바른 type으로 바꾸려면 세련되지 않은 `unsafe` operation이 필요합니다. `FromBytes`가 적절한 `from_bytes` method를 제공하면 이를 제거할 수 있습니다. 난이도는 Beginner입니다.

`CoherentAllocation improvements [COHA]`는 allocation에 안전하게 write하고 allocation 내부 slice를 얻는 방법을 `CoherentAllocation`에 추가하는 task입니다. 난이도는 Beginner이고 contact는 Abdiel Janulgue입니다.

`Generic register abstraction [REGA]`는 generalized macro로 register constant와 structure를 자동 생성하는 방법을 정립합니다. 원문의 `register!(BOOT0, ...)` 입력은 offset, backing type, PCI BAR, read-only bit field를 선언하고 `BOOT0_OFFSET`, shift·mask constant, `Boot0(u32)`와 accessor method로 확장됩니다.

사용 예제는 revocable PCI BAR guard를 얻어 `Boot0::read(&bar)`로 register를 읽고 `boot0.revision()`을 출력합니다. Work-in-progress 구현은 `drivers/gpu/nova-core/regs/macros.rs`에 있으며 현재 nova-core가 사용합니다.

이 구현을 개선하고 가능하면 proc macro를 사용하여 다른 component도 쓸 수 있도록 `kernel` crate로 옮기는 것이 목표입니다. 이동 전 필요한 기능은 register가 아닌 field value를 위한 optional I/O, `u32` 외 크기, register·개별 field visibility control, field range를 표현하는 Rust slice syntax입니다. 난이도는 Advanced, contact는 Alexandre Courbot입니다.

Enablement (Rust)
=================

Tasks that are not directly related to nova-core, but are preconditions in terms
of required APIs.

FromPrimitive API [FPRI]
------------------------

Sometimes the need arises to convert a number to a value of an enum or a
structure.

A good example from nova-core would be the ``Chipset`` enum type, which defines
the value ``AD102``. When probing the GPU the value ``0x192`` can be read from a
certain register indication the chipset AD102. Hence, the enum value ``AD102``
should be derived from the number ``0x192``. Currently, nova-core uses a custom
implementation (``Chipset::from_u32`` for this.

Instead, it would be desirable to have something like the ``FromPrimitive``
trait [1] from the num crate.

Having this generalization also helps with implementing a generic macro that
automatically generates the corresponding mappings between a value and a number.

| Complexity: Beginner
| Link: https://docs.rs/num/latest/num/trait.FromPrimitive.html

Conversion from byte slices for types implementing FromBytes [TRSM]
-------------------------------------------------------------------

We retrieve several structures from byte streams coming from the BIOS or loaded
firmware. At the moment converting the bytes slice into the proper type require
an inelegant `unsafe` operation; this will go away once `FromBytes` implements
a proper `from_bytes` method.

| Complexity: Beginner

CoherentAllocation improvements [COHA]
--------------------------------------

`CoherentAllocation` needs a safe way to write into the allocation, and to
obtain slices within the allocation.

| Complexity: Beginner
| Contact: Abdiel Janulgue

Generic register abstraction [REGA]
-----------------------------------

Work out how register constants and structures can be automatically generated
through generalized macros.

Example:

.. code-block:: rust

        register!(BOOT0, 0x0, u32, pci::Bar<SIZE>, Fields [
           MINOR_REVISION(3:0, RO),
           MAJOR_REVISION(7:4, RO),
           REVISION(7:0, RO), // Virtual register combining major and minor rev.
        ])

This could expand to something like:

.. code-block:: rust

        const BOOT0_OFFSET: usize = 0x00000000;
        const BOOT0_MINOR_REVISION_SHIFT: u8 = 0;
        const BOOT0_MINOR_REVISION_MASK: u32 = 0x0000000f;
        const BOOT0_MAJOR_REVISION_SHIFT: u8 = 4;
        const BOOT0_MAJOR_REVISION_MASK: u32 = 0x000000f0;
        const BOOT0_REVISION_SHIFT: u8 = BOOT0_MINOR_REVISION_SHIFT;
        const BOOT0_REVISION_MASK: u32 = BOOT0_MINOR_REVISION_MASK | BOOT0_MAJOR_REVISION_MASK;

        struct Boot0(u32);

        impl Boot0 {
           #[inline]
           fn read(bar: &RevocableGuard<'_, pci::Bar<SIZE>>) -> Self {
              Self(bar.readl(BOOT0_OFFSET))
           }

           #[inline]
           fn minor_revision(&self) -> u32 {
              (self.0 & BOOT0_MINOR_REVISION_MASK) >> BOOT0_MINOR_REVISION_SHIFT
           }

           #[inline]
           fn major_revision(&self) -> u32 {
              (self.0 & BOOT0_MAJOR_REVISION_MASK) >> BOOT0_MAJOR_REVISION_SHIFT
           }

           #[inline]
           fn revision(&self) -> u32 {
              (self.0 & BOOT0_REVISION_MASK) >> BOOT0_REVISION_SHIFT
           }
        }

Usage:

.. code-block:: rust

        let bar = bar.try_access().ok_or(ENXIO)?;

        let boot0 = Boot0::read(&bar);
        pr_info!("Revision: {}\n", boot0.revision());

A work-in-progress implementation currently resides in
`drivers/gpu/nova-core/regs/macros.rs` and is used in nova-core. It would be
nice to improve it (possibly using proc macros) and move it to the `kernel`
crate so it can be used by other components as well.

Features desired before this happens:

* Make I/O optional I/O (for field values that are not registers),
* Support other sizes than `u32`,
* Allow visibility control for registers and individual fields,
* Use Rust slice syntax to express fields ranges.

| Complexity: Advanced
| Contact: Alexandre Courbot

Rust enablement: kernel subsystem API

`Numerical operations [NUMM]`은 standard library에 없거나 kernel용 최적화 구현이 없는 integer operation을 제공합니다. 현재 예는 C kernel의 `fls`에 해당하는 Find Last Set Bit입니다. 이 연산을 제공할 `num` core kernel module이 설계 중입니다. 난이도는 Intermediate, contact는 Alexandre Courbot입니다.

`Delay / Sleep abstractions [DLAY]`는 kernel의 `delay()`와 `sleep()` function을 위한 Rust abstraction입니다. FUJITA Tomonori가 `read_poll_timeout_atomic()` 계열 abstraction을 작업할 계획입니다. 난이도는 Beginner입니다.

`IRQ abstractions`는 IRQ handling용 Rust API입니다. Daniel Almeida가 IRQ를 request하는 core abstraction을 작업 중입니다. 선택적인 review·test 외에 이 core API 주변의 `pci::Device` code를 설계해야 합니다. 난이도는 Intermediate이며 Daniel Almeida가 contact입니다.

`Page abstraction for foreign pages`는 Rust page abstraction이 생성하지 않았고 직접 ownership하지 않는 page를 다루는 Rust API입니다. Abdiel Janulgue와 Lina가 진행 중이며 난이도는 Advanced입니다.

`Scatterlist / sg_table abstractions`는 scatterlist와 `sg_table`의 Rust abstraction입니다. Abdiel Janulgue의 선행 작업이 있지만 아직 mailing list에 올라오지 않았습니다. 난이도는 Intermediate입니다.

`PCI MISC APIs`는 기존 PCI device/driver abstraction에 SR-IOV, config space, capability, MSI API를 추가합니다. 난이도는 Beginner입니다.

`XArray bindings [XARR]`은 auxiliary device ID를 생성하기 위해 `xa_alloc`과 `xa_alloc_cyclic` binding을 요구합니다. 난이도는 Intermediate입니다.

`Debugfs abstractions`는 debugfs API의 Rust abstraction이며 `Export GSP log buffers` task의 선행 조건입니다. 난이도는 Intermediate입니다.

Numerical operations [NUMM]
---------------------------

Nova uses integer operations that are not part of the standard library (or not
implemented in an optimized way for the kernel). These include:

- The "Find Last Set Bit" (`fls` function of the C part of the kernel)
  operation.

A `num` core kernel module is being designed to provide these operations.

| Complexity: Intermediate
| Contact: Alexandre Courbot

Delay / Sleep abstractions [DLAY]
---------------------------------

Rust abstractions for the kernel's delay() and sleep() functions.

FUJITA Tomonori plans to work on abstractions for read_poll_timeout_atomic()
(and friends) [1].

| Complexity: Beginner
| Link: https://lore.kernel.org/netdev/[email protected]/ [1]

IRQ abstractions
----------------

Rust abstractions for IRQ handling.

There is active ongoing work from Daniel Almeida [1] for the "core" abstractions
to request IRQs.

Besides optional review and testing work, the required ``pci::Device`` code
around those core abstractions needs to be worked out.

| Complexity: Intermediate
| Link: https://lore.kernel.org/lkml/[email protected]/ [1]
| Contact: Daniel Almeida

Page abstraction for foreign pages
----------------------------------

Rust abstractions for pages not created by the Rust page abstraction without
direct ownership.

There is active onging work from Abdiel Janulgue [1] and Lina [2].

| Complexity: Advanced
| Link: https://lore.kernel.org/linux-mm/[email protected]/ [1]
| Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2]

Scatterlist / sg_table abstractions
-----------------------------------

Rust abstractions for scatterlist / sg_table.

There is preceding work from Abdiel Janulgue, which hasn't made it to the
mailing list yet.

| Complexity: Intermediate
| Contact: Abdiel Janulgue

PCI MISC APIs
-------------

Extend the existing PCI device / driver abstractions by SR-IOV, config space,
capability, MSI API abstractions.

| Complexity: Beginner

XArray bindings [XARR]
----------------------

We need bindings for `xa_alloc`/`xa_alloc_cyclic` in order to generate the
auxiliary device IDs.

| Complexity: Intermediate

Debugfs abstractions
--------------------

Rust abstraction for debugfs APIs.

| Reference: Export GSP log buffers
| Complexity: Intermediate

GPU 일반 기능

`Initial Devinit support`는 BIOS Device Initialization을 구현하는 task입니다. Memory sizing·waiting·PLL configuration이 포함됩니다. 난이도는 Beginner, contact는 Dave Airlie입니다.

`MMU / PT management`는 MMU와 page table 관리 architecture를 정하는 Expert task입니다. 특히 `nova-drm`이 asynchronous Vulkan queue를 구현하려면 locking을 포함한 매우 세밀한 control이 필요합니다. 관련 code를 공유하는 편이 일반적으로 바람직하지만 실제로 어떻게, 또는 공유 자체가 유리한지 평가해야 합니다.

`VRAM memory allocator`는 allocator 선택지를 조사하는 Advanced task입니다. 후보는 RB tree(interval tree)·`drm_mm`의 Rust abstraction, `maple_tree`, native Rust collection입니다.

`Instance Memory`는 page table을 저장하는 `instmem (bar2)` 지원을 구현합니다. 난이도는 Intermediate이고 contact는 Dave Airlie입니다.

GPU (general)
=============

Initial Devinit support
-----------------------

Implement BIOS Device Initialization, i.e. memory sizing, waiting, PLL
configuration.

| Contact: Dave Airlie
| Complexity: Beginner

MMU / PT management
-------------------

Work out the architecture for MMU / page table management.

We need to consider that nova-drm will need rather fine-grained control,
especially in terms of locking, in order to be able to implement asynchronous
Vulkan queues.

While generally sharing the corresponding code is desirable, it needs to be
evaluated how (and if at all) sharing the corresponding code is expedient.

| Complexity: Expert

VRAM memory allocator
---------------------

Investigate options for a VRAM memory allocator.

Some possible options:
  - Rust abstractions for
    - RB tree (interval tree) / drm_mm
    - maple_tree
  - native Rust collections

| Complexity: Advanced

Instance Memory
---------------

Implement support for instmem (bar2) used to store page tables.

| Complexity: Intermediate
| Contact: Dave Airlie

GPU System Processor task

`Export GSP log buffers`는 GSP-RM log buffer를 driver probe 실패 뒤에도 debugfs로 공개하는 기능입니다. Timur Tabi의 최근 patch가 이를 구현했으며 nova-core 초기 개발에도 유용합니다. `Debugfs abstractions`가 선행 조건이고 난이도는 Intermediate입니다.

`GSP firmware abstraction`은 version마다 data structure와 semantic이 비호환으로 바뀔 수 있는 불안정한 GSP-RM firmware API를 추상화하는 Expert task입니다. Rust procedural macro가 이 문제를 해결하기에 적합하다는 점이 nova-core에서 Rust를 사용하는 큰 동기입니다.

제안 pattern은 네 단계입니다. 먼저 C header에서 version별 namespace의 Rust structure를 생성합니다. 다음으로 generic namespace에서 firmware interface를 구현하는 abstraction structure를 만들고 implementation 차이에 version identifier를 붙입니다. Procedural macro가 이 abstraction으로 실제 version별 implementation을 생성하고, runtime에는 공통 trait 덕분에 같은 interface가 보장되는 올바른 version type을 instantiate합니다.

Nova-core PoC driver에 이 pattern의 PoC가 있습니다. Task 목표는 기능을 다듬고 다른 driver도 사용할 수 있도록 일반화하는 것입니다.

`GSP message queue`는 kernel driver와 GSP 사이 command·status communication을 위한 low-level queue를 구현합니다. 난이도는 Advanced입니다. `Bootstrap GSP`는 boot firmware를 호출해 GSP processor를 boot하고 초기 control message를 실행하는 Intermediate task입니다.

`Client / Device APIs`는 client/device allocation용 GSP message interface와 해당 allocation API를 구현합니다. `Bar PDE handling`은 kernel driver와 GSP 사이 BAR page-table 처리를 동기화합니다. 각각 Intermediate와 Beginner입니다.

`FIFO engine`은 GSP message interface와 `chid` allocation·channel handling API를 구현하는 Advanced task입니다. `GR engine`은 graphics engine의 GSP interface와 golden context 생성·promotion API를 구현하는 Advanced task입니다.

`CE engine`은 copy engine용 GSP message interface를 구현하는 Intermediate task이고, `VFN IRQ controller`는 VFN interrupt controller 지원을 추가하는 Intermediate task입니다. 이 GSP 실행 기능들의 contact는 Dave Airlie입니다.

GPU System Processor (GSP)
==========================

Export GSP log buffers
----------------------

Recent patches from Timur Tabi [1] added support to expose GSP-RM log buffers
(even after failure to probe the driver) through debugfs.

This is also an interesting feature for nova-core, especially in the early days.

| Link: https://lore.kernel.org/nouveau/[email protected]/ [1]
| Reference: Debugfs abstractions
| Complexity: Intermediate

GSP firmware abstraction
------------------------

The GSP-RM firmware API is unstable and may incompatibly change from version to
version, in terms of data structures and semantics.

This problem is one of the big motivations for using Rust for nova-core, since
it turns out that Rust's procedural macro feature provides a rather elegant way
to address this issue:

1. generate Rust structures from the C headers in a separate namespace per version
2. build abstraction structures (within a generic namespace) that implement the
   firmware interfaces; annotate the differences in implementation with version
   identifiers
3. use a procedural macro to generate the actual per version implementation out
   of this abstraction
4. instantiate the correct version type one on runtime (can be sure that all
   have the same interface because it's defined by a common trait)

There is a PoC implementation of this pattern, in the context of the nova-core
PoC driver.

This task aims at refining the feature and ideally generalize it, to be usable
by other drivers as well.

| Complexity: Expert

GSP message queue
-----------------

Implement low level GSP message queue (command, status) for communication
between the kernel driver and GSP.

| Complexity: Advanced
| Contact: Dave Airlie

Bootstrap GSP
-------------

Call the boot firmware to boot the GSP processor; execute initial control
messages.

| Complexity: Intermediate
| Contact: Dave Airlie

Client / Device APIs
--------------------

Implement the GSP message interface for client / device allocation and the
corresponding client and device allocation APIs.

| Complexity: Intermediate
| Contact: Dave Airlie

Bar PDE handling
----------------

Synchronize page table handling for BARs between the kernel driver and GSP.

| Complexity: Beginner
| Contact: Dave Airlie

FIFO engine
-----------

Implement support for the FIFO engine, i.e. the corresponding GSP message
interface and provide an API for chid allocation and channel handling.

| Complexity: Advanced
| Contact: Dave Airlie

GR engine
---------

Implement support for the graphics engine, i.e. the corresponding GSP message
interface and provide an API for (golden) context creation and promotion.

| Complexity: Advanced
| Contact: Dave Airlie

CE engine
---------

Implement support for the copy engine, i.e. the corresponding GSP message
interface.

| Complexity: Intermediate
| Contact: Dave Airlie

VFN IRQ controller
------------------

Support for the VFN interrupt controller.

| Complexity: Intermediate
| Contact: Dave Airlie

Second-level driver용 외부 API

`nova-core base API`는 vGPU manager와 `nova-drm` 같은 second-level driver를 연결하는 공통 API 부분을 설계하는 Advanced task입니다.

`vGPU manager API`는 base API가 다루지 않는 vGPU manager 전용 부분을 설계하는 Advanced task입니다.

`nova-core C API`는 vGPU manager driver가 요구하는 API의 C wrapper를 구현하는 Intermediate task입니다.

External APIs
=============

nova-core base API
------------------

Work out the common pieces of the API to connect 2nd level drivers, i.e. vGPU
manager and nova-drm.

| Complexity: Advanced

vGPU manager API
----------------

Work out the API parts required by the vGPU manager, which are not covered by
the base API.

| Complexity: Advanced

nova-core C API
---------------

Implement a C wrapper for the APIs required by the vGPU manager driver.

| Complexity: Intermediate

CI와 targeted uAPI test

`CI pipeline` task는 continuous integration test 선택지를 조사합니다. 난이도는 Advanced입니다.

범위는 단순한 KUnit test 실행부터 graphics CTS, 여러 guest VM boot를 통한 VFIO use-case 검증까지 확장할 수 있습니다.

더 정밀한 test와 debugging을 위해 uAPI 바로 위에 놓이는 새 test suite 도입도 검토할 가치가 있습니다. Mesa project와 협력하거나 code를 공유할 가능성도 있습니다.

Testing
=======

CI pipeline
-----------

Investigate option for continuous integration testing.

This can go from as simple as running KUnit tests over running (graphics) CTS to
booting up (multiple) guest VMs to test VFIO use-cases.

It might also be worth to consider the introduction of a new test suite directly
sitting on top of the uAPI for more targeted testing and debugging. There may be
options for collaboration / shared code with the Mesa project.

| Complexity: Advanced