Falcon (FAst Logic Controller)

Falcon core와 ucode

이 문서는 Ampere GPU와 그 이전 설계를 기준으로 Falcon core와 그 위에서 실행되는 ucode를 설명합니다. 미래 설계에도 대체로 적용되겠지만 변경될 수 있으며, 특히 `nova-core` driver와 Falcon의 상호작용을 이해하는 데 초점을 둡니다. 원문 SPDX 라이선스는 `GPL-2.0`입니다.

NVIDIA GPU에는 Falcon(FAst Logic Controller)이라는 작은 RISC 계열 microcontroller가 내장되어 secure firmware task, initialization, power management를 처리합니다. 최신 GPU에는 GSP(GPU System Processor), SEC2(security engine) 같은 Falcon instance가 여러 개 있을 수 있고 RISC-V core도 통합될 수 있습니다. 이 core는 RISC-V code와 Falcon code를 모두 실행할 수 있습니다.

Falcon core에서 실행되는 code를 이 문서에서는 `ucode`라고 부릅니다. Falcon은 instruction memory `IMEM`과 data memory `DMEM`을 분리하고, FBIF(Frame Buffer Interface)를 통한 작은 DMA engine으로 system memory에서 code를 load합니다.

`nova-core` driver는 Falcon을 reset·configure하고 DMA로 firmware를 load한 다음 CPU를 시작해야 합니다.

.. SPDX-License-Identifier: GPL-2.0

==============================
Falcon (FAst Logic Controller)
==============================
The following sections describe the Falcon core and the ucode running on it.
The descriptions are based on the Ampere GPU or earlier designs; however, they
should mostly apply to future designs as well, but everything is subject to
change. The overview provided here is mainly tailored towards understanding the
interactions of nova-core driver with the Falcon.

NVIDIA GPUs embed small RISC-like microcontrollers called Falcon cores, which
handle secure firmware tasks, initialization, and power management. Modern
NVIDIA GPUs may have multiple such Falcon instances (e.g., GSP (the GPU system
processor) and SEC2 (the security engine)) and also may integrate a RISC-V core.
This core is capable of running both RISC-V and Falcon code.

The code running on the Falcon cores is also called 'ucode', and will be
referred to as such in the following sections.

Falcons have separate instruction and data memories (IMEM/DMEM) and provide a
small DMA engine (via the FBIF - "Frame Buffer Interface") to load code from
system memory. The nova-core driver must reset and configure the Falcon, load
its firmware via DMA, and start its CPU.

Security level과 root of trust

Falcon은 Non-secure(NS), Light Secure(LS), Heavy Secure(HS) mode로 실행할 수 있습니다. HS는 Privilege Level 3(PL3)이며 가장 신뢰받는 code로 chip 거의 전체에 접근합니다. HS binary에는 boot 때 hardware가 직접 검증하는 signature가 포함되어 root of trust를 형성합니다.

예를 들어 `FWSEC-FRTS` command는 GSP의 HS mode에서 실행됩니다. WPR(Write Protect Region)을 설정하고 content를 load하는 FRTS는 HS ucode만 수행할 수 있으며 host CPU나 LS ucode는 수행할 수 없습니다.

LS(PL2)와 NS(PL0)는 HS보다 낮은 보안 level입니다. 이 binary도 보통 signature를 포함하지만, Falcon에 LS/NS firmware를 load하려면 다른 Falcon에서 HS code가 실행 중이어야 합니다. 그 HS ucode가 signature를 검증하고 root of trust를 이어 갑니다.

Ampere에서 CPU는 SEC2 Falcon의 HS mode로 `Booter` ucode를 실행합니다. Booter가 runtime GSP binary인 `GSP-RM`을 인증하고 GSP Falcon의 LS mode로 실행합니다. Reset 뒤에는 GSP의 FWSEC가 PMU에 devinit engine을 LS mode로 load하는 비슷한 흐름도 사용합니다.

최초 신뢰를 만들려면 Falcon에서 시작하는 code가 immutable하고 read-only memory에 hardwire되어야 합니다. 이를 Boot ROM(`BROM`)이라 하며, CPU의 `nova-core` driver는 `regs.rs`에 정의된 `BROM` prefix register를 통해 통신합니다.

Driver는 VBIOS에서 ucode를 읽고 BROM·DMA register를 설정하여 system memory의 HS ucode를 Falcon IMEM/DMEM으로 load합니다. Falcon Boot ROM이 이를 검증합니다. 검증된 HS code는 이후 다른 Falcon의 LS/NS binary를 검증·load·start합니다. 이 단계에서는 BROM 대신 HS ucode가 signature를 계산·확인합니다.

따라서 일반 trust chain은 `Hardware BROM → HS ucode → LS/NS ucode`입니다. Ampere의 구체적인 예는 `SEC2 BROM → SEC2의 HS Booter → GSP의 LS GSP-RM`입니다.

CPU는 HS ucode를 Falcon에 load하여 hardware 검증 뒤 실행하게 할 수 있지만, 보통 LS/NS ucode를 직접 load하지 않습니다. SEC2의 LS `SEC2-RTOS`를 올릴 HS component가 없는 상황에서는 CPU가 `GSP-provided sequencer`를 이용해 GSP에 임시 HS ucode를 load합니다. 이 HS ucode가 SEC2-RTOS를 SEC2의 LS mode로 load한 다음, GSP는 자체 LS `GSP-RM` 실행을 재개합니다.

Falcon security levels
======================
Falcons can run in Non-secure (NS), Light Secure (LS), or Heavy Secure (HS)
modes.

Heavy Secured (HS) also known as Privilege Level 3 (PL3)
--------------------------------------------------------
HS ucode is the most trusted code and has access to pretty much everything on
the chip. The HS binary includes a signature in it which is verified at boot.
This signature verification is done by the hardware itself, thus establishing a
root of trust. For example, the FWSEC-FRTS command (see fwsec.rst) runs on the
GSP in HS mode. FRTS, which involves setting up and loading content into the WPR
(Write Protect Region), has to be done by the HS ucode and cannot be done by the
host CPU or LS ucode.

Light Secured (LS or PL2) and Non Secured (NS or PL0)
-----------------------------------------------------
These modes are less secure than HS. Like HS, the LS or NS ucode binary also
typically includes a signature in it. To load firmware in LS or NS mode onto a
Falcon, another Falcon needs to be running in HS mode, which also establishes the
root of trust. For example, in the case of an Ampere GPU, the CPU runs the "Booter"
ucode in HS mode on the SEC2 Falcon, which then authenticates and runs the
run-time GSP binary (GSP-RM) in LS mode on the GSP Falcon. Similarly, as an
example, after reset on an Ampere, FWSEC runs on the GSP which then loads the
devinit engine onto the PMU in LS mode.

Root of trust establishment
---------------------------
To establish a root of trust, the code running on a Falcon must be immutable and
hardwired into a read-only memory (ROM). This follows industry norms for
verification of firmware. This code is called the Boot ROM (BROM). The nova-core
driver on the CPU communicates with Falcon's Boot ROM through various Falcon
registers prefixed with "BROM" (see regs.rs).

After nova-core driver reads the necessary ucode from VBIOS, it programs the
BROM and DMA registers to trigger the Falcon to load the HS ucode from the system
memory into the Falcon's IMEM/DMEM. Once the HS ucode is loaded, it is verified
by the Falcon's Boot ROM.

Once the verified HS code is running on a Falcon, it can verify and load other
LS/NS ucode binaries onto other Falcons and start them. The process of signature
verification is the same as HS; just in this case, the hardware (BROM) doesn't
compute the signature, but the HS ucode does.

The root of trust is therefore established as follows:
     Hardware (Boot ROM running on the Falcon) -> HS ucode -> LS/NS ucode.

On an Ampere GPU, for example, the boot verification flow is:
     Hardware (Boot ROM running on the SEC2) ->
          HS ucode (Booter running on the SEC2) ->
               LS ucode (GSP-RM running on the GSP)

.. note::
     While the CPU can load HS ucode onto a Falcon microcontroller and have it
     verified by the hardware and run, the CPU itself typically does not load
     LS or NS ucode and run it. Loading of LS or NS ucode is done mainly by the
     HS ucode. For example, on an Ampere GPU, after the Booter ucode runs on the
     SEC2 in HS mode and loads the GSP-RM binary onto the GSP, it needs to run
     the "SEC2-RTOS" ucode at runtime. This presents a problem: there is no
     component to load the SEC2-RTOS ucode onto the SEC2. The CPU cannot load
     LS code, and GSP-RM must run in LS mode. To overcome this, the GSP is
     temporarily made to run HS ucode (which is itself loaded by the CPU via
     the nova-core driver using a "GSP-provided sequencer") which then loads
     the SEC2-RTOS ucode onto the SEC2 in LS mode. The GSP then resumes
     running its own GSP-RM LS ucode.

Falcon memory subsystem과 DMA

Falcon은 분리된 IMEM·DMEM과 FBDMA(Framebuffer DMA)라는 작은 DMA engine을 가집니다. FBDMA는 FBIF를 통해 Falcon 내부 IMEM/DMEM과 external memory 사이를 전송합니다. External target은 system memory와 framebuffer memory(VRAM) 모두 가능합니다.

FBDMA transfer를 수행할 때 FBIF는 memory 접근 방식인 aperture type을 결정하도록 설정합니다. `nova-core`에서는 `FalconFbifTarget` enum이 이 target을 나타냅니다.

Falcon의 IO-PMP(Input/Output Physical Memory Protection) block은 FBDMA가 external memory에 접근할 수 있는 범위를 제어합니다.

원문 개념도에서 external Framebuffer/System DRAM은 FBIF memory interface에 연결됩니다. RISC core가 FBDMA와 IO-PMP를 설정하고, FBDMA는 configured FBIF aperture와 IO-PMP 보호 경로를 사용해 IMEM·DMEM을 전송합니다. RISC core는 IMEM·DMEM에 direct core access도 하며 BROM과 양방향으로 통신합니다.

Falcon memory subsystem and DMA engine
======================================
Falcons have separate instruction and data memories (IMEM/DMEM)
and contains a small DMA engine called FBDMA (Framebuffer DMA) which does
DMA transfers to/from the IMEM/DMEM memory inside the Falcon via the FBIF
(Framebuffer Interface), to external memory.

DMA transfers are possible from the Falcon's memory to both the system memory
and the framebuffer memory (VRAM).

To perform a DMA via the FBDMA, the FBIF is configured to decide how the memory
is accessed (also known as aperture type). In the nova-core driver, this is
determined by the `FalconFbifTarget` enum.

The IO-PMP block (Input/Output Physical Memory Protection) unit in the Falcon
controls access by the FBDMA to the external memory.

Conceptual diagram (not exact) of the Falcon and its memory subsystem is as follows::

               External Memory (Framebuffer / System DRAM)
                              ^  |
                              |  |
                              |  v
     +-----------------------------------------------------+
     |                           |                         |
     |   +---------------+       |                         |
     |   |     FBIF      |-------+                         |  FALCON
     |   | (FrameBuffer  |   Memory Interface              |  PROCESSOR
     |   |  InterFace)   |                                 |
     |   |  Apertures    |                                 |
     |   |  Configures   |                                 |
     |   |  mem access   |                                 |
     |   +-------^-------+                                 |
     |           |                                         |
     |           | FBDMA uses configured FBIF apertures    |
     |           | to access External Memory
     |           |
     |   +-------v--------+      +---------------+
     |   |    FBDMA       |  cfg |     RISC      |
     |   | (FrameBuffer   |<---->|     CORE      |----->. Direct Core Access
     |   |  DMA Engine)   |      |               |      |
     |   | - Master dev.  |      | (can run both |      |
     |   +-------^--------+      | Falcon and    |      |
     |           |        cfg--->| RISC-V code)  |      |
     |           |        /      |               |      |
     |           |        |      +---------------+      |    +------------+
     |           |        |                             |    |   BROM     |
     |           |        |                             <--->| (Boot ROM) |
     |           |       /                              |    +------------+
     |           |      v                               |
     |   +---------------+                              |
     |   |    IO-PMP     | Controls access by FBDMA     |
     |   | (IO Physical  | and other IO Masters         |
     |   | Memory Protect)                              |
     |   +-------^-------+                              |
     |           |                                      |
     |           | Protected Access Path for FBDMA      |
     |           v                                      |
     |   +---------------------------------------+      |
     |   |       Memory                          |      |
     |   |   +---------------+  +------------+   |      |
     |   |   |    IMEM       |  |    DMEM    |   |<-----+
     |   |   | (Instruction  |  |   (Data    |   |
     |   |   |  Memory)      |  |   Memory)  |   |
     |   |   +---------------+  +------------+   |
     |   +---------------------------------------+
     +-----------------------------------------------------+