요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
==============================
drm/komeda Arm display driver
==============================
The drm/komeda driver supports the Arm display processor D71 and later products,
this document gives a brief overview of driver design: how it works and why
design it like that.
Overview of D71 like display IPs
================================
From D71, Arm display IP begins to adopt a flexible and modularized
architecture. A display pipeline is made up of multiple individual and
functional pipeline stages called components, and every component has some
specific capabilities that can give the flowed pipeline pixel data a
particular processing.
Typical D71 components:
Layer
-----
Layer is the first pipeline stage, which prepares the pixel data for the next
stage. It fetches the pixel from memory, decodes it if it's AFBC, rotates the
source image, unpacks or converts YUV pixels to the device internal RGB pixels,
then adjusts the color_space of pixels if needed.
Scaler
------
As its name suggests, scaler takes responsibility for scaling, and D71 also
supports image enhancements by scaler.
The usage of scaler is very flexible and can be connected to layer output
for layer scaling, or connected to compositor and scale the whole display
frame and then feed the output data into wb_layer which will then write it
into memory.
Compositor (compiz)
-------------------
Compositor blends multiple layers or pixel data flows into one single display
frame. its output frame can be fed into post image processor for showing it on
the monitor or fed into wb_layer and written to memory at the same time.
user can also insert a scaler between compositor and wb_layer to down scale
the display frame first and then write to memory.
Writeback Layer (wb_layer)
--------------------------
Writeback layer does the opposite things of Layer, which connects to compiz
and writes the composition result to memory.
Post image processor (improc)
-----------------------------
Post image processor adjusts frame data like gamma and color space to fit the
requirements of the monitor.
Timing controller (timing_ctrlr)
--------------------------------
Final stage of display pipeline, Timing controller is not for the pixel
handling, but only for controlling the display timing.
Merger
------
D71 scaler mostly only has the half horizontal input/output capabilities
compared with Layer, like if Layer supports 4K input size, the scaler only can
support 2K input/output in the same time. To achieve the ful frame scaling, D71
introduces Layer Split, which splits the whole image to two half parts and feeds
them to two Layers A and B, and does the scaling independently. After scaling
the result need to be fed to merger to merge two part images together, and then
output merged result to compiz.
Splitter
--------
Similar to Layer Split, but Splitter is used for writeback, which splits the
compiz result to two parts and then feed them to two scalers.
Possible D71 Pipeline usage
===========================
Benefitting from the modularized architecture, D71 pipelines can be easily
adjusted to fit different usages. And D71 has two pipelines, which support two
types of working mode:
- Dual display mode
Two pipelines work independently and separately to drive two display outputs.
- Single display mode
Two pipelines work together to drive only one display output.
On this mode, pipeline_B doesn't work independently, but outputs its
composition result into pipeline_A, and its pixel timing also derived from
pipeline_A.timing_ctrlr. The pipeline_B works just like a "slave" of
pipeline_A(master)
Single pipeline data flow
-------------------------
.. kernel-render:: DOT
:alt: Single pipeline digraph
:caption: Single pipeline data flow
digraph single_ppl {
rankdir=LR;
subgraph {
"Memory";
"Monitor";
}
subgraph cluster_pipeline {
style=dashed
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Scaler-0";
"Scaler-1";
"Scaler-0/1"
}
node [bgcolor=grey style=filled]
"Layer-0" -> "Scaler-0"
"Layer-1" -> "Scaler-0"
"Layer-2" -> "Scaler-1"
"Layer-3" -> "Scaler-1"
"Layer-0" -> "Compiz"
"Layer-1" -> "Compiz"
"Layer-2" -> "Compiz"
"Layer-3" -> "Compiz"
"Scaler-0" -> "Compiz"
"Scaler-1" -> "Compiz"
"Compiz" -> "Scaler-0/1" -> "Wb_layer"
"Compiz" -> "Improc" -> "Timing Controller"
}
"Wb_layer" -> "Memory"
"Timing Controller" -> "Monitor"
}
Dual pipeline with Slave enabled
--------------------------------
.. kernel-render:: DOT
:alt: Slave pipeline digraph
:caption: Slave pipeline enabled data flow
digraph slave_ppl {
rankdir=LR;
subgraph {
"Memory";
"Monitor";
}
node [shape=box]
subgraph cluster_pipeline_slave {
style=dashed
label="Slave Pipeline_B"
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Slave.Scaler-0";
"Slave.Scaler-1";
}
node [bgcolor=grey style=filled]
"Slave.Layer-0" -> "Slave.Scaler-0"
"Slave.Layer-1" -> "Slave.Scaler-0"
"Slave.Layer-2" -> "Slave.Scaler-1"
"Slave.Layer-3" -> "Slave.Scaler-1"
"Slave.Layer-0" -> "Slave.Compiz"
"Slave.Layer-1" -> "Slave.Compiz"
"Slave.Layer-2" -> "Slave.Compiz"
"Slave.Layer-3" -> "Slave.Compiz"
"Slave.Scaler-0" -> "Slave.Compiz"
"Slave.Scaler-1" -> "Slave.Compiz"
}
subgraph cluster_pipeline_master {
style=dashed
label="Master Pipeline_A"
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Scaler-0";
"Scaler-1";
"Scaler-0/1"
}
node [bgcolor=grey style=filled]
"Layer-0" -> "Scaler-0"
"Layer-1" -> "Scaler-0"
"Layer-2" -> "Scaler-1"
"Layer-3" -> "Scaler-1"
"Slave.Compiz" -> "Compiz"
"Layer-0" -> "Compiz"
"Layer-1" -> "Compiz"
"Layer-2" -> "Compiz"
"Layer-3" -> "Compiz"
"Scaler-0" -> "Compiz"
"Scaler-1" -> "Compiz"
"Compiz" -> "Scaler-0/1" -> "Wb_layer"
"Compiz" -> "Improc" -> "Timing Controller"
}
"Wb_layer" -> "Memory"
"Timing Controller" -> "Monitor"
}
Sub-pipelines for input and output
----------------------------------
A complete display pipeline can be easily divided into three sub-pipelines
according to the in/out usage.
Layer(input) pipeline
~~~~~~~~~~~~~~~~~~~~~
.. kernel-render:: DOT
:alt: Layer data digraph
:caption: Layer (input) data flow
digraph layer_data_flow {
rankdir=LR;
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Scaler-n";
}
"Layer-n" -> "Scaler-n" -> "Compiz"
}
.. kernel-render:: DOT
:alt: Layer Split digraph
:caption: Layer Split pipeline
digraph layer_data_flow {
rankdir=LR;
node [shape=box]
"Layer-0/1" -> "Scaler-0" -> "Merger"
"Layer-2/3" -> "Scaler-1" -> "Merger"
"Merger" -> "Compiz"
}
Writeback(output) pipeline
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. kernel-render:: DOT
:alt: writeback digraph
:caption: Writeback(output) data flow
digraph writeback_data_flow {
rankdir=LR;
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Scaler-n";
}
"Compiz" -> "Scaler-n" -> "Wb_layer"
}
.. kernel-render:: DOT
:alt: split writeback digraph
:caption: Writeback(output) Split data flow
digraph writeback_data_flow {
rankdir=LR;
node [shape=box]
"Compiz" -> "Splitter"
"Splitter" -> "Scaler-0" -> "Merger"
"Splitter" -> "Scaler-1" -> "Merger"
"Merger" -> "Wb_layer"
}
Display output pipeline
~~~~~~~~~~~~~~~~~~~~~~~
.. kernel-render:: DOT
:alt: display digraph
:caption: display output data flow
digraph single_ppl {
rankdir=LR;
node [shape=box]
"Compiz" -> "Improc" -> "Timing Controller"
}
In the following section we'll see these three sub-pipelines will be handled
by KMS-plane/wb_conn/crtc respectively.
Komeda Resource abstraction
===========================
struct komeda_pipeline/component
--------------------------------
To fully utilize and easily access/configure the HW, the driver side also uses
a similar architecture: Pipeline/Component to describe the HW features and
capabilities, and a specific component includes two parts:
- Data flow controlling.
- Specific component capabilities and features.
So the driver defines a common header struct komeda_component to describe the
data flow control and all specific components are a subclass of this base
structure.
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h
:internal:
Resource discovery and initialization
=====================================
Pipeline and component are used to describe how to handle the pixel data. We
still need a @struct komeda_dev to describe the whole view of the device, and
the control-abilites of device.
We have &komeda_dev, &komeda_pipeline, &komeda_component. Now fill devices with
pipelines. Since komeda is not for D71 only but also intended for later products,
of course we’d better share as much as possible between different products. To
achieve this, split the komeda device into two layers: CORE and CHIP.
- CORE: for common features and capabilities handling.
- CHIP: for register programming and HW specific feature (limitation) handling.
CORE can access CHIP by three chip function structures:
- struct komeda_dev_funcs
- struct komeda_pipeline_funcs
- struct komeda_component_funcs
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_dev.h
:internal:
Format handling
===============
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h
:internal:
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.h
:internal:
Attach komeda_dev to DRM-KMS
============================
Komeda abstracts resources by pipeline/component, but DRM-KMS uses
crtc/plane/connector. One KMS-obj cannot represent only one single component,
since the requirements of a single KMS object cannot simply be achieved by a
single component, usually that needs multiple components to fit the requirement.
Like set mode, gamma, ctm for KMS all target on CRTC-obj, but komeda needs
compiz, improc and timing_ctrlr to work together to fit these requirements.
And a KMS-Plane may require multiple komeda resources: layer/scaler/compiz.
So, one KMS-Obj represents a sub-pipeline of komeda resources.
- Plane: `Layer(input) pipeline`_
- Wb_connector: `Writeback(output) pipeline`_
- Crtc: `Display output pipeline`_
So, for komeda, we treat KMS crtc/plane/connector as users of pipeline and
component, and at any one time a pipeline/component only can be used by one
user. And pipeline/component will be treated as private object of DRM-KMS; the
state will be managed by drm_atomic_state as well.
How to map plane to Layer(input) pipeline
-----------------------------------------
Komeda has multiple Layer input pipelines, see:
- `Single pipeline data flow`_
- `Dual pipeline with Slave enabled`_
The easiest way is binding a plane to a fixed Layer pipeline, but consider the
komeda capabilities:
- Layer Split, See `Layer(input) pipeline`_
Layer_Split is quite complicated feature, which splits a big image into two
parts and handles it by two layers and two scalers individually. But it
imports an edge problem or effect in the middle of the image after the split.
To avoid such a problem, it needs a complicated Split calculation and some
special configurations to the layer and scaler. We'd better hide such HW
related complexity to user mode.
- Slave pipeline, See `Dual pipeline with Slave enabled`_
Since the compiz component doesn't output alpha value, the slave pipeline
only can be used for bottom layers composition. The komeda driver wants to
hide this limitation to the user. The way to do this is to pick a suitable
Layer according to plane_state->zpos.
So for komeda, the KMS-plane doesn't represent a fixed komeda layer pipeline,
but multiple Layers with same capabilities. Komeda will select one or more
Layers to fit the requirement of one KMS-plane.
Make component/pipeline to be drm_private_obj
---------------------------------------------
Add :c:type:`drm_private_obj` to :c:type:`komeda_component`, :c:type:`komeda_pipeline`
.. code-block:: c
struct komeda_component {
struct drm_private_obj obj;
...
}
struct komeda_pipeline {
struct drm_private_obj obj;
...
}
Tracking component_state/pipeline_state by drm_atomic_state
-----------------------------------------------------------
Add :c:type:`drm_private_state` and user to :c:type:`komeda_component_state`,
:c:type:`komeda_pipeline_state`
.. code-block:: c
struct komeda_component_state {
struct drm_private_state obj;
void *binding_user;
...
}
struct komeda_pipeline_state {
struct drm_private_state obj;
struct drm_crtc *crtc;
...
}
komeda component validation
---------------------------
Komeda has multiple types of components, but the process of validation are
similar, usually including the following steps:
.. code-block:: c
int komeda_xxxx_validate(struct komeda_component_xxx xxx_comp,
struct komeda_component_output *input_dflow,
struct drm_plane/crtc/connector *user,
struct drm_plane/crtc/connector_state, *user_state)
{
setup 1: check if component is needed, like the scaler is optional depending
on the user_state; if unneeded, just return, and the caller will
put the data flow into next stage.
Setup 2: check user_state with component features and capabilities to see
if requirements can be met; if not, return fail.
Setup 3: get component_state from drm_atomic_state, and try set to set
user to component; fail if component has been assigned to another
user already.
Setup 3: configure the component_state, like set its input component,
convert user_state to component specific state.
Setup 4: adjust the input_dflow and prepare it for the next stage.
}
komeda_kms Abstraction
----------------------
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_kms.h
:internal:
komde_kms Functions
-------------------
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_crtc.c
:internal:
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_plane.c
:internal:
Build komeda to be a Linux module driver
========================================
Now we have two level devices:
- komeda_dev: describes the real display hardware.
- komeda_kms_dev: attaches or connects komeda_dev to DRM-KMS.
All komeda operations are supplied or operated by komeda_dev or komeda_kms_dev,
the module driver is only a simple wrapper to pass the Linux command
(probe/remove/pm) into komeda_dev or komeda_kms_dev.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
D71 계열 표시 IP와 구성요소
1-75`drm/komeda` 드라이버는 Arm D71 표시 프로세서와 그 이후 제품을 지원합니다. 이 문서는 드라이버가 어떻게 동작하며, 왜 현재와 같은 구조로 설계되었는지를 개괄합니다. 원문의 SPDX 라이선스는 `GPL-2.0`입니다.
D71부터 Arm 표시 IP는 유연하고 모듈화된 구조를 채택합니다. 하나의 display pipeline은 component라고 부르는 독립적인 기능 단계 여러 개로 이루어지며, 각 component는 흐르는 pixel data에 특정 처리를 적용하는 고유 기능과 한계를 가집니다.
`Layer`는 첫 단계입니다. 메모리에서 pixel을 가져오고, AFBC이면 decode하며, source image를 회전하고, YUV pixel을 unpack하거나 장치 내부 RGB 형식으로 변환합니다. 필요하면 pixel의 `color_space`도 조정해 다음 단계가 사용할 데이터를 준비합니다.
`Scaler`는 크기 조절을 담당하고 D71에서는 image enhancement도 수행합니다. Layer 출력 뒤에 연결해 개별 layer를 확대·축소할 수 있고, compositor 뒤에 연결해 전체 display frame을 조절한 다음 `wb_layer`로 보내 메모리에 기록할 수도 있습니다.
`Compositor (compiz)`는 여러 layer 또는 pixel data flow를 하나의 display frame으로 합성합니다. 출력은 monitor 표시를 위해 `improc`로 보낼 수 있고, 동시에 `wb_layer`로 보내 메모리에 기록할 수도 있습니다. Compositor와 writeback 사이에 scaler를 넣으면 전체 frame을 축소한 뒤 기록할 수 있습니다.
`Writeback Layer (wb_layer)`는 Layer와 반대 방향으로 동작합니다. `compiz`에 연결하여 합성 결과를 메모리에 씁니다. `Post image processor (improc)`는 gamma와 color space 같은 frame 속성을 monitor 요구사항에 맞게 조정합니다. 마지막 단계인 `Timing controller (timing_ctrlr)`는 pixel 자체를 처리하지 않고 display timing만 제어합니다.
D71의 scaler는 보통 Layer의 절반 수준인 수평 입출력 폭만 처리합니다. 예를 들어 Layer가 4K 입력을 받더라도 scaler는 같은 시점에 2K만 처리할 수 있습니다. 전체 frame을 scale하기 위해 Layer Split이 이미지를 두 절반으로 나누고 Layer A/B와 두 scaler가 각각 처리한 뒤 `Merger`가 다시 결합해 `compiz`로 보냅니다. `Splitter`는 writeback 방향에서 같은 역할을 하며 `compiz` 결과를 둘로 나누어 두 scaler에 공급합니다.
구성요소가 입력 pixel을 표시 또는 메모리 출력으로 전달하는 기본 흐름입니다.
각 component의 주된 책임과 연결 위치입니다.
.. SPDX-License-Identifier: GPL-2.0
==============================
drm/komeda Arm display driver
==============================
The drm/komeda driver supports the Arm display processor D71 and later products,
this document gives a brief overview of driver design: how it works and why
design it like that.
Overview of D71 like display IPs
================================
From D71, Arm display IP begins to adopt a flexible and modularized
architecture. A display pipeline is made up of multiple individual and
functional pipeline stages called components, and every component has some
specific capabilities that can give the flowed pipeline pixel data a
particular processing.
Typical D71 components:
Layer
-----
Layer is the first pipeline stage, which prepares the pixel data for the next
stage. It fetches the pixel from memory, decodes it if it's AFBC, rotates the
source image, unpacks or converts YUV pixels to the device internal RGB pixels,
then adjusts the color_space of pixels if needed.
Scaler
------
As its name suggests, scaler takes responsibility for scaling, and D71 also
supports image enhancements by scaler.
The usage of scaler is very flexible and can be connected to layer output
for layer scaling, or connected to compositor and scale the whole display
frame and then feed the output data into wb_layer which will then write it
into memory.
Compositor (compiz)
-------------------
Compositor blends multiple layers or pixel data flows into one single display
frame. its output frame can be fed into post image processor for showing it on
the monitor or fed into wb_layer and written to memory at the same time.
user can also insert a scaler between compositor and wb_layer to down scale
the display frame first and then write to memory.
Writeback Layer (wb_layer)
--------------------------
Writeback layer does the opposite things of Layer, which connects to compiz
and writes the composition result to memory.
Post image processor (improc)
-----------------------------
Post image processor adjusts frame data like gamma and color space to fit the
requirements of the monitor.
Timing controller (timing_ctrlr)
--------------------------------
Final stage of display pipeline, Timing controller is not for the pixel
handling, but only for controlling the display timing.
Merger
------
D71 scaler mostly only has the half horizontal input/output capabilities
compared with Layer, like if Layer supports 4K input size, the scaler only can
support 2K input/output in the same time. To achieve the ful frame scaling, D71
introduces Layer Split, which splits the whole image to two half parts and feeds
them to two Layers A and B, and does the scaling independently. After scaling
the result need to be fed to merger to merge two part images together, and then
output merged result to compiz.
Splitter
--------
Similar to Layer Split, but Splitter is used for writeback, which splits the
compiz result to two parts and then feed them to two scalers.
D71 동작 모드와 단일 pipeline 흐름
76-140모듈화 구조 덕분에 D71 pipeline은 사용 목적에 맞게 쉽게 조합할 수 있습니다. D71에는 두 pipeline이 있으며, 두 display 출력을 각각 구동하는 `Dual display mode`와 두 pipeline이 하나의 display 출력을 함께 구동하는 `Single display mode`를 지원합니다.
Single display mode에서 `pipeline_B`는 독립적으로 동작하지 않습니다. 합성 결과를 `pipeline_A`로 보내며 pixel timing도 `pipeline_A.timing_ctrlr`에서 파생됩니다. 따라서 `pipeline_B`는 master인 `pipeline_A`의 slave처럼 동작합니다.
단일 pipeline DOT 도식에서는 `Layer-0`부터 `Layer-3`까지가 직접 `Compiz`에 들어가거나 선택적으로 `Scaler-0/1`을 거쳐 들어갑니다. `Compiz` 출력은 표시 경로인 `Improc → Timing Controller → Monitor`와 writeback 경로인 `Scaler-0/1 → Wb_layer → Memory`로 나뉩니다. 회색 점선 scaler는 해당 연결이 선택적임을 나타냅니다.
두 pipeline의 독립성과 출력 수를 기준으로 비교합니다.
원문 DOT의 모든 처리 분기를 구조화한 흐름입니다.
Possible D71 Pipeline usage
===========================
Benefitting from the modularized architecture, D71 pipelines can be easily
adjusted to fit different usages. And D71 has two pipelines, which support two
types of working mode:
- Dual display mode
Two pipelines work independently and separately to drive two display outputs.
- Single display mode
Two pipelines work together to drive only one display output.
On this mode, pipeline_B doesn't work independently, but outputs its
composition result into pipeline_A, and its pixel timing also derived from
pipeline_A.timing_ctrlr. The pipeline_B works just like a "slave" of
pipeline_A(master)
Single pipeline data flow
-------------------------
.. kernel-render:: DOT
:alt: Single pipeline digraph
:caption: Single pipeline data flow
digraph single_ppl {
rankdir=LR;
subgraph {
"Memory";
"Monitor";
}
subgraph cluster_pipeline {
style=dashed
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Scaler-0";
"Scaler-1";
"Scaler-0/1"
}
node [bgcolor=grey style=filled]
"Layer-0" -> "Scaler-0"
"Layer-1" -> "Scaler-0"
"Layer-2" -> "Scaler-1"
"Layer-3" -> "Scaler-1"
"Layer-0" -> "Compiz"
"Layer-1" -> "Compiz"
"Layer-2" -> "Compiz"
"Layer-3" -> "Compiz"
"Scaler-0" -> "Compiz"
"Scaler-1" -> "Compiz"
"Compiz" -> "Scaler-0/1" -> "Wb_layer"
"Compiz" -> "Improc" -> "Timing Controller"
}
"Wb_layer" -> "Memory"
"Timing Controller" -> "Monitor"
}
Dual pipeline with Slave enabled
Slave가 활성화된 이중 pipeline
141-212Slave pipeline 도식은 `Slave Pipeline_B`와 `Master Pipeline_A`를 함께 보여 줍니다. Slave 쪽의 `Slave.Layer-0`부터 `Slave.Layer-3`까지는 직접 또는 `Slave.Scaler-0/1`을 거쳐 `Slave.Compiz`에 합성됩니다.
`Slave.Compiz` 결과는 Master의 `Compiz` 입력으로 들어갑니다. Master의 자체 Layer와 Scaler 출력도 같은 `Compiz`에 들어가므로, 최종 합성은 pipeline_A에서 이루어집니다. 이후 writeback은 `Scaler-0/1 → Wb_layer → Memory`, 표시는 `Improc → Timing Controller → Monitor` 경로를 따릅니다.
이 구조에서 pipeline_B는 별도 monitor를 구동하지 않고 하위 합성 결과만 master에 제공합니다. 최종 timing과 출력은 pipeline_A가 담당하므로, 두 pipeline을 사용하더라도 KMS 관점에서는 하나의 display output으로 보입니다.
두 pipeline을 하나의 출력으로 결합하는 원문 DOT 흐름입니다.
Single display mode에서 각 pipeline이 맡는 범위입니다.
--------------------------------
.. kernel-render:: DOT
:alt: Slave pipeline digraph
:caption: Slave pipeline enabled data flow
digraph slave_ppl {
rankdir=LR;
subgraph {
"Memory";
"Monitor";
}
node [shape=box]
subgraph cluster_pipeline_slave {
style=dashed
label="Slave Pipeline_B"
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Slave.Scaler-0";
"Slave.Scaler-1";
}
node [bgcolor=grey style=filled]
"Slave.Layer-0" -> "Slave.Scaler-0"
"Slave.Layer-1" -> "Slave.Scaler-0"
"Slave.Layer-2" -> "Slave.Scaler-1"
"Slave.Layer-3" -> "Slave.Scaler-1"
"Slave.Layer-0" -> "Slave.Compiz"
"Slave.Layer-1" -> "Slave.Compiz"
"Slave.Layer-2" -> "Slave.Compiz"
"Slave.Layer-3" -> "Slave.Compiz"
"Slave.Scaler-0" -> "Slave.Compiz"
"Slave.Scaler-1" -> "Slave.Compiz"
}
subgraph cluster_pipeline_master {
style=dashed
label="Master Pipeline_A"
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Scaler-0";
"Scaler-1";
"Scaler-0/1"
}
node [bgcolor=grey style=filled]
"Layer-0" -> "Scaler-0"
"Layer-1" -> "Scaler-0"
"Layer-2" -> "Scaler-1"
"Layer-3" -> "Scaler-1"
"Slave.Compiz" -> "Compiz"
"Layer-0" -> "Compiz"
"Layer-1" -> "Compiz"
"Layer-2" -> "Compiz"
"Layer-3" -> "Compiz"
"Scaler-0" -> "Compiz"
"Scaler-1" -> "Compiz"
"Compiz" -> "Scaler-0/1" -> "Wb_layer"
"Compiz" -> "Improc" -> "Timing Controller"
}
"Wb_layer" -> "Memory"
"Timing Controller" -> "Monitor"
}
Sub-pipelines for input and output
입력·writeback·display 출력 sub-pipeline
213-297완전한 display pipeline은 입출력 용도에 따라 세 sub-pipeline으로 나눌 수 있습니다. 뒤의 KMS 연결에서는 이 세 경로를 각각 KMS plane, writeback connector(`wb_conn`), CRTC가 담당합니다.
`Layer(input) pipeline`의 일반 경로는 `Layer-n → optional Scaler-n → Compiz`입니다. 큰 이미지를 분할하는 경로에서는 `Layer-0/1 → Scaler-0`과 `Layer-2/3 → Scaler-1`이 병렬로 처리되고, 두 결과가 `Merger → Compiz` 순서로 합쳐집니다.
`Writeback(output) pipeline`의 일반 경로는 `Compiz → optional Scaler-n → Wb_layer`입니다. 분할 writeback에서는 `Compiz → Splitter` 뒤에 `Scaler-0`과 `Scaler-1`이 병렬로 놓이고, `Merger`가 두 결과를 합쳐 `Wb_layer`로 보냅니다.
`Display output pipeline`은 `Compiz → Improc → Timing Controller`로 이어집니다. Improc가 화면 특성에 맞게 frame을 보정하고 Timing Controller가 출력 timing을 생성합니다.
Komeda data flow를 KMS 객체 역할에 대응시킵니다.
일반 입력 경로입니다.
수평 폭 한계를 넘는 입력의 병렬 처리 경로입니다.
일반 writeback 경로입니다.
두 scaler를 사용하는 writeback 경로입니다.
Monitor 출력으로 이어지는 최종 경로입니다.
----------------------------------
A complete display pipeline can be easily divided into three sub-pipelines
according to the in/out usage.
Layer(input) pipeline
~~~~~~~~~~~~~~~~~~~~~
.. kernel-render:: DOT
:alt: Layer data digraph
:caption: Layer (input) data flow
digraph layer_data_flow {
rankdir=LR;
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Scaler-n";
}
"Layer-n" -> "Scaler-n" -> "Compiz"
}
.. kernel-render:: DOT
:alt: Layer Split digraph
:caption: Layer Split pipeline
digraph layer_data_flow {
rankdir=LR;
node [shape=box]
"Layer-0/1" -> "Scaler-0" -> "Merger"
"Layer-2/3" -> "Scaler-1" -> "Merger"
"Merger" -> "Compiz"
}
Writeback(output) pipeline
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. kernel-render:: DOT
:alt: writeback digraph
:caption: Writeback(output) data flow
digraph writeback_data_flow {
rankdir=LR;
node [shape=box]
{
node [bgcolor=grey style=dashed]
"Scaler-n";
}
"Compiz" -> "Scaler-n" -> "Wb_layer"
}
.. kernel-render:: DOT
:alt: split writeback digraph
:caption: Writeback(output) Split data flow
digraph writeback_data_flow {
rankdir=LR;
node [shape=box]
"Compiz" -> "Splitter"
"Splitter" -> "Scaler-0" -> "Merger"
"Splitter" -> "Scaler-1" -> "Merger"
"Merger" -> "Wb_layer"
}
Display output pipeline
~~~~~~~~~~~~~~~~~~~~~~~
.. kernel-render:: DOT
:alt: display digraph
:caption: display output data flow
digraph single_ppl {
rankdir=LR;
node [shape=box]
"Compiz" -> "Improc" -> "Timing Controller"
}
In the following section we'll see these three sub-pipelines will be handled
by KMS-plane/wb_conn/crtc respectively.
Komeda 리소스 추상화·발견·format 처리
298-350하드웨어를 충분히 활용하고 쉽게 접근·설정하기 위해 드라이버도 하드웨어와 같은 `Pipeline/Component` 구조를 사용합니다. 각 component는 data flow 제어와 해당 component 고유의 capability·feature라는 두 부분을 포함합니다.
공통 header 구조체 `struct komeda_component`가 data flow 제어를 기술하고, 구체적인 component는 이 base structure의 subclass가 됩니다. 내부 API 문서는 `drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h`에서 생성합니다.
Pipeline과 component가 pixel data 처리 방법을 나타낸다면, `struct komeda_dev`는 장치 전체 모습과 장치 수준의 제어 능력을 나타냅니다. `komeda_dev`에 pipeline을 채우되 D71 이후 제품 간 코드를 최대한 공유하도록 장치를 `CORE`와 `CHIP` 두 계층으로 나눕니다.
`CORE`는 공통 feature와 capability 처리를 맡고, `CHIP`은 register programming과 하드웨어별 feature·limitation을 처리합니다. CORE는 `struct komeda_dev_funcs`, `struct komeda_pipeline_funcs`, `struct komeda_component_funcs`라는 세 chip function structure를 통해 CHIP에 접근합니다. 장치 내부 API는 `komeda_dev.h`에서 생성합니다.
Format capability와 framebuffer 처리는 각각 `drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h`와 `drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.h`의 내부 `kernel-doc`에서 가져옵니다.
하드웨어 구성요소에서 제품별 구현으로 내려가는 계층입니다.
공통 정책과 제품별 프로그래밍의 분리입니다.
Komeda Resource abstraction
===========================
struct komeda_pipeline/component
--------------------------------
To fully utilize and easily access/configure the HW, the driver side also uses
a similar architecture: Pipeline/Component to describe the HW features and
capabilities, and a specific component includes two parts:
- Data flow controlling.
- Specific component capabilities and features.
So the driver defines a common header struct komeda_component to describe the
data flow control and all specific components are a subclass of this base
structure.
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h
:internal:
Resource discovery and initialization
=====================================
Pipeline and component are used to describe how to handle the pixel data. We
still need a @struct komeda_dev to describe the whole view of the device, and
the control-abilites of device.
We have &komeda_dev, &komeda_pipeline, &komeda_component. Now fill devices with
pipelines. Since komeda is not for D71 only but also intended for later products,
of course we’d better share as much as possible between different products. To
achieve this, split the komeda device into two layers: CORE and CHIP.
- CORE: for common features and capabilities handling.
- CHIP: for register programming and HW specific feature (limitation) handling.
CORE can access CHIP by three chip function structures:
- struct komeda_dev_funcs
- struct komeda_pipeline_funcs
- struct komeda_component_funcs
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_dev.h
:internal:
Format handling
===============
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h
:internal:
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.h
:internal:
Attach komeda_dev to DRM-KMS
komeda_dev를 DRM-KMS에 연결
351-401Komeda는 리소스를 pipeline/component로 추상화하지만 DRM-KMS는 `crtc/plane/connector` 객체를 사용합니다. KMS 객체 하나의 요구사항은 component 하나만으로 충족되지 않는 경우가 많습니다. 예를 들어 CRTC의 mode, gamma, CTM 설정에는 `compiz`, `improc`, `timing_ctrlr`가 함께 필요하고, KMS Plane도 `layer/scaler/compiz` 여러 리소스를 요구할 수 있습니다.
따라서 KMS 객체 하나는 Komeda 리소스의 sub-pipeline 하나를 나타냅니다. Plane은 `Layer(input) pipeline`, `Wb_connector`는 `Writeback(output) pipeline`, Crtc는 `Display output pipeline`의 사용자입니다.
Komeda는 KMS `crtc/plane/connector`를 pipeline과 component의 사용자로 취급합니다. 한 시점에 pipeline/component는 사용자 하나에게만 할당할 수 있습니다. 이 리소스들을 DRM-KMS의 private object로 취급하고, 상태도 `drm_atomic_state`가 함께 관리합니다.
Plane을 고정 Layer pipeline에 묶는 방식은 단순하지만 Layer Split과 slave pipeline의 제약을 userspace에 노출합니다. Layer Split은 큰 이미지를 두 Layer와 두 scaler로 처리하면서 중앙 경계 artifact를 피하기 위한 복잡한 split 계산과 특수 설정이 필요하므로 드라이버가 숨겨야 합니다.
또한 `compiz`가 alpha 값을 출력하지 않으므로 slave pipeline은 bottom layer 합성에만 사용할 수 있습니다. 드라이버는 `plane_state->zpos`에 따라 적절한 Layer를 선택하여 이 한계를 userspace에서 감춥니다.
결국 KMS Plane은 고정된 Komeda layer pipeline 하나가 아니라 같은 capability를 가진 여러 Layer를 대표합니다. Komeda는 Plane 하나의 요구사항을 충족하도록 실행 시점에 Layer 하나 이상을 선택합니다.
하나의 KMS 객체가 여러 component 조합을 대표합니다.
하드웨어 제약을 userspace에 노출하지 않는 매핑 절차입니다.
============================
Komeda abstracts resources by pipeline/component, but DRM-KMS uses
crtc/plane/connector. One KMS-obj cannot represent only one single component,
since the requirements of a single KMS object cannot simply be achieved by a
single component, usually that needs multiple components to fit the requirement.
Like set mode, gamma, ctm for KMS all target on CRTC-obj, but komeda needs
compiz, improc and timing_ctrlr to work together to fit these requirements.
And a KMS-Plane may require multiple komeda resources: layer/scaler/compiz.
So, one KMS-Obj represents a sub-pipeline of komeda resources.
- Plane: `Layer(input) pipeline`_
- Wb_connector: `Writeback(output) pipeline`_
- Crtc: `Display output pipeline`_
So, for komeda, we treat KMS crtc/plane/connector as users of pipeline and
component, and at any one time a pipeline/component only can be used by one
user. And pipeline/component will be treated as private object of DRM-KMS; the
state will be managed by drm_atomic_state as well.
How to map plane to Layer(input) pipeline
-----------------------------------------
Komeda has multiple Layer input pipelines, see:
- `Single pipeline data flow`_
- `Dual pipeline with Slave enabled`_
The easiest way is binding a plane to a fixed Layer pipeline, but consider the
komeda capabilities:
- Layer Split, See `Layer(input) pipeline`_
Layer_Split is quite complicated feature, which splits a big image into two
parts and handles it by two layers and two scalers individually. But it
imports an edge problem or effect in the middle of the image after the split.
To avoid such a problem, it needs a complicated Split calculation and some
special configurations to the layer and scaler. We'd better hide such HW
related complexity to user mode.
- Slave pipeline, See `Dual pipeline with Slave enabled`_
Since the compiz component doesn't output alpha value, the slave pipeline
only can be used for bottom layers composition. The komeda driver wants to
hide this limitation to the user. The way to do this is to pick a suitable
Layer according to plane_state->zpos.
So for komeda, the KMS-plane doesn't represent a fixed komeda layer pipeline,
but multiple Layers with same capabilities. Komeda will select one or more
Layers to fit the requirement of one KMS-plane.
DRM private object와 atomic state 추적
402-438Component와 pipeline을 DRM atomic state에 포함하기 위해 `komeda_component`와 `komeda_pipeline`에 `struct drm_private_obj obj`를 추가합니다. 이 객체는 일반 KMS 객체 밖에 있는 driver-private 리소스도 atomic transaction의 일부로 관리하게 합니다.
상태 구조체에는 `struct drm_private_state obj`와 사용자를 기록합니다. `komeda_component_state`의 `void *binding_user`는 component를 현재 점유한 사용자를 나타내고, `komeda_pipeline_state`의 `struct drm_crtc *crtc`는 pipeline이 연결된 CRTC를 나타냅니다.
원문의 두 C 코드 블록은 구조체 배치를 그대로 제시합니다. `drm_private_obj`와 `drm_private_state`를 통해 component/pipeline 상태의 duplicate·swap·rollback이 `drm_atomic_state` 수명 주기와 함께 이루어집니다.
리소스와 transaction 상태의 연결점입니다.
KMS 사용자와 Komeda private resource를 한 transaction으로 묶습니다.
Make component/pipeline to be drm_private_obj
---------------------------------------------
Add :c:type:`drm_private_obj` to :c:type:`komeda_component`, :c:type:`komeda_pipeline`
.. code-block:: c
struct komeda_component {
struct drm_private_obj obj;
...
}
struct komeda_pipeline {
struct drm_private_obj obj;
...
}
Tracking component_state/pipeline_state by drm_atomic_state
-----------------------------------------------------------
Add :c:type:`drm_private_state` and user to :c:type:`komeda_component_state`,
:c:type:`komeda_pipeline_state`
.. code-block:: c
struct komeda_component_state {
struct drm_private_state obj;
void *binding_user;
...
}
struct komeda_pipeline_state {
struct drm_private_state obj;
struct drm_crtc *crtc;
...
}
Component validation과 komeda_kms API
439-477Komeda에는 여러 종류의 component가 있지만 validation 절차는 공통적입니다. 예시 `komeda_xxxx_validate()`는 component, 입력 data flow, `drm_plane/crtc/connector` 사용자와 해당 user state를 받습니다.
먼저 component가 필요한지 확인합니다. Scaler처럼 user state에 따라 선택적인 component가 필요 없으면 즉시 반환하고 caller가 data flow를 다음 단계로 전달합니다. 필요하다면 user state 요구사항을 component feature·capability와 비교하여 충족할 수 없는 경우 실패합니다.
그다음 `drm_atomic_state`에서 component state를 가져와 사용자를 할당합니다. 이미 다른 사용자가 점유했다면 실패합니다. 할당에 성공하면 입력 component를 기록하고 user state를 component 고유 state로 변환한 뒤, `input_dflow`를 다음 단계가 사용할 형태로 조정합니다.
원문 pseudocode에는 `Setup 3` 번호가 두 번 등장하지만 의미상 첫 번째는 사용자 예약, 두 번째는 component state 설정입니다. 원문의 번호와 코드는 변경하지 않고 보존합니다.
Komeda KMS 추상화의 내부 문서는 `drivers/gpu/drm/arm/display/komeda/komeda_kms.h`에서, CRTC와 Plane 함수 문서는 각각 `komeda_crtc.c`, `komeda_plane.c`에서 생성합니다. 원문 제목의 `komde_kms Functions` 철자도 source block에 그대로 보존됩니다.
원문 pseudocode의 검증·예약·설정 순서입니다.
kernel-doc이 추출하는 abstraction과 함수 구현입니다.
komeda component validation
---------------------------
Komeda has multiple types of components, but the process of validation are
similar, usually including the following steps:
.. code-block:: c
int komeda_xxxx_validate(struct komeda_component_xxx xxx_comp,
struct komeda_component_output *input_dflow,
struct drm_plane/crtc/connector *user,
struct drm_plane/crtc/connector_state, *user_state)
{
setup 1: check if component is needed, like the scaler is optional depending
on the user_state; if unneeded, just return, and the caller will
put the data flow into next stage.
Setup 2: check user_state with component features and capabilities to see
if requirements can be met; if not, return fail.
Setup 3: get component_state from drm_atomic_state, and try set to set
user to component; fail if component has been assigned to another
user already.
Setup 3: configure the component_state, like set its input component,
convert user_state to component specific state.
Setup 4: adjust the input_dflow and prepare it for the next stage.
}
komeda_kms Abstraction
----------------------
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_kms.h
:internal:
komde_kms Functions
-------------------
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_crtc.c
:internal:
.. kernel-doc:: drivers/gpu/drm/arm/display/komeda/komeda_plane.c
:internal:
Linux module driver 계층
478-488최종 구조에는 두 수준의 device가 있습니다. `komeda_dev`는 실제 display hardware를 기술하고, `komeda_kms_dev`는 `komeda_dev`를 DRM-KMS에 연결합니다.
모든 Komeda 동작은 `komeda_dev` 또는 `komeda_kms_dev`가 제공하거나 수행합니다. Linux module driver 자체는 `probe/remove/pm` 명령을 두 device 계층으로 전달하는 단순한 wrapper입니다.
Module wrapper와 두 device abstraction의 역할 분리입니다.
하드웨어 표현과 KMS 연결을 분리합니다.
Build komeda to be a Linux module driver
========================================
Now we have two level devices:
- komeda_dev: describes the real display hardware.
- komeda_kms_dev: attaches or connects komeda_dev to DRM-KMS.
All komeda operations are supplied or operated by komeda_dev or komeda_kms_dev,
the module driver is only a simple wrapper to pass the Linux command
(probe/remove/pm) into komeda_dev or komeda_kms_dev.
요약·해설
komeda-kms.rst:1-488Arm D71 이후 Komeda display hardware의 component pipeline과 이를 DRM-KMS Plane·writeback connector·CRTC에 원자적으로 매핑하는 설계를 설명합니다.
하드웨어 data flow에서 Linux module 경계까지의 구조입니다.
Pixel 처리와 KMS resource 관리의 두 축입니다.