요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
:orphan:
==============
MSM Preemption
==============
Preemption allows Adreno GPUs to switch to a higher priority ring when work is
pushed to it, reducing latency for high priority submissions.
When preemption is enabled 4 rings are initialized, corresponding to different
priority levels. Having multiple rings is purely a software concept as the GPU
only has registers to keep track of one graphics ring.
The kernel is able to switch which ring is currently being processed by
requesting preemption. When certain conditions are met, depending on the
priority level, the GPU will save its current state in a series of buffers,
then restores state from a similar set of buffers specified by the kernel. It
then resumes execution and fires an IRQ to let the kernel know the context
switch has completed.
This mechanism can be used by the kernel to switch between rings. Whenever a
submission occurs the kernel finds the highest priority ring which isn't empty
and preempts to it if said ring is not the one being currently executed. This is
also done whenever a submission completes to make sure execution resumes on a
lower priority ring when a higher priority ring is done.
Preemption levels
-----------------
Preemption can only occur at certain boundaries. The exact conditions can be
configured by changing the preemption level, this allows to compromise between
latency (ie. the time that passes between when the kernel requests preemption
and when the SQE begins saving state) and overhead (the amount of state that
needs to be saved).
The GPU offers 3 levels:
Level 0
Preemption only occurs at the submission level. This requires the least amount
of state to be saved as the execution of userspace submitted IBs is never
interrupted, however it offers very little benefit compared to not enabling
preemption of any kind.
Level 1
Preemption occurs at either bin level, if using GMEM rendering, or draw level
in the sysmem rendering case.
Level 2
Preemption occurs at draw level.
Level 1 is the mode that is used by the msm driver.
Additionally the GPU allows to specify a `skip_save_restore` option. This
disables the saving and restoring of all registers except those relating to the
operation of the SQE itself, reducing overhead. Saving and restoring is only
skipped when using GMEM with Level 1 preemption. When enabling this userspace is
expected to set the state that isn't preserved whenever preemption occurs which
is done by specifying preamble and postambles. Those are IBs that are executed
before and after preemption.
Preemption buffers
------------------
A series of buffers are necessary to store the state of rings while they are not
being executed. There are different kinds of preemption records and most of
those require one buffer per ring. This is because preemption never occurs
between submissions on the same ring, which always run in sequence when the ring
is active. This means that only one context per ring is effectively active.
SMMU_INFO
This buffer contains info about the current SMMU configuration such as the
ttbr0 register. The SQE firmware isn't actually able to save this record.
As a result SMMU info must be saved manually from the CP to a buffer and the
SMMU record updated with info from said buffer before triggering
preemption.
NON_SECURE
This is the main preemption record where most state is saved. It is mostly
opaque to the kernel except for the first few words that must be initialized
by the kernel.
SECURE
This saves state related to the GPU's secure mode.
NON_PRIV
The intended purpose of this record is unknown. The SQE firmware actually
ignores it and therefore msm doesn't handle it.
COUNTER
This record is used to save and restore performance counters.
Handling the permissions of those buffers is critical for security. All but the
NON_PRIV records need to be inaccessible from userspace, so they must be mapped
in the kernel address space with the MSM_BO_MAP_PRIV flag.
For example, making the NON_SECURE record accessible from userspace would allow
any process to manipulate a saved ring's RPTR which can be used to skip the
execution of some packets in a ring and execute user commands with higher
privileges.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
Adreno ring preemption
1-29Preemption은 우선순위가 높은 ring에 work가 들어오면 Adreno GPU가 그 ring으로 전환하게 하여 high-priority submission의 latency를 줄입니다. 원문 SPDX 라이선스는 `GPL-2.0`입니다.
Preemption이 활성화되면 서로 다른 priority level에 대응하는 ring 네 개를 초기화합니다. 여러 ring은 순수한 software 개념이며 GPU에는 graphics ring 하나의 상태만 추적할 register가 있습니다.
Kernel이 preemption을 요청하고 priority에 따른 조건이 충족되면 GPU는 현재 state를 여러 buffer에 저장합니다. 이어서 kernel이 지정한 비슷한 buffer 집합에서 state를 복원하고 execution을 재개한 뒤 context switch 완료를 알리는 IRQ를 발생시킵니다.
Submission이 들어올 때 kernel은 비어 있지 않은 ring 중 priority가 가장 높은 것을 찾습니다. 그 ring이 현재 실행 중인 ring이 아니면 preempt합니다. Submission 완료 시에도 같은 판단을 수행하여 높은 priority ring이 끝난 뒤 낮은 priority ring의 실행이 재개되도록 합니다.
Software ring 선택부터 GPU context switch 완료까지의 흐름입니다.
Kernel이 ring 우선순위를 다시 평가하는 두 시점입니다.
.. SPDX-License-Identifier: GPL-2.0
:orphan:
==============
MSM Preemption
==============
Preemption allows Adreno GPUs to switch to a higher priority ring when work is
pushed to it, reducing latency for high priority submissions.
When preemption is enabled 4 rings are initialized, corresponding to different
priority levels. Having multiple rings is purely a software concept as the GPU
only has registers to keep track of one graphics ring.
The kernel is able to switch which ring is currently being processed by
requesting preemption. When certain conditions are met, depending on the
priority level, the GPU will save its current state in a series of buffers,
then restores state from a similar set of buffers specified by the kernel. It
then resumes execution and fires an IRQ to let the kernel know the context
switch has completed.
This mechanism can be used by the kernel to switch between rings. Whenever a
submission occurs the kernel finds the highest priority ring which isn't empty
and preempts to it if said ring is not the one being currently executed. This is
also done whenever a submission completes to make sure execution resumes on a
lower priority ring when a higher priority ring is done.
Preemption levels
-----------------Preemption level과 save/restore 절충
30-64Preemption은 정해진 boundary에서만 일어납니다. Preemption level을 바꾸면 kernel 요청부터 SQE가 state 저장을 시작할 때까지의 latency와 저장해야 할 state 양인 overhead 사이를 절충할 수 있습니다.
Level 0은 submission boundary에서만 preempt합니다. Userspace가 제출한 IB 실행을 중단하지 않아 저장할 state가 가장 적지만, preemption을 끈 경우와 비교해 이점도 매우 작습니다.
Level 1은 GMEM rendering에서는 bin level, sysmem rendering에서는 draw level에서 preempt합니다. Level 2는 draw level에서 preempt합니다. MSM 드라이버가 사용하는 mode는 Level 1입니다.
GPU의 `skip_save_restore` option은 SQE 자체 동작과 관련된 register를 제외한 모든 register의 저장·복원을 끄므로 overhead를 줄입니다. 이 생략은 GMEM과 Level 1 preemption을 함께 사용할 때만 적용됩니다.
이 option을 켜면 userspace는 preemption 때 보존되지 않은 state를 다시 설정해야 합니다. 이를 위해 preemption 전후에 실행되는 IB인 preamble과 postamble을 지정합니다.
Boundary와 state 저장 비용을 비교합니다.
Overhead 절감과 userspace 책임의 관계입니다.
Preemption can only occur at certain boundaries. The exact conditions can be
configured by changing the preemption level, this allows to compromise between
latency (ie. the time that passes between when the kernel requests preemption
and when the SQE begins saving state) and overhead (the amount of state that
needs to be saved).
The GPU offers 3 levels:
Level 0
Preemption only occurs at the submission level. This requires the least amount
of state to be saved as the execution of userspace submitted IBs is never
interrupted, however it offers very little benefit compared to not enabling
preemption of any kind.
Level 1
Preemption occurs at either bin level, if using GMEM rendering, or draw level
in the sysmem rendering case.
Level 2
Preemption occurs at draw level.
Level 1 is the mode that is used by the msm driver.
Additionally the GPU allows to specify a `skip_save_restore` option. This
disables the saving and restoring of all registers except those relating to the
operation of the SQE itself, reducing overhead. Saving and restoring is only
skipped when using GMEM with Level 1 preemption. When enabling this userspace is
expected to set the state that isn't preserved whenever preemption occurs which
is done by specifying preamble and postambles. Those are IBs that are executed
before and after preemption.
Preemption buffers
------------------
Preemption record와 보안
65-99실행 중이 아닌 ring의 state를 보관하려면 여러 buffer가 필요합니다. Preemption record 종류 대부분은 ring마다 buffer 하나가 필요합니다. 같은 ring의 submission 사이에서는 preemption이 일어나지 않고 ring이 active일 때 항상 순서대로 실행되므로, 실질적으로 ring마다 active context는 하나뿐입니다.
`SMMU_INFO`에는 `ttbr0` 같은 현재 SMMU 설정이 들어갑니다. SQE firmware는 이 record를 직접 저장할 수 없으므로, preemption을 trigger하기 전에 CP가 SMMU 정보를 별도 buffer에 수동 저장하고 그 정보로 SMMU record를 갱신해야 합니다.
`NON_SECURE`는 대부분의 state가 저장되는 주 preemption record입니다. Kernel이 초기화해야 하는 처음 몇 word를 제외하면 kernel에는 거의 opaque합니다. `SECURE`는 GPU secure mode 관련 state를 저장합니다.
`NON_PRIV`의 의도는 알려져 있지 않습니다. SQE firmware가 이를 무시하므로 MSM도 처리하지 않습니다. `COUNTER` record는 performance counter를 저장하고 복원합니다.
이 buffer들의 permission 처리는 보안상 매우 중요합니다. `NON_PRIV`를 제외한 모든 record는 userspace가 접근할 수 없어야 하므로 `MSM_BO_MAP_PRIV` flag로 kernel address space에 mapping해야 합니다.
예를 들어 userspace가 `NON_SECURE` record에 접근하면 어떤 process든 저장된 ring의 `RPTR`을 조작할 수 있습니다. 그러면 ring packet 일부를 건너뛰고 더 높은 privilege로 user command를 실행할 수 있으므로 반드시 차단해야 합니다.
각 buffer에 저장되는 상태와 처리 주체입니다.
Privileged state를 userspace 조작에서 보호합니다.
A series of buffers are necessary to store the state of rings while they are not
being executed. There are different kinds of preemption records and most of
those require one buffer per ring. This is because preemption never occurs
between submissions on the same ring, which always run in sequence when the ring
is active. This means that only one context per ring is effectively active.
SMMU_INFO
This buffer contains info about the current SMMU configuration such as the
ttbr0 register. The SQE firmware isn't actually able to save this record.
As a result SMMU info must be saved manually from the CP to a buffer and the
SMMU record updated with info from said buffer before triggering
preemption.
NON_SECURE
This is the main preemption record where most state is saved. It is mostly
opaque to the kernel except for the first few words that must be initialized
by the kernel.
SECURE
This saves state related to the GPU's secure mode.
NON_PRIV
The intended purpose of this record is unknown. The SQE firmware actually
ignores it and therefore msm doesn't handle it.
COUNTER
This record is used to save and restore performance counters.
Handling the permissions of those buffers is critical for security. All but the
NON_PRIV records need to be inaccessible from userspace, so they must be mapped
in the kernel address space with the MSM_BO_MAP_PRIV flag.
For example, making the NON_SECURE record accessible from userspace would allow
any process to manipulate a saved ring's RPTR which can be used to skip the
execution of some packets in a ring and execute user commands with higher
privileges.
요약·해설
msm-preemption.rst:1-99Adreno priority ring 전환, preemption level, state record와 보안 요건을 설명합니다.
이 문서가 다루는 주요 항목입니다.