요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
1
============================
2
Kernel-provided User Helpers
3
============================
5
These are segment of kernel provided user code reachable from user space
6
at a fixed address in kernel memory. This is used to provide user space
7
with some operations which require kernel help because of unimplemented
8
native feature and/or instructions in many ARM CPUs. The idea is for this
9
code to be executed directly in user mode for best efficiency but which is
10
too intimate with the kernel counter part to be left to user libraries.
11
In fact this code might even differ from one CPU to another depending on
12
the available instruction set, or whether it is a SMP systems. In other
13
words, the kernel reserves the right to change this code as needed without
14
warning. Only the entry points and their results as documented here are
15
guaranteed to be stable.
17
This is different from (but doesn't preclude) a full blown VDSO
18
implementation, however a VDSO would prevent some assembly tricks with
19
constants that allows for efficient branching to those code segments. And
20
since those code segments only use a few cycles before returning to user
21
code, the overhead of a VDSO indirect far call would add a measurable
22
overhead to such minimalistic operations.
24
User space is expected to bypass those helpers and implement those things
25
inline (either in the code emitted directly by the compiler, or part of
26
the implementation of a library call) when optimizing for a recent enough
27
processor that has the necessary native support, but only if resulting
28
binaries are already to be incompatible with earlier ARM processors due to
29
usage of similar native instructions for other things. In other words
30
don't make binaries unable to run on earlier processors just for the sake
31
of not using these kernel helpers if your compiled code is not going to
32
use new instructions for other purpose.
34
New helpers may be added over time, so an older kernel may be missing some
35
helpers present in a newer kernel. For this reason, programs must check
36
the value of __kuser_helper_version (see below) before assuming that it is
37
safe to call any particular helper. This check should ideally be
38
performed only once at process startup time, and execution aborted early
39
if the required helpers are not provided by the kernel version that
40
process is running on.
42
kuser_helper_version
43
--------------------
45
Location: 0xffff0ffc
47
Reference declaration::
49
extern int32_t __kuser_helper_version;
51
Definition:
53
This field contains the number of helpers being implemented by the
54
running kernel. User space may read this to determine the availability
55
of a particular helper.
57
Usage example::
59
#define __kuser_helper_version (*(int32_t *)0xffff0ffc)
61
void check_kuser_version(void)
62
{
63
if (__kuser_helper_version < 2) {
64
fprintf(stderr, "can't do atomic operations, kernel too old\n");
65
abort();
66
}
67
}
69
Notes:
71
User space may assume that the value of this field never changes
72
during the lifetime of any single process. This means that this
73
field can be read once during the initialisation of a library or
74
startup phase of a program.
76
kuser_get_tls
77
-------------
79
Location: 0xffff0fe0
81
Reference prototype::
83
void * __kuser_get_tls(void);
85
Input:
87
lr = return address
89
Output:
91
r0 = TLS value
93
Clobbered registers:
95
none
97
Definition:
99
Get the TLS value as previously set via the __ARM_NR_set_tls syscall.
101
Usage example::
103
typedef void * (__kuser_get_tls_t)(void);
104
#define __kuser_get_tls (*(__kuser_get_tls_t *)0xffff0fe0)
106
void foo()
107
{
108
void *tls = __kuser_get_tls();
109
printf("TLS = %p\n", tls);
110
}
112
Notes:
114
- Valid only if __kuser_helper_version >= 1 (from kernel version 2.6.12).
116
kuser_cmpxchg
117
-------------
119
Location: 0xffff0fc0
121
Reference prototype::
123
int __kuser_cmpxchg(int32_t oldval, int32_t newval, volatile int32_t *ptr);
125
Input:
127
r0 = oldval
128
r1 = newval
129
r2 = ptr
130
lr = return address
132
Output:
134
r0 = success code (zero or non-zero)
135
C flag = set if r0 == 0, clear if r0 != 0
137
Clobbered registers:
139
r3, ip, flags
141
Definition:
143
Atomically store newval in `*ptr` only if `*ptr` is equal to oldval.
144
Return zero if `*ptr` was changed or non-zero if no exchange happened.
145
The C flag is also set if `*ptr` was changed to allow for assembly
146
optimization in the calling code.
148
Usage example::
150
typedef int (__kuser_cmpxchg_t)(int oldval, int newval, volatile int *ptr);
151
#define __kuser_cmpxchg (*(__kuser_cmpxchg_t *)0xffff0fc0)
153
int atomic_add(volatile int *ptr, int val)
154
{
155
int old, new;
157
do {
158
old = *ptr;
159
new = old + val;
160
} while(__kuser_cmpxchg(old, new, ptr));
162
return new;
163
}
165
Notes:
167
- This routine already includes memory barriers as needed.
169
- Valid only if __kuser_helper_version >= 2 (from kernel version 2.6.12).
171
kuser_memory_barrier
172
--------------------
174
Location: 0xffff0fa0
176
Reference prototype::
178
void __kuser_memory_barrier(void);
180
Input:
182
lr = return address
184
Output:
186
none
188
Clobbered registers:
190
none
192
Definition:
194
Apply any needed memory barrier to preserve consistency with data modified
195
manually and __kuser_cmpxchg usage.
197
Usage example::
199
typedef void (__kuser_dmb_t)(void);
200
#define __kuser_dmb (*(__kuser_dmb_t *)0xffff0fa0)
202
Notes:
204
- Valid only if __kuser_helper_version >= 3 (from kernel version 2.6.15).
206
kuser_cmpxchg64
207
---------------
209
Location: 0xffff0f60
211
Reference prototype::
213
int __kuser_cmpxchg64(const int64_t *oldval,
214
const int64_t *newval,
215
volatile int64_t *ptr);
217
Input:
219
r0 = pointer to oldval
220
r1 = pointer to newval
221
r2 = pointer to target value
222
lr = return address
224
Output:
226
r0 = success code (zero or non-zero)
227
C flag = set if r0 == 0, clear if r0 != 0
229
Clobbered registers:
231
r3, lr, flags
233
Definition:
235
Atomically store the 64-bit value pointed by `*newval` in `*ptr` only if `*ptr`
236
is equal to the 64-bit value pointed by `*oldval`. Return zero if `*ptr` was
237
changed or non-zero if no exchange happened.
239
The C flag is also set if `*ptr` was changed to allow for assembly
240
optimization in the calling code.
242
Usage example::
244
typedef int (__kuser_cmpxchg64_t)(const int64_t *oldval,
245
const int64_t *newval,
246
volatile int64_t *ptr);
247
#define __kuser_cmpxchg64 (*(__kuser_cmpxchg64_t *)0xffff0f60)
249
int64_t atomic_add64(volatile int64_t *ptr, int64_t val)
250
{
251
int64_t old, new;
253
do {
254
old = *ptr;
255
new = old + val;
256
} while(__kuser_cmpxchg64(&old, &new, ptr));
258
return new;
259
}
261
Notes:
263
- This routine already includes memory barriers as needed.
265
- Due to the length of this sequence, this spans 2 conventional kuser
266
"slots", therefore 0xffff0f80 is not used as a valid entry point.
268
- Valid only if __kuser_helper_version >= 5 (from kernel version 3.1).
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
Kernel-provided User Helpers
1-41kernel memory의 고정 주소에 놓이고 userspace에서 접근할 수 있는 kernel 제공 user code segment입니다. 많은 ARM CPU에 native feature나 instruction이 없어 kernel 도움이 필요한 operation을 효율적으로 제공합니다.
이 code는 user mode에서 직접 실행되지만 kernel counterpart와 밀접해 user library에 맡기기 어렵습니다. CPU instruction set이나 SMP 여부에 따라 구현이 달라질 수 있으며 kernel은 예고 없이 code를 바꿀 수 있습니다. 여기 문서화된 entry point와 결과만 stable ABI로 보장합니다.
full VDSO 구현과는 다른 방식이며 VDSO를 배제하지도 않습니다. VDSO는 constant를 이용한 assembly branch 최적화를 막을 수 있고, helper 자체가 몇 cycle 만에 돌아오므로 indirect far call overhead도 측정 가능할 정도로 큽니다.
충분히 최신 processor를 target으로 하고 다른 native instruction 때문에 binary가 이미 구형 ARM과 호환되지 않는다면 compiler inline code나 library call 구현으로 helper를 우회할 수 있습니다. helper 하나를 피하려는 이유만으로 기존 processor 호환성을 버려서는 안 됩니다.
시간이 지나며 새 helper가 추가되므로 old kernel에는 helper가 없을 수 있습니다. program은 helper를 호출하기 전에 `__kuser_helper_version`을 확인해야 합니다. process startup에서 한 번 검사하고 필요한 helper가 없으면 일찍 종료하는 것이 좋습니다.
kuser_helper_version
42-75| 항목 | 값 |
|---|---|
| location | `0xffff0ffc` |
| symbol | `__kuser_helper_version` |
| 의미 | 실행 중인 kernel이 구현한 helper 수 |
extern int32_t __kuser_helper_version;userspace는 값을 읽어 특정 helper의 availability를 판단합니다. 다음 예는 version이 2보다 작으면 atomic operation을 제공하지 못하는 오래된 kernel로 보고 종료합니다.
#define __kuser_helper_version (*(int32_t *)0xffff0ffc)
void check_kuser_version(void)
{
if (__kuser_helper_version < 2) {
fprintf(stderr, "can't do atomic operations, kernel too old\n");
abort();
}
}한 process의 lifetime 동안 값은 바뀌지 않는다고 가정할 수 있으므로 library initialization 또는 program startup에서 한 번만 읽어도 됩니다.
kuser_get_tls
76-115| 항목 | 값 |
|---|---|
| location | `0xffff0fe0` |
| prototype | `void * __kuser_get_tls(void)` |
| input | `lr` = return address |
| output | `r0` = TLS value |
| clobbered | 없음 |
| version | `__kuser_helper_version >= 1`, kernel 2.6.12부터 |
void * __kuser_get_tls(void);`__ARM_NR_set_tls` syscall로 앞서 설정한 TLS value를 반환합니다. 고정 주소를 function pointer로 선언해 직접 호출할 수 있습니다.
typedef void * (__kuser_get_tls_t)(void);
#define __kuser_get_tls (*(__kuser_get_tls_t *)0xffff0fe0)
void foo()
{
void *tls = __kuser_get_tls();
printf("TLS = %p\n", tls);
}kuser_cmpxchg
116-170| 항목 | 값 |
|---|---|
| location | `0xffff0fc0` |
| prototype | `int __kuser_cmpxchg(int32_t oldval, int32_t newval, volatile int32_t *ptr)` |
| input | `r0=oldval`, `r1=newval`, `r2=ptr`, `lr=return address` |
| output | `r0` success code, 성공이면 0. C flag는 `r0 == 0`이면 set |
| clobbered | `r3`, `ip`, flags |
| version | `__kuser_helper_version >= 2`, kernel 2.6.12부터 |
int __kuser_cmpxchg(int32_t oldval, int32_t newval, volatile int32_t *ptr);`*ptr`이 `oldval`과 같을 때만 `newval`을 atomic하게 저장합니다. 교환했으면 0, 하지 않았으면 non-zero를 반환합니다. assembly caller 최적화를 위해 성공 시 C flag도 set합니다.
아래 `atomic_add()` 예는 현재 값을 읽고 새 값을 계산한 뒤 `__kuser_cmpxchg()`가 성공할 때까지 반복합니다.
typedef int (__kuser_cmpxchg_t)(int oldval, int newval, volatile int *ptr);
#define __kuser_cmpxchg (*(__kuser_cmpxchg_t *)0xffff0fc0)
int atomic_add(volatile int *ptr, int val)
{
int old, new;
do {
old = *ptr;
new = old + val;
} while(__kuser_cmpxchg(old, new, ptr));
return new;
}필요한 memory barriers는 routine 자체에 이미 포함되어 있습니다.
kuser_memory_barrier
171-205| 항목 | 값 |
|---|---|
| location | `0xffff0fa0` |
| prototype | `void __kuser_memory_barrier(void)` |
| input | `lr` = return address |
| output | 없음 |
| clobbered | 없음 |
| version | `__kuser_helper_version >= 3`, kernel 2.6.15부터 |
void __kuser_memory_barrier(void);직접 수정한 data와 `__kuser_cmpxchg` 사용 사이의 consistency를 보존하는 데 필요한 memory barrier를 적용합니다.
typedef void (__kuser_dmb_t)(void);
#define __kuser_dmb (*(__kuser_dmb_t *)0xffff0fa0)kuser_cmpxchg64
206-268| 항목 | 값 |
|---|---|
| location | `0xffff0f60` |
| prototype | `int __kuser_cmpxchg64(const int64_t *oldval, const int64_t *newval, volatile int64_t *ptr)` |
| input | `r0` = oldval pointer, `r1` = newval pointer, `r2` = target pointer, `lr` = return address |
| output | `r0` success code, 성공이면 0. C flag는 `r0 == 0`이면 set |
| clobbered | `r3`, `lr`, flags |
| version | `__kuser_helper_version >= 5`, kernel 3.1부터 |
int __kuser_cmpxchg64(const int64_t *oldval,
const int64_t *newval,
volatile int64_t *ptr);`*ptr`이 `*oldval`이 가리키는 64-bit value와 같을 때만 `*newval`의 64-bit value를 atomic하게 저장합니다. 바뀌었으면 0, 교환이 없으면 non-zero이며 성공 시 C flag도 set합니다.
`atomic_add64()` 예는 old/new 64-bit value의 주소를 전달해 성공할 때까지 compare-exchange를 반복합니다.
typedef int (__kuser_cmpxchg64_t)(const int64_t *oldval,
const int64_t *newval,
volatile int64_t *ptr);
#define __kuser_cmpxchg64 (*(__kuser_cmpxchg64_t *)0xffff0f60)
int64_t atomic_add64(volatile int64_t *ptr, int64_t val)
{
int64_t old, new;
do {
old = *ptr;
new = old + val;
} while(__kuser_cmpxchg64(&old, &new, ptr));
return new;
}routine은 필요한 memory barriers를 포함합니다. sequence가 길어 conventional kuser slot 두 개를 차지하므로 `0xffff0f80`은 유효한 entry point로 쓰지 않습니다.
요약과 해설
kernel_user_helpers.rst:1-268helper 구현 code는 kernel과 CPU에 따라 달라질 수 있지만 고정 entry address와 결과는 ABI입니다. process startup에서 version을 한 번 확인한 뒤 필요한 helper만 호출해야 합니다.
주소는 높은 helper version word에서 낮은 64-bit compare-exchange routine 순으로 배치됩니다.