The Virtual Stateless Decoder Driver (visl) | Linux 6.18.37

요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.

1. 요약·해설

원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.

visl debugging 흐름

visl.rst:1-185

visl은 실제 decode 결과 대신 관찰 가능한 trace와 buffer 정보를 만들어 userspace stateless decoder의 동작을 비교·검증합니다.

검증 대상	Interface	용도
Decode loop	V4L2 stateless API	hardware나 upstream codec uAPI 없이 userspace client 시험
Control trace	per-codec ftrace event	제출한 V4L2 control 내용 확인
Capture frame	V4L2 test pattern generator	reference·timestamp·queue 상태를 buffer에 기록
Bitstream dump	VISL_DEBUGFS	/sys/kernel/debug/visl/bitstream에서 OUTPUT buffer 검사

2. 영어 원문 전체

번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.

원문 전체 펼치기

1 .. SPDX-License-Identifier: GPL-2.0

3 The Virtual Stateless Decoder Driver (visl)

4 ===========================================

6 A virtual stateless decoder device for stateless uAPI development

7 purposes.

9 This tool's objective is to help the development and testing of

10 userspace applications that use the V4L2 stateless API to decode media.

12 A userspace implementation can use visl to run a decoding loop even when

13 no hardware is available or when the kernel uAPI for the codec has not

14 been upstreamed yet. This can reveal bugs at an early stage.

16 This driver can also trace the contents of the V4L2 controls submitted

17 to it. It can also dump the contents of the vb2 buffers through a

18 debugfs interface. This is in many ways similar to the tracing

19 infrastructure available for other popular encode/decode APIs out there

20 and can help develop a userspace application by using another (working)

21 one as a reference.

23 .. note::

25 No actual decoding of video frames is performed by visl. The

26 V4L2 test pattern generator is used to write various debug information

27 to the capture buffers instead.

29 Module parameters

30 -----------------

32 - visl_debug: Activates debug info, printing various debug messages through

33 dprintk. Also controls whether per-frame debug info is shown. Defaults to off.

34 Note that enabling this feature can result in slow performance through serial.

36 - visl_transtime_ms: Simulated process time in milliseconds. Slowing down the

37 decoding speed can be useful for debugging.

39 - visl_dprintk_frame_start, visl_dprintk_frame_nframes: Dictates a range of

40 frames where dprintk is activated. This only controls the dprintk tracing on a

41 per-frame basis. Note that printing a lot of data can be slow through serial.

43 - keep_bitstream_buffers: Controls whether bitstream (i.e. OUTPUT) buffers are

44 kept after a decoding session. Defaults to false so as to reduce the amount of

45 clutter. keep_bitstream_buffers == false works well when live debugging the

46 client program with GDB.

48 - bitstream_trace_frame_start, bitstream_trace_nframes: Similar to

49 visl_dprintk_frame_start, visl_dprintk_nframes, but controls the dumping of

50 buffer data through debugfs instead.

52 - tpg_verbose: Write extra information on each output frame to ease debugging

53 the API. When set to true, the output frames are not stable for a given input

54 as some information like pointers or queue status will be added to them.

56 What is the default use case for this driver?

57 ---------------------------------------------

59 This driver can be used as a way to compare different userspace implementations.

60 This assumes that a working client is run against visl and that the ftrace and

61 OUTPUT buffer data is subsequently used to debug a work-in-progress

62 implementation.

64 Even though no video decoding is actually done, the output frames can be used

65 against a reference for a given input, except if tpg_verbose is set to true.

67 Depending on the tpg_verbose parameter value, information on reference frames,

68 their timestamps, the status of the OUTPUT and CAPTURE queues and more can be

69 read directly from the CAPTURE buffers.

71 Supported codecs

72 ----------------

74 The following codecs are supported:

76 - FWHT

77 - MPEG2

78 - VP8

79 - VP9

80 - H.264

81 - HEVC

82 - AV1

84 visl trace events

85 -----------------

86 The trace events are defined on a per-codec basis, e.g.:

88 .. code-block:: bash

90 $ ls /sys/kernel/tracing/events/ | grep visl

91 visl_av1_controls

92 visl_fwht_controls

93 visl_h264_controls

94 visl_hevc_controls

95 visl_mpeg2_controls

96 visl_vp8_controls

97 visl_vp9_controls

99 For example, in order to dump HEVC SPS data:

100

101 .. code-block:: bash

102

103 $ echo 1 > /sys/kernel/tracing/events/visl_hevc_controls/v4l2_ctrl_hevc_sps/enable

104

105 The SPS data will be dumped to the trace buffer, i.e.:

106

107 .. code-block:: bash

108

109 $ cat /sys/kernel/tracing/trace

110 video_parameter_set_id 0

111 seq_parameter_set_id 0

112 pic_width_in_luma_samples 1920

113 pic_height_in_luma_samples 1080

114 bit_depth_luma_minus8 0

115 bit_depth_chroma_minus8 0

116 log2_max_pic_order_cnt_lsb_minus4 4

117 sps_max_dec_pic_buffering_minus1 6

118 sps_max_num_reorder_pics 2

119 sps_max_latency_increase_plus1 0

120 log2_min_luma_coding_block_size_minus3 0

121 log2_diff_max_min_luma_coding_block_size 3

122 log2_min_luma_transform_block_size_minus2 0

123 log2_diff_max_min_luma_transform_block_size 3

124 max_transform_hierarchy_depth_inter 2

125 max_transform_hierarchy_depth_intra 2

126 pcm_sample_bit_depth_luma_minus1 0

127 pcm_sample_bit_depth_chroma_minus1 0

128 log2_min_pcm_luma_coding_block_size_minus3 0

129 log2_diff_max_min_pcm_luma_coding_block_size 0

130 num_short_term_ref_pic_sets 0

131 num_long_term_ref_pics_sps 0

132 chroma_format_idc 1

133 sps_max_sub_layers_minus1 0

134 flags AMP_ENABLED|SAMPLE_ADAPTIVE_OFFSET|TEMPORAL_MVP_ENABLED|STRONG_INTRA_SMOOTHING_ENABLED

135

136

137 Dumping OUTPUT buffer data through debugfs

138 ------------------------------------------

139

140 If the **VISL_DEBUGFS** Kconfig is enabled, visl will populate

141 **/sys/kernel/debug/visl/bitstream** with OUTPUT buffer data according to the

142 values of bitstream_trace_frame_start and bitstream_trace_nframes. This can

143 highlight errors as broken clients may fail to fill the buffers properly.

144

145 A single file is created for each processed OUTPUT buffer. Its name contains an

146 integer that denotes the buffer sequence, i.e.:

147

148 .. code-block:: c

149

150 snprintf(name, 32, "bitstream%d", run->src->sequence);

151

152 Dumping the values is simply a matter of reading from the file, i.e.:

153

154 For the buffer with sequence == 0:

155

156 .. code-block:: bash

157

158 $ xxd /sys/kernel/debug/visl/bitstream/bitstream0

159 00000000: 2601 af04 d088 bc25 a173 0e41 a4f2 3274 &......%.s.A..2t

160 00000010: c668 cb28 e775 b4ac f53a ba60 f8fd 3aa1 .h.(.u...:.`..:.

161 00000020: 46b4 bcfc 506c e227 2372 e5f5 d7ea 579f F...Pl.'#r....W.

162 00000030: 6371 5eb5 0eb8 23b5 ca6a 5de5 983a 19e4 cq^...#..j]..:..

163 00000040: e8c3 4320 b4ba a226 cbc1 4138 3a12 32d6 ..C ...&..A8:.2.

164 00000050: fef3 247b 3523 4e90 9682 ac8e eb0c a389 ..${5#N.........

165 00000060: ddd0 6cfc 0187 0e20 7aae b15b 1812 3d33 ..l.... z..[..=3

166 00000070: e1c5 f425 a83a 00b7 4f18 8127 3c4c aefb ...%.:..O..'<L..

167

168 For the buffer with sequence == 1:

169

170 .. code-block:: bash

171

172 $ xxd /sys/kernel/debug/visl/bitstream/bitstream1

173 00000000: 0201 d021 49e1 0c40 aa11 1449 14a6 01dc [email protected]....

174 00000010: 7023 889a c8cd 2cd0 13b4 dab0 e8ca 21fe p#....,.......!.

175 00000020: c4c8 ab4c 486e 4e2f b0df 96cc c74e 8dde ...LHnN/.....N..

176 00000030: 8ce7 ee36 d880 4095 4d64 30a0 ff4f 0c5e [email protected].^

177 00000040: f16b a6a1 d806 ca2a 0ece a673 7bea 1f37 .k.....*...s{..7

178 00000050: 370f 5bb9 1dc4 ba21 6434 bc53 0173 cba0 7.[....!d4.S.s..

179 00000060: dfe6 bc99 01ea b6e0 346b 92b5 c8de 9f5d ........4k.....]

180 00000070: e7cc 3484 1769 fef2 a693 a945 2c8b 31da ..4..i.....E,.1.

181

182 And so on.

183

184 By default, the files are removed during STREAMOFF. This is to reduce the amount

185 of clutter.

186

3. 한국어 전문 번역

영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.

visl 개요

1-27

이 문서는 GPL-2.0 라이선스를 따릅니다. `visl`은 stateless uAPI 개발을 위한 virtual stateless decoder device입니다.

목표는 V4L2 stateless API로 media를 decode하는 userspace application의 개발과 test를 돕는 것입니다. Hardware가 없거나 codec용 kernel uAPI가 아직 upstream되지 않아도 decoding loop를 실행해 bug를 조기에 찾을 수 있습니다.

Driver는 제출된 `V4L2 controls` 내용을 trace하고 debugfs interface를 통해 `vb2 buffers` 내용을 dump할 수 있습니다. 다른 encode/decode API의 tracing infrastructure와 비슷하게 동작하므로 정상 동작하는 application을 reference로 삼아 다른 userspace application을 개발할 수 있습니다.

`visl`은 video frame을 실제로 decode하지 않습니다. 대신 V4L2 test pattern generator가 capture buffer에 여러 debug 정보를 기록합니다.

Module parameter

29-54

Parameter	기능	기본값·주의
visl_debug	debug message와 per-frame debug info를 dprintk로 출력	기본 off; serial 출력에서 성능 저하 가능
visl_transtime_ms	millisecond 단위 simulated processing time	decode 속도를 늦춰 debugging할 때 유용
visl_dprintk_frame_start, visl_dprintk_frame_nframes	per-frame dprintk가 활성화될 frame 범위	많은 serial 출력은 느릴 수 있음
keep_bitstream_buffers	decode session 뒤 bitstream(OUTPUT) buffer 보존 여부	기본 false; GDB live debugging에 적합
bitstream_trace_frame_start, bitstream_trace_nframes	debugfs로 buffer data를 dump할 frame 범위	dprintk frame range parameter와 유사
tpg_verbose	각 output frame에 API debugging용 추가 정보 기록	true이면 pointer·queue status 때문에 같은 input에도 frame이 달라짐

기본 사용 사례

56-69

이 driver는 서로 다른 userspace implementation을 비교하는 데 사용할 수 있습니다. 정상 client를 visl에서 실행한 뒤 ftrace와 OUTPUT buffer data로 개발 중 implementation을 debug한다고 가정합니다.

실제 video decoding은 하지 않지만 `tpg_verbose=true`가 아니라면 output frame을 주어진 input의 reference와 비교할 수 있습니다.

`tpg_verbose` 값에 따라 reference frame, timestamp, OUTPUT·CAPTURE queue 상태 등의 정보를 CAPTURE buffer에서 직접 읽을 수 있습니다.

지원 codec

71-82

지원하는 codec은 다음과 같습니다.

FWHT
MPEG2
VP8
VP9
H.264
HEVC
AV1

visl trace event

84-135

Trace event는 다음 예처럼 codec별로 정의됩니다.

$ ls /sys/kernel/tracing/events/ | grep visl
visl_av1_controls
visl_fwht_controls
visl_h264_controls
visl_hevc_controls
visl_mpeg2_controls
visl_vp8_controls
visl_vp9_controls

예를 들어 HEVC SPS data dump를 활성화하려면 다음 명령을 실행합니다.

$ echo 1 >  /sys/kernel/tracing/events/visl_hevc_controls/v4l2_ctrl_hevc_sps/enable

SPS data는 다음과 같이 trace buffer에 dump됩니다.

$ cat /sys/kernel/tracing/trace
video_parameter_set_id 0
seq_parameter_set_id 0
pic_width_in_luma_samples 1920
pic_height_in_luma_samples 1080
bit_depth_luma_minus8 0
bit_depth_chroma_minus8 0
log2_max_pic_order_cnt_lsb_minus4 4
sps_max_dec_pic_buffering_minus1 6
sps_max_num_reorder_pics 2
sps_max_latency_increase_plus1 0
log2_min_luma_coding_block_size_minus3 0
log2_diff_max_min_luma_coding_block_size 3
log2_min_luma_transform_block_size_minus2 0
log2_diff_max_min_luma_transform_block_size 3
max_transform_hierarchy_depth_inter 2
max_transform_hierarchy_depth_intra 2
pcm_sample_bit_depth_luma_minus1 0
pcm_sample_bit_depth_chroma_minus1 0
log2_min_pcm_luma_coding_block_size_minus3 0
log2_diff_max_min_pcm_luma_coding_block_size 0
num_short_term_ref_pic_sets 0
num_long_term_ref_pics_sps 0
chroma_format_idc 1
sps_max_sub_layers_minus1 0
flags AMP_ENABLED|SAMPLE_ADAPTIVE_OFFSET|TEMPORAL_MVP_ENABLED|STRONG_INTRA_SMOOTHING_ENABLED

debugfs OUTPUT buffer dump

137-185

`VISL_DEBUGFS` Kconfig가 활성화되면 visl은 `bitstream_trace_frame_start`와 `bitstream_trace_nframes` 값에 따라 `/sys/kernel/debug/visl/bitstream`을 OUTPUT buffer data로 채웁니다. 깨진 client가 buffer를 올바르게 채우지 못하는 오류를 찾는 데 유용합니다.

처리한 OUTPUT buffer마다 파일 하나를 만들며, 파일명에는 buffer sequence를 나타내는 integer가 들어갑니다.

snprintf(name, 32, "bitstream%d", run->src->sequence);

값은 파일을 읽기만 하면 확인할 수 있습니다. `sequence == 0`인 buffer의 예입니다.

$ xxd /sys/kernel/debug/visl/bitstream/bitstream0
00000000: 2601 af04 d088 bc25 a173 0e41 a4f2 3274  &......%.s.A..2t
00000010: c668 cb28 e775 b4ac f53a ba60 f8fd 3aa1  .h.(.u...:.`..:.
00000020: 46b4 bcfc 506c e227 2372 e5f5 d7ea 579f  F...Pl.'#r....W.
00000030: 6371 5eb5 0eb8 23b5 ca6a 5de5 983a 19e4  cq^...#..j]..:..
00000040: e8c3 4320 b4ba a226 cbc1 4138 3a12 32d6  ..C ...&..A8:.2.
00000050: fef3 247b 3523 4e90 9682 ac8e eb0c a389  ..${5#N.........
00000060: ddd0 6cfc 0187 0e20 7aae b15b 1812 3d33  ..l.... z..[..=3
00000070: e1c5 f425 a83a 00b7 4f18 8127 3c4c aefb  ...%.:..O..'<L..

`sequence == 1`인 buffer의 예입니다.

$ xxd /sys/kernel/debug/visl/bitstream/bitstream1
00000000: 0201 d021 49e1 0c40 aa11 1449 14a6 01dc  [email protected]....
00000010: 7023 889a c8cd 2cd0 13b4 dab0 e8ca 21fe  p#....,.......!.
00000020: c4c8 ab4c 486e 4e2f b0df 96cc c74e 8dde  ...LHnN/.....N..
00000030: 8ce7 ee36 d880 4095 4d64 30a0 ff4f 0c5e  [email protected].^
00000040: f16b a6a1 d806 ca2a 0ece a673 7bea 1f37  .k.....*...s{..7
00000050: 370f 5bb9 1dc4 ba21 6434 bc53 0173 cba0  7.[....!d4.S.s..
00000060: dfe6 bc99 01ea b6e0 346b 92b5 c8de 9f5d  ........4k.....]
00000070: e7cc 3484 1769 fef2 a693 a945 2c8b 31da  ..4..i.....E,.1.

이후 buffer도 같은 방식입니다. 기본적으로 file은 STREAMOFF 중 제거되어 불필요한 파일 누적을 줄입니다.