← Documents Documentation/admin-guide/media/ipu6-isys.rst GitHub 원문 ↗

Linux 6.18.37 · Administration / Media

Intel IPU6 Input System Driver

IPU6 ISYS의 multi-stream CSI-2 graph, 지원 format과 raw·metadata capture 구성을 설명합니다.

Source pathDocumentation/admin-guide/media/ipu6-isys.rst
Source versionLinux v6.18.37
TranslationDUJINLABS 전문 번역 + 해설

요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.

1. 요약·해설

원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.

IPU6 sensor-to-memory architecture

ipu6-isys.rst:1-161

IPU6 common driver와 ISYS V4L2 driver의 책임, 네 CSI-2 receiver와 32개 capture node의 graph, OV01A10 raw capture와 OV2740 image·metadata multi-stream route를 연결해 설명합니다.

IPU6 ISYS capture path
camera sensorIPU6 CSI-2 receiverISYS firmware streamDMA capture nodememory buffer

Sensor stream은 CSI-2 receiver의 route를 따라 firmware stream과 DMA capture node를 거쳐 memory buffer로 전달됩니다.

2. 영어 원문 전체

번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.

원문 전체 펼치기
1 .. SPDX-License-Identifier: GPL-2.0
2
3 .. include:: <isonum.txt>
4
5 ========================================================
6 Intel Image Processing Unit 6 (IPU6) Input System driver
7 ========================================================
8
9 Copyright |copy| 2023--2024 Intel Corporation
10
11 Introduction
12 ============
13
14 This file documents the Intel IPU6 (6th generation Image Processing Unit)
15 Input System (MIPI CSI2 receiver) drivers located under
16 drivers/media/pci/intel/ipu6.
17
18 The Intel IPU6 can be found in certain Intel SoCs but not in all SKUs:
19
20 * Tiger Lake
21 * Jasper Lake
22 * Alder Lake
23 * Raptor Lake
24 * Meteor Lake
25
26 Intel IPU6 is made up of two components - Input System (ISYS) and Processing
27 System (PSYS).
28
29 The Input System mainly works as MIPI CSI-2 receiver which receives and
30 processes the image data from the sensors and outputs the frames to memory.
31
32 There are 2 driver modules - intel-ipu6 and intel-ipu6-isys. intel-ipu6 is an
33 IPU6 common driver which does PCI configuration, firmware loading and parsing,
34 firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit)
35 configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2
36 sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected
37 to the IPU6 ISYS through V4L2 sub-device sensor drivers.
38
39 .. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more
40 information about the IPU6 hardware.
41
42 Input system driver
43 ===================
44
45 The Input System driver mainly configures CSI-2 D-PHY, constructs the firmware
46 stream configuration, sends commands to firmware, gets response from hardware
47 and firmware and then returns buffers to user. The ISYS is represented as
48 several V4L2 sub-devices as well as video nodes.
49
50 .. kernel-figure:: ipu6_isys_graph.svg
51 :alt: ipu6 isys media graph with multiple streams support
52
53 IPU6 ISYS media graph with multiple streams support
54
55 The graph has been produced using the following command:
56
57 .. code-block:: none
58
59 fdp -Gsplines=true -Tsvg < dot > dot.svg
60
61 Capturing frames with IPU6 ISYS
62 -------------------------------
63
64 IPU6 ISYS is used to capture frames from the camera sensors connected to the
65 CSI2 ports. The supported input formats of ISYS are listed in table below:
66
67 .. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
68
69 .. flat-table::
70 :header-rows: 1
71
72 * - IPU6 ISYS supported input formats
73
74 * - RGB565, RGB888
75
76 * - UYVY8, YUYV8
77
78 * - RAW8, RAW10, RAW12
79
80 .. _ipu6_isys_capture_examples:
81
82 Examples
83 ~~~~~~~~
84
85 Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this
86 machine, ov01a10 sensor is connected to IPU ISYS CSI-2 port 2, which can
87 generate images at sBGGR10 with resolution 1280x800.
88
89 Using the media controller APIs, we can configure ov01a10 sensor by
90 media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS.
91
92 .. code-block:: none
93
94 # Example 1 capture frame from ov01a10 camera sensor
95 # This example assumes /dev/media0 as the IPU ISYS media device
96 export MDEV=/dev/media0
97
98 # Establish the link for the media devices using media-ctl
99 media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]"
100
101 # Set the format for the media devices
102 media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]"
103 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]"
104 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]"
105
106 Once the media pipeline is configured, desired sensor specific settings
107 (such as exposure and gain settings) can be set, using the yavta tool.
108
109 e.g
110
111 .. code-block:: none
112
113 # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36
114 export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036")
115
116 yavta -w 0x009e0903 400 $SDEV
117 yavta -w 0x009e0913 1000 $SDEV
118 yavta -w 0x009e0911 2000 $SDEV
119
120 Once the desired sensor settings are set, frame captures can be done as below.
121
122 e.g
123
124 .. code-block:: none
125
126 yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \
127 -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0")
128
129 With the above command, 10 frames are captured at 1280x800 resolution with
130 sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files.
131
132 Here is another example of IPU6 ISYS RAW and metadata capture from camera
133 sensor ov2740 on Lenovo X1 Yoga laptop.
134
135 .. code-block:: none
136
137 media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]"
138 media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[1]"
139 media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[1]"
140
141 # set routing
142 media-ctl -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]"
143
144 media-ctl -V "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]"
145 media-ctl -V "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]"
146 media-ctl -V "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]"
147 media-ctl -V "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]"
148
149 CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0")
150 ./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \
151 -f SGRBG10 ${CAPTURE_DEV}
152
153 CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1")
154 ./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \
155 --file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META}
156
157 References
158 ==========
159
160 .. [#f1] https://git.ideasonboard.org/media-ctl.git
161 .. [#f2] https://git.ideasonboard.org/yavta.git
162

3. 한국어 전문 번역

영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.

Intel IPU6 Input System driver

1-10

이 문서는 GPL-2.0 라이선스를 따르는 Intel Image Processing Unit 6(IPU6) Input System driver 안내입니다. Copyright (C) 2023-2024 Intel Corporation.

IPU6 ISYS 소개

11-41

이 문서는 `drivers/media/pci/intel/ipu6`에 있는 Intel IPU6(6th generation Image Processing Unit) Input System(MIPI CSI-2 receiver) driver를 설명합니다.

Intel IPU6는 다음 Intel SoC의 일부 SKU에 포함됩니다.

  • Tiger Lake
  • Jasper Lake
  • Alder Lake
  • Raptor Lake
  • Meteor Lake

Intel IPU6는 Input System(ISYS)과 Processing System(PSYS), 두 component로 구성됩니다.

Input System은 주로 MIPI CSI-2 receiver로 동작하며 sensor의 image data를 받아 처리하고 frame을 memory로 출력합니다.

Driver module은 `intel-ipu6`와 `intel-ipu6-isys` 두 개입니다. `intel-ipu6` common driver는 PCI configuration, firmware loading과 parsing, firmware authentication, DMA mapping, IPU-MMU(internal Memory Mapping Unit) configuration을 담당합니다.

`intel_ipu6_isys`는 V4L2, Media Controller, V4L2 sub-device interface를 구현합니다. IPU6 ISYS driver는 V4L2 sub-device sensor driver를 통해 ISYS에 연결된 camera sensor를 지원합니다.

IPU6 hardware의 자세한 내용은 `Documentation/driver-api/media/drivers/ipu6.rst`를 참조하십시오.

IPU6 driver 책임 분리
intel-ipu6PCI · firmware · DMA · IPU-MMU
intel-ipu6-isyscamera sensorIPU6 CSI-2 receiverISYS firmware streamDMA capture nodememory buffer

Common driver가 device와 firmware 기반을 준비하고 ISYS driver가 sensor-to-memory V4L2 pipeline을 제공합니다.

Input System driver와 media graph

42-60

Input System driver는 CSI-2 D-PHY를 구성하고 firmware stream configuration을 만든 뒤 firmware에 command를 보냅니다. Hardware와 firmware response를 받아 처리하고 완료된 buffer를 user에게 돌려줍니다.

ISYS는 여러 V4L2 sub-device와 video node로 표현됩니다. 원문 media graph source는 `Documentation/admin-guide/media/ipu6_isys_graph.svg`입니다.

IPU6 ISYS multi-stream media graph
sensorIntel IPU6 CSI2 0 /dev/v4l-subdev0source pads 1-8Capture 0-7 /dev/video0-7
sensorIntel IPU6 CSI2 1 /dev/v4l-subdev1source pads 1-8Capture 8-15 /dev/video8-15
sensorIntel IPU6 CSI2 2 /dev/v4l-subdev2source pads 1-8Capture 16-23 /dev/video16-23
sensorIntel IPU6 CSI2 3 /dev/v4l-subdev3source pads 1-8Capture 24-31 /dev/video24-31

네 CSI-2 receiver는 각각 sink pad 0과 여러 routed source stream을 가지며, 전체 32개 ISYS capture video node로 route할 수 있습니다.

원문 graph는 다음 Graphviz command로 생성되었습니다.

fdp -Gsplines=true -Tsvg < dot > dot.svg

ISYS capture format

61-81

IPU6 ISYS는 CSI-2 port에 연결된 camera sensor에서 frame을 capture합니다. 지원 input format은 다음과 같습니다.

IPU6 ISYS 지원 input format
RGB565, RGB888
UYVY8, YUYV8
RAW8, RAW10, RAW12

Capture 예제 환경

82-91

첫 번째 예는 Dell XPS 9315 laptop에서 IPU6 ISYS raw capture를 수행합니다. 이 machine의 OV01A10 sensor는 IPU ISYS CSI-2 port 2에 연결되며 1280x800 resolution의 sBGGR10 image를 생성할 수 있습니다.

Media Controller API와 `media-ctl` `f1`, `yavta` `f2`를 사용해 OV01A10이 IPU6 ISYS로 frame을 전송하도록 구성합니다.

OV01A10 sensor control

106-123

Media pipeline을 구성한 뒤 exposure와 gain 같은 sensor별 설정을 `yavta`로 지정할 수 있습니다.

# and that ov01a10 sensor is connected to i2c bus 3 with address 0x36
export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036")

yavta -w 0x009e0903 400 $SDEV
yavta -w 0x009e0913 1000 $SDEV
yavta -w 0x009e0911 2000 $SDEV

원하는 sensor 설정을 적용한 뒤 다음과 같이 frame을 capture합니다.

OV01A10 frame capture

124-131 
yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \
        -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0")

이 명령은 1280x800 resolution과 sBGGR10 format으로 frame 10개를 capture합니다. Capture frame은 `/tmp/frame-#.bin` file로 저장됩니다.

OV2740 raw와 metadata 동시 capture

132-156

두 번째 예는 Lenovo X1 Yoga laptop의 OV2740 camera sensor에서 IPU6 ISYS raw image와 metadata를 동시에 capture합니다.

media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]"
media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[1]"
media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[1]"

# set routing
media-ctl -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]"

media-ctl -V "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]"
media-ctl -V "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]"
media-ctl -V "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]"
media-ctl -V "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]"

CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0")
./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \
    -f SGRBG10 ${CAPTURE_DEV}

CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1")
./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \
    --file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META}
OV2740 multi-stream routing
ov2740 14-0036:0 stream 0CSI2 1 0/0→1/0Capture 0 SGRBG10 1932x1092/tmp/frame-#.bin
ov2740 14-0036:0 stream 1CSI2 1 0/1→2/1Capture 1 GENERIC_8 97x1 META_CAPTURE/tmp/meta-#.bin

CSI2 1의 sink stream 두 개를 서로 다른 source pad와 capture node로 route해 image와 metadata를 동시에 받습니다.

`media-ctl -R`은 sink stream 0을 source pad 1 stream 0으로, sink stream 1을 source pad 2 stream 1로 route합니다. Image stream은 `SGRBG10/1932x1092`, metadata stream은 `GENERIC_8/97x1` format을 사용합니다.

Reference tool

157-161