압축 영상 형식

컨테이너·Access Unit 기반 형식

이 문서는 V4L2가 압축 이미지와 영상 payload를 식별할 때 사용하는 FourCC를 정리합니다. 압축 형식에서는 한 buffer가 완전한 picture 또는 Access Unit을 담아야 하는지, 연속 bytestream을 허용하는지, 별도 control로 stream 종류를 정하는지가 핵심 계약입니다.

.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later

******************
Compressed Formats
******************


.. _compressed-formats:

.. raw:: latex

    \small

.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|

.. cssclass:: longtable

.. flat-table:: Compressed Image Formats
    :header-rows:  1
    :stub-columns: 0
    :widths:       3 1 4

    * - Identifier
      - Code
      - Details
    * .. _V4L2-PIX-FMT-JPEG:

      - ``V4L2_PIX_FMT_JPEG``
      - 'JPEG'
      - TBD. See also :ref:`VIDIOC_G_JPEGCOMP <VIDIOC_G_JPEGCOMP>`,
	:ref:`VIDIOC_S_JPEGCOMP <VIDIOC_G_JPEGCOMP>`.
    * .. _V4L2-PIX-FMT-MPEG:

      - ``V4L2_PIX_FMT_MPEG``
      - 'MPEG'
      - MPEG multiplexed stream. The actual format is determined by
	extended control ``V4L2_CID_MPEG_STREAM_TYPE``, see
	:ref:`mpeg-control-id`.
    * .. _V4L2-PIX-FMT-H264:

      - ``V4L2_PIX_FMT_H264``
      - 'H264'
      - H264 Access Unit.
	The decoder expects one Access Unit per buffer.
	The encoder generates one Access Unit per buffer.
	If :ref:`VIDIOC_ENUM_FMT` reports ``V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM``
	then the decoder has no requirements since it can parse all the
	information from the raw bytestream.
    * .. _V4L2-PIX-FMT-H264-NO-SC:

      - ``V4L2_PIX_FMT_H264_NO_SC``
      - 'AVC1'
      - H264 video elementary stream without start codes.
    * .. _V4L2-PIX-FMT-H264-MVC:

      - ``V4L2_PIX_FMT_H264_MVC``
      - 'M264'
      - H264 MVC video elementary stream.

H.264 slice와 MPEG 계열

`V4L2_PIX_FMT_H264_SLICE`는 H.264 bitstream에서 꺼낸 slice header 포함 parsed slice data이며 start code 유무를 모두 허용합니다. Stateless decoder pipeline용이므로 `V4L2_CID_STATELESS_H264_DECODE_MODE`와 `V4L2_CID_STATELESS_H264_START_CODE`를 최소 한 번 설정해야 합니다.

Frame metadata는 `V4L2_CID_STATELESS_H264_SPS`, `PPS`, `SCALING_MATRIX`, `SLICE_PARAMS`, `DECODE_PARAMS` control로 전달합니다. 정확히 output buffer 하나와 capture buffer 하나를 제공하며 output에는 대응 capture frame 전체를 복호화할 만큼의 macroblock이 있어야 합니다. 문법 기준은 H.264 7.3.2.8의 slice layer without partitioning RBSP syntax와 후속 절입니다.

    * .. _V4L2-PIX-FMT-H264-SLICE:

      - ``V4L2_PIX_FMT_H264_SLICE``
      - 'S264'
      - H264 parsed slice data, including slice headers, either with or
	without the start code, as extracted from the H264 bitstream.
	This format is adapted for stateless video decoders that implement an
	H264 pipeline with the :ref:`stateless_decoder`.
	This pixelformat has two modifiers that must be set at least once
	through the ``V4L2_CID_STATELESS_H264_DECODE_MODE``
        and ``V4L2_CID_STATELESS_H264_START_CODE`` controls.
	In addition, metadata associated with the frame to decode are
	required to be passed through the ``V4L2_CID_STATELESS_H264_SPS``,
	``V4L2_CID_STATELESS_H264_PPS``,
	``V4L2_CID_STATELESS_H264_SCALING_MATRIX``,
	``V4L2_CID_STATELESS_H264_SLICE_PARAMS`` and
	``V4L2_CID_STATELESS_H264_DECODE_PARAMS`` controls.  See the
	:ref:`associated Codec Control IDs <v4l2-codec-stateless-h264>`.
        Exactly one output and one capture buffer must be provided for use
	with this pixel format. The output buffer must contain the
	appropriate number of macroblocks to decode a full
	corresponding frame to the matching capture buffer.

	The syntax for this format is documented in :ref:`h264`, section
	7.3.2.8 "Slice layer without partitioning RBSP syntax" and the following
	sections.

    * .. _V4L2-PIX-FMT-H263:

      - ``V4L2_PIX_FMT_H263``
      - 'H263'
      - H263 video elementary stream.
    * .. _V4L2-PIX-FMT-SPK:

      - ``V4L2_PIX_FMT_SPK``
      - 'SPK0'
      - Sorenson Spark is an implementation of H.263 for use in Flash Video and Adobe Flash files
    * .. _V4L2-PIX-FMT-MPEG1:

      - ``V4L2_PIX_FMT_MPEG1``
      - 'MPG1'
      - MPEG1 Picture. Each buffer starts with a Picture header, followed
	by other headers as needed and ending with the Picture data.
	If :ref:`VIDIOC_ENUM_FMT` reports ``V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM``
	then the decoder has no requirements since it can parse all the
	information from the raw bytestream.
    * .. _V4L2-PIX-FMT-MPEG2:

      - ``V4L2_PIX_FMT_MPEG2``
      - 'MPG2'
      - MPEG2 Picture. Each buffer starts with a Picture header, followed
	by other headers as needed and ending with the Picture data.
	If :ref:`VIDIOC_ENUM_FMT` reports ``V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM``
	then the decoder has no requirements since it can parse all the
	information from the raw bytestream.
    * .. _V4L2-PIX-FMT-MPEG2-SLICE:

      - ``V4L2_PIX_FMT_MPEG2_SLICE``
      - 'MG2S'
      - MPEG-2 parsed slice data, as extracted from the MPEG-2 bitstream.
	This format is adapted for stateless video decoders that implement a
	MPEG-2 pipeline with the :ref:`stateless_decoder`.
	Metadata associated with the frame to decode is required to be passed
	through the ``V4L2_CID_STATELESS_MPEG2_SEQUENCE`` and
        ``V4L2_CID_STATELESS_MPEG2_PICTURE`` controls.
        Quantisation matrices can optionally be specified through the
	``V4L2_CID_STATELESS_MPEG2_QUANTISATION`` control.
	See the :ref:`associated Codec Control IDs <v4l2-codec-stateless-mpeg2>`.
	Exactly one output and one capture buffer must be provided for use with
	this pixel format. The output buffer must contain the appropriate number
	of macroblocks to decode a full corresponding frame to the matching
	capture buffer.

MPEG-4·VC-1·VP8·VP9

MPEG-4, Xvid와 VC-1 형식은 elementary 또는 표준 annex stream을 식별합니다. VP8·VP9의 일반 형식은 압축 frame 하나를 buffer 하나에 대응시키고, `_FRAME` 형식은 container에서 추출한 parsed frame과 stateless metadata control을 결합합니다.

    * .. _V4L2-PIX-FMT-MPEG4:

      - ``V4L2_PIX_FMT_MPEG4``
      - 'MPG4'
      - MPEG4 video elementary stream.
    * .. _V4L2-PIX-FMT-XVID:

      - ``V4L2_PIX_FMT_XVID``
      - 'XVID'
      - Xvid video elementary stream.
    * .. _V4L2-PIX-FMT-VC1-ANNEX-G:

      - ``V4L2_PIX_FMT_VC1_ANNEX_G``
      - 'VC1G'
      - VC1, SMPTE 421M Annex G compliant stream.
    * .. _V4L2-PIX-FMT-VC1-ANNEX-L:

      - ``V4L2_PIX_FMT_VC1_ANNEX_L``
      - 'VC1L'
      - VC1, SMPTE 421M Annex L compliant stream.
    * .. _V4L2-PIX-FMT-VP8:

      - ``V4L2_PIX_FMT_VP8``
      - 'VP80'
      - VP8 compressed video frame. The encoder generates one
	compressed frame per buffer, and the decoder requires one
	compressed frame per buffer.
    * .. _V4L2-PIX-FMT-VP8-FRAME:

      - ``V4L2_PIX_FMT_VP8_FRAME``
      - 'VP8F'
      - VP8 parsed frame, including the frame header, as extracted from the container.
	This format is adapted for stateless video decoders that implement an
	VP8 pipeline with the :ref:`stateless_decoder`.
	Metadata associated with the frame to decode is required to be passed
	through the ``V4L2_CID_STATELESS_VP8_FRAME`` control.
	See the :ref:`associated Codec Control IDs <v4l2-codec-stateless-vp8>`.
	Exactly one output and one capture buffer must be provided for use with
	this pixel format. The output buffer must contain the appropriate number
	of macroblocks to decode a full corresponding frame to the matching
	capture buffer.

    * .. _V4L2-PIX-FMT-VP9:

      - ``V4L2_PIX_FMT_VP9``
      - 'VP90'
      - VP9 compressed video frame. The encoder generates one
	compressed frame per buffer, and the decoder requires one
	compressed frame per buffer.
    * .. _V4L2-PIX-FMT-VP9-FRAME:

      - ``V4L2_PIX_FMT_VP9_FRAME``
      - 'VP9F'
      - VP9 parsed frame, including the frame header, as extracted from the container.
	This format is adapted for stateless video decoders that implement a
	VP9 pipeline with the :ref:`stateless_decoder`.
	Metadata associated with the frame to decode is required to be passed
	through the ``V4L2_CID_STATELESS_VP9_FRAME`` and
	the ``V4L2_CID_STATELESS_VP9_COMPRESSED_HDR`` controls.
	See the :ref:`associated Codec Control IDs <v4l2-codec-stateless-vp9>`.
	Exactly one output and one capture buffer must be provided for use with
	this pixel format. The output buffer must contain the appropriate number
	of macroblocks to decode a full corresponding frame to the matching
	capture buffer.

HEVC·FWHT·RealVideo·AV1

`V4L2_PIX_FMT_HEVC`는 H.265 Access Unit 형식으로 encoder와 decoder가 buffer 하나당 Access Unit 하나를 교환합니다. `V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM`이 있으면 raw bytestream 자체에서 정보를 분석할 수 있어 decoder의 buffer 경계 요구가 사라집니다.

`V4L2_PIX_FMT_HEVC_SLICE`는 HEVC parsed slice이며 stateless mem2mem·Media Request pipeline에 맞춰집니다. `V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE`와 `START_CODE`를 최소 한 번 설정하고 SPS, PPS, SLICE_PARAMS control을 전달합니다. 각 buffer에는 대응 frame 전체를 복호화할 만큼의 macroblock이 있어야 합니다.

    * .. _V4L2-PIX-FMT-HEVC:

      - ``V4L2_PIX_FMT_HEVC``
      - 'HEVC'
      - HEVC/H.265 Access Unit.
	The decoder expects one Access Unit per buffer.
	The encoder generates one Access Unit per buffer.
	If :ref:`VIDIOC_ENUM_FMT` reports ``V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM``
	then the decoder has no	requirements since it can parse all the
	information from the raw bytestream.
    * .. _V4L2-PIX-FMT-HEVC-SLICE:

      - ``V4L2_PIX_FMT_HEVC_SLICE``
      - 'S265'
      - HEVC parsed slice data, as extracted from the HEVC bitstream.
	This format is adapted for stateless video decoders that implement a
	HEVC pipeline (using the :ref:`mem2mem` and :ref:`media-request-api`).
	This pixelformat has two modifiers that must be set at least once
	through the ``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE``
        and ``V4L2_CID_MPEG_VIDEO_HEVC_START_CODE`` controls.
	Metadata associated with the frame to decode is required to be passed
	through the following controls:
        ``V4L2_CID_MPEG_VIDEO_HEVC_SPS``,
        ``V4L2_CID_MPEG_VIDEO_HEVC_PPS``, and
        ``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS``.
	See the :ref:`associated Codec Control IDs <v4l2-codec-stateless-hevc>`.
	Buffers associated with this pixel format must contain the appropriate
	number of macroblocks to decode a full corresponding frame.
    * .. _V4L2-PIX-FMT-FWHT:

      - ``V4L2_PIX_FMT_FWHT``
      - 'FWHT'
      - Video elementary stream using a codec based on the Fast Walsh Hadamard
        Transform. This codec is implemented by the vicodec ('Virtual Codec')
	driver. See the codec-fwht.h header for more details.
	:ref:`VIDIOC_ENUM_FMT` reports ``V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM``
	since the decoder can parse all the information from the raw bytestream.
    * .. _V4L2-PIX-FMT-FWHT-STATELESS:

      - ``V4L2_PIX_FMT_FWHT_STATELESS``
      - 'SFWH'
      - Same format as V4L2_PIX_FMT_FWHT but requires stateless codec implementation.
        Metadata associated with the frame to decode is required to be passed
        through the ``V4L2_CID_STATELESS_FWHT_PARAMS`` control.
	See the :ref:`associated Codec Control ID <codec-stateless-fwht>`.
    * .. _V4L2-PIX-FMT-RV30:

      - ``V4L2_PIX_FMT_RV30``
      - 'RV30'
      - RealVideo, or also spelled as Real Video, is a suite of
        proprietary video compression formats developed by
        RealNetworks - the specific format changes with the version.
        RealVideo codecs are identified by four-character codes.
        RV30 corresponds to RealVideo 8, suspected to be based
        largely on an early draft of H.264
    * .. _V4L2-PIX-FMT-RV40:

      - ``V4L2_PIX_FMT_RV40``
      - 'RV40'
      - RV40 represents RealVideo 9 and RealVideo 10.
        RealVideo 9, suspected to be based on H.264.
        RealVideo 10, aka RV9 EHQ, This refers to an improved encoder
        for the RV9 format that is fully backwards compatible with
        RV9 players - the format and decoder did not change, only
        the encoder did. As a result, it uses the same FourCC.

    * .. _V4L2-PIX-FMT-AV1-FRAME:

      - ``V4L2_PIX_FMT_AV1_FRAME``
      - 'AV1F'
      - AV1 parsed frame, including the frame header, as extracted from the container.
        This format is adapted for stateless video decoders that implement a AV1
        pipeline with the :ref:`stateless_decoder`. Metadata associated with the
        frame to decode is required to be passed through the
        ``V4L2_CID_STATELESS_AV1_SEQUENCE``, ``V4L2_CID_STATELESS_AV1_FRAME``,
        and ``V4L2_CID_STATELESS_AV1_TILE_GROUP_ENTRY`` controls.
        See the :ref:`associated Codec Control IDs <v4l2-codec-stateless-av1>`.
        Exactly one output and one capture buffer must be provided for use with
        this pixel format. The output buffer must contain the appropriate number
        of macroblocks to decode a full corresponding frame to the matching
        capture buffer.

.. raw:: latex

    \normalsize