Recently, I was preparing to make Android Camera2 related applications, and happened to encounter YUV format related problems, so I still want to write a blog to understand YUV format data.

introduce

YUV is a color space, and color coding based on YUV is a common encoding method for streaming media.

Y ‘uv, YUV, YCbCr, YPbPr, etc. can be called YUV. “Y” refers to Luminance (Luma), “U” and “V” refer to Chrominance (Chroma), Y ‘UV, YUV, YCbCr and YPbPr, which are often confused or overlapped.

YUV formats come in two broad categories: planar and Packed.

  • Planar formats: For Planar YUV formats, Y for all pixels is stored consecutively, followed by U for all pixels, and then V for all pixels, such as YYYY YYYY UU VV.

  • Packed formats: For Packed YUV formats, Y,U and V of each pixel are stored consecutively and alternately, such as YUV YUV YUV YUV, which is similar to RGB.

history

Y’UV was invented because of the transition between color and black and white television. Black and white video only has Y (Luma, Luminance) video, which is gray scale. To the specification of color TV, YUV/YIQ format to process color TV images, UV as C (Chrominance or Chroma), if the C signal is ignored, then the remaining Y (Luma) signal with the previous black and white video number the same, In this way, the compatibility of color and black and white TV sets will be solved. The biggest advantage of Y’UV is that it requires very little bandwidth.

Because UV respectively represent different color signal, so directly use R and B signal to represent chromaticity UV. That is, the UV signal tells the TV to shift the color of a pixel without changing its brightness. Or the UV signal tells the display to offset the brightness of a certain color against a certain baseline. The higher the UV value, the more saturated the pixel will be.

Color image recording format, common RGB, YUV, CMYK and so on. The earliest idea of color TV is to use RGB three primary colors to transmit simultaneously. This design method is three times of the original black and white bandwidth, which was not a good design at that time. RGB appeals to the human eye color induction, YUV focuses on the visual sensitivity to brightness, Y represents brightness, UV represents chroma (so black and white film can omit UV, similar to RGB), respectively, with Cr and Cb to represent, so YUV records are usually presented in Y:UV format.

The Y, U and V data of a picture will be displayed separately as shown below:

Common YUV format

In fact, the storage format of YUV is closely related to its sampling method. There are three mainstream sampling methods: YUV4:4:4, YUV4:2:2, and YUV4:2:0.

A black dot represents the Y component of the sampled pixel, and a hollow circle represents the UV component of the sampled pixelCopy the code

YUV444

4:4:4 means complete sampling. Each pixel is sampled with individual YUV component information, which contains the most comprehensive YUV information.

YUV422

On the basis of YUV444 coding, a 2 * 1 matrix is used for secondary sampling, that is, the UV information is sampled in the horizontal direction every one row, and the complete sampling is carried out in the vertical direction. Each two Y shares a set of UV components.

  • YUYV format

    Byte arrangement: YUYV YUYV YUYV YUYV

  • UYVY format

    Byte arrangement: UYVY UYVY UYVY UYVY

  • YUV422P format

    Byte sequence :YYYY YYYY UUUU VVVV

YUV420

YUV444 is also the most commonly used sampling format. On the basis of YUV444 format, a 2 * 2 matrix is used for secondary sampling of pixels. Four pixels have a single Y component and share the same UV information, with a total of 6 bytes. A frame of image occupies the total space (W * H + W * H / 2) bytes, less than half of YUV444 space.

  • I420 format byte format YYYY YYYY UU VV, all in plane format

  • NV21 format byte arrangement YYYY YYYY UV UV, Y plane and UV plane, UV interior is compact

  • YV12 Format Bytes YYYY YYYY VV UU, flat, V before U

V4L2 document translation (10)

4:2:2 and 4:2:0 convert

The simplest way:

Yuv4:2:2 –> YUV4:2:0 Y is unchanged, and the U and V signal values are sampled in an interlaced row (vertical direction)

Yuv4:2:0 –> YUV4:2:2 Y unchanged, copy each line of U and V signal values respectively to form two consecutive lines of data

YUV_420_888

YUV_420_888 is an internal Android YUV format encountered in the development of Android Camera2 live broadcast. At that time, I also struggled with this format for a long time. I wanted to use YUV420P format for the live video stream. But it is not clear what kind of format this YUV_420_888 corresponds to in the end, and there is little explanation of this format on the Internet. After a bit of searching, I finally got it. Here it is.

The following two blogs introduce YUV_420_888 and how to parse it, which is very clear, but I will not write it myself.

What is the yuv420 data from android camera2?

Android: Image class analysis (YUV_420_888)