YUV color coding analysis

YUV

YUV is a kind of color space, the color coding based on YUV is a common coding method for streaming media. Y said lumen, U, V color, concentration, the expression was originally intended for color TV and black and white TV signal compatibility. For each image, determine the brightness of Y, UV to confirm its chroma.

Y’CbCr also known as YUV, is a compressed version of YUV, the difference is Y’CbCr in the field of digital image, YUV is used to simulate the signal field, often say MPEG, DVD, camera in YUV is Y’CbCr, the two conversion to the RGBA transformation matrix is different. Y’for brightness, Cb, Cr component represents the current color to blue and red offset.

YUV color coding analysis
Y’=0.5, Cb, Cr constitute the color plane

If the output value of the three components of Y’CbCr, then it will be like this.

YUV color coding analysis
from top to bottom followed by Y’, Cb, Cr

For convenience, in the following YUV specifically for Y’CbCr.

The role of YUV color coding

YUV coding is an important component of image/video pipeline. For example, commonly used I420 with respect to RGB24 (RGB three components of each of the 8 bytes) encoding format, only half the storage capacity. Reducing the bandwidth pressure in stream data transmission.

Application of YUV color coding analysis
YUV color coding in video pipeline

YUV color coding format

The color coding scheme of YUV is determined by its chroma sampling and storage mode.

Chroma sampling

J:A:B
chroma sampling J: minimum horizontal sampling width is generally 4
chroma subsampling A: minimum level sampling area of the first line of the
B: minimum level of chroma sampling sampling area of second lines

YUV color coding analysis

Note that 4:2:0 is not only the first row of the chroma, is the first row and the second row of samples: 4:2:0 –> 4:0:2 –> 4:2:0……
can be seen, no matter what kind of sampling, the brightness is full sampling, different in the U, V component sampling rate. You can see the commonly used 4:2:0, U, V are semi sampling, so the amount of data after sampling is half RGB24. (RGB24 equivalent to full sampling)

YUV storage mode

YUV storage is divided into two main types: Packeted and Planar.
Packeted mode similar to the storage of RGB, the pixel matrix for storage. In
Planar, the YUV components are stored in a matrix, each of which is called a plane.
YUV420 that is stored in a flat format, color sampling for 4:2:0 color coding format. Where YUV420P is three planar storage, YUV420SP for the two plane storage.
commonly used I420 (YUV420P), NV12 (YUV420SP), YV12 (YUV420P), NV21 (YUV420SP) and so on are YUV420, NV12 is a two plane storage mode, Y for a plane, staggered UV for another plane.

As a result, I420 is stored as a Planar, sampling method for 4:2:0, data is composed of a YYYYYYYYUUVV color coding format. In addition to
, NV12 data composition: YYYYYYYYUVUV. YV12 data composition: YYYYYYYYVVUU. NV21 data composition: YYYYYYYYVVUU.
is usually used for remote transmission of I420 data, and the local camera is a collection of NV12 data. (iOS)

YUV color coding analysis
I420 for transmission
YUV color coding analysis
camera data collected is NV12

Conversion between YUV and RGB

In rendering, whether it is OpenGL or iOS, does not support the direct rendering of YUV data, the bottom is converted to RGB.

//RGB --> YUV Y = 0.299 R + 0.587 G + 0.114 B = U - 0.1687 R - 0.3313 G + 0.5 B + 128 V = 0.5 R - 0.4187 G - 0.0813 B + 128 //YUV --> RGB; / / because U, V may appear negative, single storage for convenience with a byte: 0-255 when you read, the original value of -128 regression. R = Y + 1.402 (Cr-128) G = Y - (Cb-128) - (Cr-128) B = Y + 1.772 (Cb-128)

YUV data rendering

Taking NV12 as an example:

Void convertNv12ToRgb (unsigned char *rgbout, unsigned char *pdata, int DataWidth, int DataHeight) {unsigned long idx=0; unsigned char *ubase; unsigned *ybase, char y, u, V; ybase = pdata; / / get the Y plane address Ubase = pdata+DataWidth * DataHeight; / / get the plane U address, NV12 U, is due to V are stored in a civilian, V is u+1 for (int j=0; j< DataHeight; j++) {idx= (DataHeight-j-1) *DataWidth*3; / / RGB data to ensure that the value of the reverse generated is stored in rgbbuf, the bitmap is upside down for (int i=0; i< DataWidth; i++) {unsigned char R, G, B; y=ybase[i + J * DataWidth]; / / one pixel corresponds to a Y u=ubase[j/2 * DataWidth+ (i/2) *2]; / / every four y corresponding to a UV V=ubase[j/2 * DataWidth+ (i/2) *2+1]; / / we must pay attention to is u+1 (unsigned b= char) (y+1.779* (u- 128)); g= (unsigned (char) y-0.7169* (V - 128) -0.3455* (U - 128); r= (unsigned) char (y+) 1.4075* (V - 128)); rgbout[idx ++]=b rgbout[idx++]=g;; rgbout[idx++]=r;}}}

Sometimes different YUV formats need to be converted to each other

Unsigned char* convertNV12ToI420 (unsigned char *data, int dataWidth, int dataHeight) {unsigned char *ybase, *ubase; ybase = data; Ubase = data + dataWidth*dataHeight; unsigned char* = tmpData (unsigned char*) malloc (dataWidth* dataHeight * 1.5 int); offsetOfU = dataWidth*dataHeight; int = offsetOfV dataWidth*dataHeight* 5/4 (tmpData, ybase, memcpy; dataWidth*dataHeight for (int); I = 0; I < dataWidth*dataHeight/2; i++) {if (I% 2 = = 0) {tmpData[offsetOfU] = ubase[i]; tmpData[offsetOfV] = offsetOfU++;}else{ubase[i]; offsetOfV++ free;}} (data); return tmpData;}

Or need to rotate the data obtained

Void rotate90NV12 (unsigned char *dst const unsigned, char *src, int srcWidth, int srcHeight) {int wh = srcWidth * srcHeight; int uvHeight = srcHeight / int = 2; uvWidth srcWidth / 2; / / int rotation Y I = 0, j = 0; int srcPos = 0, nPos = 0; for (I I = 0; < srcHeight; i++) {nPos = srcHeight - 1 - I; for (J = 0; J < srcWidth; j++) {dst[j * srcHeight + nPos] = src[srcPos++];}} srcPos (I = wh; for = 0; I < uvHeight; i++) {nPos = (uvHeight - 1 - I) * 2 (J = 0; for; J < uvWidth; j++) {dst[wh + J * srcHeight + nPos] = src[srcPos++]; dst[wh + J * srcHeight + nPos + 1] = src[srcPos++];}}}
Void rotate270YUV420sp (unsigned char *dst const unsigned, char *src, int srcWidth, int srcHeight) {int nWidth = 0, nHeight = 0; int wh int = 0; uvHeight = 0; if (srcWidth = nWidth ||! SrcHeight! = nHeight) {nWidth = srcWidth; nHeight = srcHeight; wh = srcWidth * srcHeight uvHeight = srcHeight; > > 1; //uvHeight = height / 2} / / int rotation Y k = 0; for (int i = 0; I < srcWidth; i++) {int nPos = srcWidth - 1; for (int j = 0; J < srcHeight; j++) {dst[k] = src[nPos i]; k++; nPos = srcWidth;}} for (int i = 0; I < srcWidth; i+=2) {int nPos = wh + srcWidth - 1 (int j; for J = 0; < uvHeight; j++) {dst[k] = src[nPos - I - 1]; dst[k + 1] = src[nPos - i]; k = 2; nPos = srcWidth;}}}

In iOS, you can use core graphics to draw RGB data into UIImage.

- (UIImage) convertBitmapRGBA8ToUIImage: (unsigned * char) buffer withWidth: (int) width withHeight: (int) height to RGBA32 {/ / RGBA = char* (char*) malloc (width*height*4); for (int i=0; I < width*height; ++i) {RGBA [4*i] = buffer[3*i]; rgba[4*i+1] = buffer[3*i+1]; rgba[4*i+2] = buffer[3*i+2]; rgba[4*i+3] = 255;} size_t bufferLength = width * height * CGDataProviderRef 4; provider = CGDataProviderCreateWithData (NULL, RGBA, bufferLength, NULL); size_t bitsPerComponent size_t = 8; bitsPerPixel = 32; size_t = 4 * bytesPerRow width; CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB (if (CO); LorSpaceRef = = NULL) {NSLog (@ Error allocating color space "); CGDataProviderRelease (provider); return nil;} CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault kCGImageAlphaPremultipliedLast CGColorRenderingIntent |; renderingIntent = kCGRenderingIntentDefault; CGImageRef = Iref CGImageCreate (width, height, bitsPerComponent, bitsPerPixel, bytesPerRow, colorSpaceRef, bitmapInfo, provider, provider, NULL / data, / / decode / / should YES, interpolate renderingIntent); uint32_t* pixels = (uint32_t*) malloc (bufferLength); if (pixels = = NULL) {NSLog ("Error: Memory not allocated for @ bitmap"); CGDataProviderRelease (provider); CGColorSpaceRelease (colorSpaceRef); CGImageRelease (Iref); return nil;} CGContextRef context = CGBitmapContextCreate (pixels, width, height, bitsPerComponent, bytesPerRow, ColorSpaceRef, bitmapInfo); if (context = = NULL) {NSLog (@ Error context not created "); free (pixels);} UIImage *image = nil; if (context) {CGContextDrawImage (context, CGRectMake (0.0F, 0.0F, width, height), Iref); CGImageRef imageRef = CGBitmapContextCreateImage (context) image = [UIImage imageWithCGImage:imageRef scale:[UIScreen; mainScreen].scale orientation:UIImageOrientationUp]; if ([UIImage respondsToSelector:@selector (imageWithCGImage: scale:orientation:)] image = [UIImage imageWithCGImage:imageRef) {scale:1.0 orientation:UIImageOrientationUp]} else {image = [UI; Image imageWithCGImage:imageRef] CGImageRelease (imageRef);};}; CGContextRelease (context) CGColorSpaceRelease (colorSpaceRef); CGImageRelease (Iref); CGDataProviderRelease (provider); if (pixels) {free} (pixels); return image;}