Hard coding and soft coding for VIDEO H264 on ios platform

An overview of the

In the last article, we learned the related concepts of video and the process of H264 codec. In this article, we mainly do code implementation, and its content is summarized as follows:

Using VideoToolBox to do H264 hard coding for real-time video
ffmpeg
- Install FFMPEG on the MAC platform
- Simple common ffmpeg command
- How to compile the FFMPEG library and environment for ios development on MAC platform
- A brief introduction to ffMPEG library
Using FFMPEG to do H264 soft coding for real-time video

Sample code:

The h264 hard-coded
Soft h264 encoding
Welcome star&fork

Code structure:

Run screenshot:

If you are not sure about video codec concepts, please refer to the previous article – Basic video parameters and H264 codec concepts

Using VideoToolBox to do H264 hard coding for real-time video

The general steps are as follows:

Initializing the encoder

Step 1: Set the width and height of the video

- (void)settingEncoderParametersWithWidth:(int)width height:(int)height fps:(int)fps
{
   self.width  = width;
   self.height = height;
   self.fps    = fps;
}
Copy the code

Step 2: Set the encoder type to kCMVideoCodecType_H264 with the VTSessionSetProperty method and KVTCompressionPropertyKey_ExpectedFrameRate, kVTCompressionPropertyKey_AverageBitRate key setting parameters such as frame rate and bit rate respectively.


 - (void)prepareForEncoder
{
    if (self.width == 0 || self.height == 0) {
        NSLog(@"AppHWH264Encoder, VTSession need width and height for init, width = %d, height = %d",self.width,self.height);
        return;
    }
    
    [m_lock lock];
    OSStatus status =  noErr;
    status =  VTCompressionSessionCreate(NULL, self.width, self.height, kCMVideoCodecType_H264, NULL, NULL, NULL, miEncoderVideoCallBack, (__bridge void *)self, &compressionSession);
    if(status ! = noErr) { NSLog(@"AppHWH264Encoder , create encoder session failed,res=%d",status);
        return;
    }
    
    if (self.fps) {
        int v = self.fps;
        CFNumberRef ref = CFNumberCreate(NULL, kCFNumberSInt32Type, &v);
        status = VTSessionSetProperty(compressionSession, kVTCompressionPropertyKey_ExpectedFrameRate, ref);
        CFRelease(ref);
        if(status ! = noErr) { NSLog(@"AppHWH264Encoder, create encoder session failed, fps=%d,res=%d",self.fps,status);
            return; }}if (self.bitrate) {
        int v = self.bitrate;
        CFNumberRef ref = CFNumberCreate(NULL, kCFNumberSInt32Type, &v);
        status = VTSessionSetProperty(compressionSession, kVTCompressionPropertyKey_AverageBitRate,ref);
        CFRelease(ref);
        if(status ! = noErr) { NSLog(@"AppHWH264Encoder, create encoder session failed, bitrate=%d,res=%d",self.bitrate,status);
            return;
        }
    }
    
    status  = VTCompressionSessionPrepareToEncodeFrames(compressionSession);
    if(status ! = noErr) { NSLog(@"AppHWH264Encoder, create encoder session failed,res=%d",status);
        return; }}Copy the code

Hardcode CMSampleBufferRef with VTCompressionSession

Pass the CMSampleBuffer captured by the camera directly to the following encoding method:

- (void)encoder:(CMSampleBufferRef)sampleBuffer
{
    if(! self.isInitHWH264Encoder) { [self prepareForEncoder]; self.isInitHWH264Encoder = YES; } CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); CMTime presentationTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer); OSStatus status = VTCompressionSessionEncodeFrame(compressionSession, imageBuffer, presentationTime, kCMTimeInvalid, NULL, NULL, NULL);if(status ! = noErr) { VTCompressionSessionInvalidate(compressionSession); VTCompressionSessionCompleteFrames(compressionSession, kCMTimeInvalid); CFRelease(compressionSession); compressionSession = NULL; self.isInitHWH264Encoder = NO; NSLog(@"AppHWH264Encoder, encoder failed");
        return; }}Copy the code

In the callback function, the encoded code is converted to H264 stream structure

Here, the main is to parse SPS,PPS, and then add the start code after the assembly of NALU unit (here must be very familiar with the H264 bit stream structure, if not clear, can refer to the previous article)

 NSLog(@"%s",__func__);
    if(status ! = noErr) { NSLog(@"AppHWH264Encoder, encoder failed, res=%d",status);
        return;
    }
    if(! CMSampleBufferDataIsReady(sampleBuffer)) { NSLog(@"AppHWH264Encoder, samplebuffer is not ready");
        return;
    }
    
    MIHWH264Encoder *encoder = (__bridge MIHWH264Encoder*)outputCallbackRefCon;
    
    CMBlockBufferRef block = CMSampleBufferGetDataBuffer(sampleBuffer);
    
    BOOL isKeyframe = false;
    CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, false);
    if(attachments ! = NULL) { CFDictionaryRef attachment =(CFDictionaryRef)CFArrayGetValueAtIndex(attachments, 0); CFBooleanRef dependsOnOthers = (CFBooleanRef)CFDictionaryGetValue(attachment, kCMSampleAttachmentKey_DependsOnOthers); isKeyframe = (dependsOnOthers == kCFBooleanFalse); }if(isKeyframe)
    {
        CMFormatDescriptionRef format = CMSampleBufferGetFormatDescription(sampleBuffer);
        size_t spsSize, ppsSize;
        size_t parmCount;
        const uint8_t*sps, *pps;
        
        int NALUnitHeaderLengthOut;
        
        CMVideoFormatDescriptionGetH264ParameterSetAtIndex(format, 0, &sps, &spsSize, &parmCount, &NALUnitHeaderLengthOut );
        CMVideoFormatDescriptionGetH264ParameterSetAtIndex(format, 1, &pps, &ppsSize, &parmCount, &NALUnitHeaderLengthOut );
        
        uint8_t *spsppsNALBuff = (uint8_t*)malloc(spsSize+4+ppsSize+4);
        memcpy(spsppsNALBuff, "\x00\x00\x00\x01", 4);
        memcpy(&spsppsNALBuff[4], sps, spsSize);
        memcpy(&spsppsNALBuff[4+spsSize], "\x00\x00\x00\x01", 4);
        memcpy(&spsppsNALBuff[4+spsSize+4], pps, ppsSize);
        NSLog(@"AppHWH264Encoder, encoder video ,find IDR frame");
        //        AVFormatControl::GetInstance()->addH264Data(spsppsNALBuff, (int)(spsSize+ppsSize+8), dtsAfter, YES, NO);
        
        [encoder.delegate acceptEncoderData:spsppsNALBuff length:(int)(spsSize+ppsSize + 8) naluType:H264Data_NALU_TYPE_IDR];
    }
    
    size_t blockBufferLength;
    uint8_t *bufferDataPointer = NULL;
    CMBlockBufferGetDataPointer(block, 0, NULL, &blockBufferLength, (char **)&bufferDataPointer);
    
    const size_t startCodeLength = 4;
    static const uint8_t startCode[] = {0x00, 0x00, 0x00, 0x01};
    
    size_t bufferOffset = 0;
    static const int AVCCHeaderLength = 4;
    while (bufferOffset < blockBufferLength - AVCCHeaderLength)
    {
        uint32_t NALUnitLength = 0;
        memcpy(&NALUnitLength, bufferDataPointer+bufferOffset, AVCCHeaderLength);
        NALUnitLength = CFSwapInt32BigToHost(NALUnitLength);
        memcpy(bufferDataPointer+bufferOffset, startCode, startCodeLength);
        bufferOffset += AVCCHeaderLength + NALUnitLength;
    }
    
    //    AVFormatControl::GetInstance()->addH264Data(bufferDataPointer, (int)blockBufferLength,dtsAfter, NO, isKeyframe);
    
    [encoder.delegate acceptEncoderData:bufferDataPointer length:(int)blockBufferLength naluType:H264Data_NALU_TYPE_NOIDR];
Copy the code

Go to the sandbox and play the H264 file:

ffplay hwEncoder.h264 
Copy the code

ffmpeg

Install FFMPEG on the MAC platform

Installation is divided into source installation and command line installation. There are many tutorials on this section on the Internet, so I will only introduce a simple installation -> command line installation.

We use BREW to install ffmpeg and ffplay commands.

For information about BREW and its usage, see Homebrew installation and use

Blank installation:

If you have never installed FFmPEG on your computer before, you can simply install it using the following command.

brew install ffmpeg --with-ffplay
Copy the code

After the installation is successful, you can use ffplay –help to check whether the installation is successful.

Non-blank installation:

If you have ffMPEG installed on your computer, you need to uninstall it and install it again.

Unloading method:

brew uninstall ffmpeg
Copy the code

You can also download ffmpeg,ffplay and other command line tools from ffmpeg’s official website, copy them to your bin directory, or run them directly. The main purpose of this section is to be able to use ffMPEG command line tools to parse audio and video files on MAC.

Simple common ffmpeg command

The best way to find ffmpeg commands is directly on the ffmpeg website. There are also a lot of examples on the Internet, some relatively simple commands will naturally be used to remember. Here are some of the most common commands I read in a book:

ffprobe

Ffprobe is a tool used to view header information of media files.

View the audio file header information
```
Ffprobe dusk.mp3Copy the code
```

Display result:

Input #0, mp3, from 'twilight ': Metadata: title: Twilight album: Tagging time: 2012-08-08T02:48:38 TYER: 1998-01-01 Duration: 00:02:45.75, start: 0.025056, bitrate: 131 KB /s Stream #0:0: Audio: Mp3, 44100 Hz, STEREO, S16P, 128 KB /s Metadata: encoder: LAME3.97 Stream #0:1: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 240x240 [SAR 1:1 DAR 1:1], 90k tbr, 90k tbn, 90k tbc Metadata: title : e comment : Cover (front) ```Copy the code

View video header information
```
ffprobe test.mp4
Copy the code
```

That’s how you can view the headers of audio and video files. Here are a few more advanced uses.

Output format information such as format_name, duration, file size, bit_rate, and number of streams nb_streams
```
ffprobe -show_format test.mp4
Copy the code
```
Output the most detailed information for each stream in JSON format
```
ffprobe -print_format json -show_streams test.mp4
Copy the code
```

Display frame information

ffprobe -show_frames test.mp4
Copy the code

Viewing Package Information

ffprobe -show_packets test.mp4
Copy the code

ffplay

Ffplay is a media file player based on FFMPEG framework and libSDL rendering audio and video library. It comes with libSDL 1.2.

Play audio and video files

Ffplay test.mp4/ evening.mp3Copy the code

Play a video and loop it 10 times
```
ffplay test.mp4 -loop 10
Copy the code
```

Ffplay can specify which audio stream or video stream to use for playback

Mkv-ast 1 // the first audio stream in the video is played. If ast is followed by 2, the second audio stream is played. If there is no second audio stream, the mute is played. Mkv-vst 1 // if ast is followed by 2, the second stream will be played. If there is no second stream, the screen will be black, i.e. nothing will be displayed.Copy the code

Play the YUV file

ffplay -f rawvideo -video_size width*height testVideo.yuv
Copy the code

Play the PCM file

ffplay song.pcm -f s16le -channels 2 -ar 44100
Copy the code

ffplay -f s16le -ar 44100 -ac 1 song.pcm
Copy the code

-f indicates the audio format, and you can use the ffmpeg-formats command to see a list of supported formats:

``` qis-Mac-mini:tvuDebug qi$ ffmpeg -formats | grep PCM DE alaw PCM A-law DE f32be PCM 32-bit floating-point big-endian  DE f32le PCM 32-bit floating-point little-endian DE f64be PCM 64-bit floating-point big-endian DE f64le PCM 64-bit floating-point little-endian DE mulaw PCM mu-law DE s16be PCM signed 16-bit big-endian DE s16le PCM signed 16-bit little-endian DE s24be PCM signed 24-bit big-endian DE s24le PCM signed 24-bit little-endian DE s32be PCM signed 32-bit big-endian DE s32le PCM signed 32-bit little-endian DE s8 PCM signed 8-bit DE u16be PCM unsigned 16-bit big-endian DE u16le PCM unsigned 16-bit little-endian DE u24be PCM unsigned 24-bit big-endian DE u24le PCM unsigned 24-bit little-endian DE u32be PCM unsigned 32-bit big-endian DE u32le PCM unsigned 32-bit little-endian DE u8 PCM unsigned 8-bit ```Copy the code

Play YUV420P format video frames

ffplay -f rawvideo -pixel_format yuv420p -s 480*480 texture.yuv
Copy the code

Playing RGB images

ffplay -f rawvideo -pixel_format rgb24 -s 480*480 texture.rgb
Copy the code

For the video player, we have to mention a problem is the synchronization of audio and picture. There are three ways to realize the synchronization of audio and picture in FFPlay, which are: audio as the main timeline as the synchronization source; The main video timeline as the synchronization source; An external clock is used as the synchronization source.

The default alignment in ffPlay is based on audio, so how does this work?

Both video and audio frames received by the player are timestamp (PTS clock) to identify when it is actually presented. The actual alignment strategy is as follows: Compare the current playing time of the video with the current playing time of the audio. If the video is playing too fast, increase the delay or repeat the video to slow down the playback speed. If the video is slow, catch up with the audio by reducing the delay or dropping frames. The key lies in the comparison of audio and video time and the calculation of delay. Of course, a Threshold will be set during the comparison. If the preset Threshold is exceeded, adjustments should be made (frame loss rendering or repeated rendering), which is the alignment strategy.

For ffplay you can specify exactly which alignment is used:

Audio and video synchronization based on audio
```
ffplay test.mp4 -sync audio
Copy the code
```
Audio and video synchronization based on video
```
ffplay test.mp4 -sync video
Copy the code
```
Audio and video synchronization based on the external clock
```
ffplay test.mp4 -sync ext
Copy the code
```

ffmpeg

Ffmpeg is a powerful media file conversion tool. It can convert media files in any format, and it can also use its own audiofilters and videofilters for processing and editing. In short, it can do anything you want to do with offline video processing.

Lists all formats supported by FFMPEG
```
ffmpeg -formats
Copy the code
```

Cut a media file, either audio or video

Ffmpeg-i input. Mp4-ss 00:00:50.0 -codec copy -t 20 output. Mp4 -ss specifies the offset time and -t specifies the durationCopy the code

If you record a long video in your phone that cannot be shared in wechat, you can use FFMPEG to split the file into multiple files
```
ffmpeg -i input.mp4 -t 00:00:50 -c copy small-1.mp4 -ss 00:00:50 -codec copy small-2.mp4
Copy the code
```

Extract audio files from a video file

ffmpeg -i input.mp4 -vn -acodec copy output.m4a
Copy the code

Mute the audio in a video, that is, keep only the video

ffmpeg -i input.mp4 -an -vcodec copy output.mp4
Copy the code

Extract video streams from MP4 files and export them as raw H264 data

ffmpeg -i output.mp4 -an -vcodec copy -bsf:v h264_mp4toannexb output.h264
Copy the code

Use AAC audio data and H264 video to generate MP4 files
```
ffmpeg -i test.aac -i test.h264 -acodec copy -bsf:a aac_adtstoasc -vcodec copy -f mp4 output.mp4
Copy the code
```
A bitstream filter named AAC_ADtSTOASc is used in the above code. The AAC format also has two encapsulation formats.

Convert the encoding format of audio files

ffmpeg -i input.wav -acodec libfdk_aac output.aac
Copy the code

PCM raw data from WAV audio files everywhere

ffmpeg -i input.wav -acodec pcm_s16le -f s16le output.pcm
Copy the code

In this way, PCM data representing a sample with 16 bits can be exported, and the byte order of each sample is the format of small-tail representation. The number of channels and the sampling rate are all THE PCM data of the number of channels and the sampling rate of WAV files.

Re-encode the video file, copy the audio stream, and encapsulate it in MP4 format
```
ffmpeg -i input.flv -vcodec libx264 -acodec copy output.mp4
Copy the code
```

Convert an MP4 video into a GIT GIF

ffmpeg -i input.mp4 -vf scale=100:-1 -t 5 -r 10 image.gif
Copy the code

The above code according to the resolution ratio fixed width changed to 100(using VideoFilter scaleFilter), frame rate changed to 10(-R), only processing the first 5 seconds (-t) of the video, generated GIF.

This command can be used to generate images from a video, for example, to analyze what each frame of a video is
```
Ffmpeg-i output.mp4-r 0.25 frames_% 04D.png ffmPEg-I output.mp4-r 0.25 frames_% 04D.pngCopy the code
```

This command takes a video frame every four seconds and generates an image, starting with frames_0001.png.

A GIF can be made from a group of images. If you take a group of photos in a row, you can use the following command to create a GIF
```
ffmpeg -i frames_%04.png -r 5 output.gif
Copy the code
```
Using the Volume Effect, you can change the volume in an audio media file
```
ffmpeg -i input.wav -af 'volume = 0.5' output.wav
Copy the code
```

Wav file, which can be played directly to listen to, or put into some audio editing software to directly see the effect of the waveform amplitude.

Use of the fade-in effector

ffmpeg -i input.wav -filter_complex afade=t=in:ss=0:d=5 output.wav
Copy the code

Wav. You can drag the files before and after processing to the Audacity audio editing software to view the waveform.

Use of fade-out effector

ffmpeg -i input.wav -filter_complex afade=t=out:st=200:d=5 output.wav
Copy the code

Wav file from 200s, do 5s fade effect, and put in output.wav file

* Combine two sounds, such as a piece of background music

	ffmpeg -i vocal.wav -i accompany.wav -filter_complex amix=inputs=2:duration=shortest output.wav
Copy the code

Wav = silp. wav = silp. wav = silp. wav = silp. wav

The use of an effector that changes the speed but does not change the tone of sound
```
Ffmpeg -I vocal. Wav -filter_complex atempo=0.5 output.wavCopy the code
```

Wav generates output.wav at the speed of 0.5 times, the time will be twice as long as input. But the pitch doesn’t change, and that’s what we call shifting gears.

Add a watermark effect to the video

ffmpeg -i input.mp4 -i changeba_icon.png -filter_complex '[0:v][1:v]overlay=main_w-overlay_w-10:10:1[out]' -map '[out]' output.mp4
Copy the code

The above command contains several built-in parameters, main_w for the main video width, overlay_w for the watermark width, main_h for the main video height, overlay_h for the watermark height.

Use of video enhancers

Ffmpeg-i input. Fly -c:v libx264-b :v 800k -c:a libfdk_aac -vf eq=brightness=0.25-f mp4 output.mp4
Copy the code

The highlight parameter is bitrate, which ranges from -1.0 to 1.0. The default value is 0

Add contrast to the video

FLV -c:v libx264 -b:v 800k -c:a libfdk_aac-vf eq=contrast=1.5-f mp4 output.mp4
Copy the code

The contrast parameter is contrast, and the value ranges from -2.0 to 2.0. The default value is 1.0

The use of video rotation effector

ffmpeg -i input.mp4 -vf "transpose=1" -b:v 600k output.mp4
Copy the code

The use of video clipping effects

ffmpeg -i input.mp4 -an -vf "crop=240:480:120:0" -vcodec libx264 -b:v 600k output.mp4
Copy the code

Compile ffMPEG library for ios development on MAC platform

Install homebrew

Homebrew is a free and open source package management system designed to simplify software installation on Mac OS X. It was originally written by Max Howell. It has been praised for its extensibility [1] and is well known in the Ruby on Rails community.

how to install HomeBrew

Download the compilation script file

We use gas-preprocessor to compile ffMPEG scripts.

Copy the gas-preprocessor.pl file to /usr/local/bin/ and then grant executable permission to the file:

chmod 777 /usr/local/bin/gas-preprocessor.pl
Copy the code

Install yasm

In the computer world, Yasm is an assembler and disassembler under Intel x86 architecture. It can be used to write 16-bit, 32-bit (IA-32), and 64-bit (x86-64) programs. Yasm is a completely rewritten Netwide assembler (NASM). Yasm is often used interchangeably with NASM and supports both x86 and x86-64 architectures.

Install ` yasm

brew install yasm
Copy the code

Ffmpeg ios compilation script

FFmpeg iOS build script

The above is ffMPEG compilation script, can compile the ios platform library. Details can be found on the wiki.

To compile the library we need, run the following command directly (optionally which CPU architecture to compile) :

 ./build-ffmpeg-iOS-framework.sh 
Copy the code

After a successful compilation, the directory structure looks like this:

Introduce FFMPEG into your project

Add the ffMPEG library ffMPEG-ios to the project as follows:

libiconv.tdb
libbz2.tbd
libz.tbd

A brief introduction to FFMPEG

Ffmpeg library

Ffmpeg contains a total of eight libraries:

Avcodec: Codec (most important library)
Avformat: Encapsulates format processing
Avfilter: filter effect processing
Avdevice: Input and output of various devices
Avutil: Tool library (most libraries need this library support)
Postproc: post-processing
Swresample: Audio sampling data format conversion
Swscale: Video pixel data format conversion

Ffmpeg data structure analysis

AVFormatContext: encapsulating format context structure, is also the global structure, holds video file encapsulation format information
- Iformat: Enter the AVInputFormat of the video
- Nb_streams: Indicates the number of avstreams of videos
- Streams: An AVStream[] array of input videos
- Duration: Enter the duration of the video (in subtleties)
- Bit_rate: input video bit rate
AVInputFormat: Each package format (such as FLV,MKV,MP4, AVI) corresponds to a structure
- Name: indicates the name of the encapsulation format
- Long_name: Long name of the encapsulation format
- Extensions: Indicates the extension of the package format
- Id: INDICATES the ID of the encapsulation format
AVStream: a structure for each video (audio) stream in a video file
- Id: serial number
- Codec: AVCodecContext corresponding to this stream
- Time_base: Time base for the flow
- R_frame_rate: frame rate of the stream
AVCodecContext: Encoder context structure that holds information about the video (audio) codec
- Codec: AVCodec of the codec
- Width,height: the width and height of the image (for video only)
- Pix_fmt: Pixel format (for video only)
- Sample_rate: sampling rate (audio only)
- -Leonard: The number of channels
- Sample_fmt: Sampling format (audio only)
AVCodec: One for each video (audio) codec, such as an H264 decoder
- Name: codec name
- Long_name: codec length name
- Type: codec type
- Id: indicates the ID of the codec
AVPacket: Stores a frame of compressed encoded data
- PTS: Displays the timestamp
- DTS: timestamp of decoding
- Data: compressed and encoded data
- Size: indicates the size of compressed encoded data
- Stream_index: AVStream to which it belongs
AVFrame: Stores a frame of decoded pixel (sampled) data
- Data: decoded image pixel data (audio sampling data)
- Linesize: The size of a line of pixels in an image for video, the size of an entire audio frame for audio
- Width,height: the width and height of the image (for video only)
- Key_frame: Whether it is a key frame (for video only)
- Pict_type: frame type (for video only). For instance I, B, P

Using FFMPEG to do H264 soft coding for real-time video

The general process is as follows:

#import "MISoftH264Encoder.h"



@implementation MISoftH264Encoder
{
    AVFormatContext             *pFormatCtx;
    AVOutputFormat              *out_fmt;
    AVStream                    *video_stream;
    AVCodecContext              *pCodecCtx;
    AVCodec                     *pCodec;
    AVPacket                    pkt;
    uint8_t                     *picture_buf;
    AVFrame                     *pFrame;
    int                         picture_size;
    int                         y_size;
    int                         framecnt;
    char                        *out_file;
    
    int                         encoder_h264_frame_width;
    int                         encoder_h264_frame_height;
}

- (instancetype)init
{
    if (self = [super init]) {

    }
    return self;
}

static MISoftH264Encoder *miSoftEncoder_Instance = nil;
+ (instancetype)getInstance
{
    if (miSoftEncoder_Instance == NULL) {
        miSoftEncoder_Instance = [[MISoftH264Encoder alloc] init];
    }
    return miSoftEncoder_Instance;
}

- (void)setFileSavedPath:(NSString *)path
{
    NSUInteger len = [path length];
    char *filepath = (char*)malloc(sizeof(char) * (len + 1));
    [path getCString:filepath maxLength:len + 1 encoding:[NSString defaultCStringEncoding]];
    out_file = filepath;
}

- (int)setEncoderVideoWidth:(int)width height:(int)height bitrate:(int)bitrate { framecnt = 0; encoder_h264_frame_width = width; encoder_h264_frame_height = height; av_register_all(); pFormatCtx = avformat_alloc_context(); // Set the output file path out_fmt = av_guess_format(NULL, out_file, NULL); pFormatCtx->oformat = out_fmt; // Open file buffer input/output, flags is AVIO_FLAG_READ_WRITE, read and writeif (avio_open(&pFormatCtx->pb, out_file, AVIO_FLAG_READ_WRITE) < 0){
        printf("Failed to open output file! \n");
        return- 1; } // Create a new output stream for writing files video_stream = avFormat_new_stream (pFormatCtx, 0); Video_stream ->time_base.num = 1; video_stream->time_base.den = 30;if (video_stream==NULL){
        return- 1; PCodecCtx = video_stream->codec; // Set the encoder encoding format (is an ID), each encoder corresponds to its own ID, for example, h264 encoding id is AV_CODEC_ID_H264 pCodecCtx->codec_id = out_fmt->video_codec; pCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO; pCodecCtx->pix_fmt = AV_PIX_FMT_YUV420P; // AV_PIX_FMT_YUV420P pCodecCtx->width = encoder_h264_frame_width; pCodecCtx->height = encoder_h264_frame_height; pCodecCtx->time_base.num = 1; pCodecCtx->time_base.den = 30; pCodecCtx->bit_rate = bitrate; PCodecCtx ->qmin =10; pCodecCtx->qmax = 51; // // Set the size of the image group layer (GOP--> interval between two I frames) // pCodecCtx->gop_size = 30; // // // set the maximum number of B frames. B frames are the pre and post prediction frames of video picture space. B frames have a higher compression rate than I and P frames, that is to say, at the same bit rate, // //, the more B frames of video, the clearer. // pCodecCtx->max_b_frames = 5; // pCodecCtx->max_b_frames = 5; // // // Optional AVDictionary *param = 0; // H.264if(pCodecCtx->codec_id == AV_CODEC_ID_H264) {// Adjust the encoding speed and quality balance using the -- PRESET parameter. av_dict_set(&param,"preset"."slow", 0); // Specify the type of tune, visual optimization, or special case by using the --tune parameter value. // zerolatency: zerolatency, used when very low latency is required, such as the encoding of live video"tune"."zerolatency", 0); } // Outputs the printed information internally throughprintf// av_dump_format(pFormatCtx, 0, out_file, 1); PCodec = avcodec_find_encoder(pCodecCtx->codec_id);if(! pCodec) {printf("Can not find encoder! \n");
        return- 1; } // Open the encoder and set the parameter paramif (avcodec_open2(pCodecCtx, pCodec,&param) < 0) {
        printf("Failed to open encoder! \n");
        return- 1; } // Initialize the raw data object: AVFrame pFrame = av_frame_alloc(); // Get the true size of the image in pixel format (in this case YUV), For example convert 1080 * 1920 to int avpicture_fill((AVPicture *)pFrame, picture_buf, pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height); Avformat_write_header (pFormatCtx, NULL); avformat_write_header(pFormatCtx, NULL) Av_new_packet (& PKT, picture_size); // Create av_new_packet(& PKT, picture_size);return0; } /* * encoderToH264:(CMSampleBufferRef)sampleBuffer {// encoderToH264:(CMSampleBufferRef Through CMSampleBufferRef object access CVPixelBufferRef object CVPixelBufferRef imageBuffer = CMSampleBufferGetImageBuffer (sampleBuffer); // Lock the imageBuffer memory address to start encodingif(CVPixelBufferLockBaseAddress(imageBuffer, 0) == kCVReturnSuccess) { // 3. Read the YUV value from CVPixelBufferRef UInt8 * bufferPtr = (UInt8 *) CVPixelBufferGetBaseAddressOfPlane (imageBuffer, 0); UInt8 *bufferPtr1 = (UInt8 *)CVPixelBufferGetBaseAddressOfPlane(imageBuffer,1); size_t width = CVPixelBufferGetWidth(imageBuffer); size_t height = CVPixelBufferGetHeight(imageBuffer); size_t bytesrow0 = CVPixelBufferGetBytesPerRowOfPlane(imageBuffer,0); size_t bytesrow1 = CVPixelBufferGetBytesPerRowOfPlane(imageBuffer,1); UInt8 *yuv420_data = (UInt8 *)malloc(width * height *3/2); UInt8 *pY = bufferPtr ; UInt8 *pUV = bufferPtr1; UInt8 *pU = yuv420_data + width*height; UInt8 *pV = pU + width*height/4;for(int i =0; i<height; i++) { memcpy(yuv420_data+i*width,pY+i*bytesrow0,width); }for(int j = 0; j<height/2; j++) {for(int i =0; i<width/2; i++) { *(pU++) = pUV[i<<1]; *(pV++) = pUV[(i<<1) + 1]; } pUV+=bytesrow1; Picture_buf = yuv420_data; picture_buf = yuv420_data; y_size = pCodecCtx->width * pCodecCtx->height; pFrame->data[0] = picture_buf; // Y pFrame->data[1] = picture_buf+ y_size; // U pFrame->data[2] = picture_buf+ y_size*5/4; // V // 4. Set pFrame-> PTS = framecnt; int got_picture = 0; ->width = encoder_h264_frame_width; pFrame->height = encoder_h264_frame_height; pFrame->format = AV_PIX_FMT_YUV420P; // Use the AVFormat to encode the original data. Int RET = AVCODEC_enCODE_videO2 (pCodecCtx, & PKT, pFrame, & GOt_picture);if(ret < 0) {
            printf("Failed to encode! \n");
        }else if (ret == 0){
            if (pkt.buf) {
                printf("encode success, data length: %d \n",pkt.buf->size); }} // Write AVPacket to output after encoding successfullyif(got_picture == 1) {// the description is not empty, now write data to the output file framecnt++; pkt.stream_index = video_stream->index; ret = av_write_frame(pFormatCtx, &pkt); av_free_packet(&pkt); } free(yuv420_data); } CVPixelBufferUnlockBaseAddress(imageBuffer, 0); } /* * Free resources */ - (void)freeH264Resource {// 1 AVFormatContext int ret = flush_encoder(pFormatCtx,0);if (ret < 0) {
        printf("Flushing encoder failed\n"); } // Output av_write_trailer(pFormatCtx) if AVPacket has not been output; // Close the resourceif (video_stream){
        avcodec_close(video_stream->codec);
        av_free(pFrame);
    }
    avio_close(pFormatCtx->pb);
    avformat_free_context(pFormatCtx);
}

int flush_encoder(AVFormatContext *fmt_ctx,unsigned int stream_index)
{
    int ret;
    int got_frame;
    AVPacket enc_pkt;
    if(! (fmt_ctx->streams[stream_index]->codec->codec->capabilities & CODEC_CAP_DELAY))return 0;
    
    while (1) {
        enc_pkt.data = NULL;
        enc_pkt.size = 0;
        av_init_packet(&enc_pkt);
        ret = avcodec_encode_video2 (fmt_ctx->streams[stream_index]->codec, &enc_pkt,
                                     NULL, &got_frame);
        av_frame_free(NULL);
        if (ret < 0)
            break;
        if(! got_frame){ ret=0;break;
        }
        ret = av_write_frame(fmt_ctx, &enc_pkt);
        if (ret < 0)
            break;
    }
    return ret;
}


@end

Copy the code

Enter the sandbox directory and use ffPlay to play the H264 file:

ffplay softEncoder.h264
Copy the code

In the next article we will cover how to do mux for audio and video.

Hard coding and soft coding for VIDEO H264 on ios platform

An overview of the

Using VideoToolBox to do H264 hard coding for real-time video

Initializing the encoder

Hardcode CMSampleBufferRef with VTCompressionSession

In the callback function, the encoded code is converted to H264 stream structure

ffmpeg

Install FFMPEG on the MAC platform

Simple common ffmpeg command

ffprobe

ffplay

ffmpeg

Compile ffMPEG library for ios development on MAC platform

Install homebrew

Download the compilation script file

Install yasm

Ffmpeg ios compilation script

Introduce FFMPEG into your project

A brief introduction to FFMPEG

Ffmpeg library

Ffmpeg data structure analysis

Using FFMPEG to do H264 soft coding for real-time video

Related Posts

Have you had enough on Chinese Valentine’s Day? To digest my interview questions (with answers)

Alamofire (5) — Response

IOS UIScrollView, UIWebView, UICollectionView to achieve text and text mix