This article was originally published on the wechat official account — Interesting Things in the world, handling, reprint please note the source, otherwise will be held liable for copyright. Wechat id: A1018998632, QQ group: 859640274

• 1. Copy a Douyin app from scratch — Start
• 4. Copy a Douyin App from scratch — log and burial points and the initial architecture of the back end
• 5. Copy a Douyin App from scratch — App architecture update and network layer customization
• 7. Copy a Douyin App from scratch — a minimalist video player based on FFmpeg
• 8. Copy a Douyin App from scratch — build a cross-platform video editing SDK project

Making the address

Hi, everybody. It’s been five weeks since I last posted on this topic, and with two non-posts in between, many people probably thought I was going to give up. But that is not the case, mainly work is a little busy, and I actually have a lot of things in audio and video need to learn and organize. So from the beginning of this article we will enter the audio and video field to study, the Android field of the article will be in the middle of the integration of audio and video code when interspersed. In fact, there are a lot of things to talk about in Android, but time waits for no one. Without further ado, let’s get into the article. The reading time is estimated at 20 minutes.

This article is divided into the following sections and can be read as needed

• 1. Have a chat — mainly to announce something that can be skipped if you are not interested.
• 2. Audio/video Preknowledge — Make a list of things you need to know before learning audio/video technology.
• 3. Getting Started with CMake — Learn how to organize and compile a C/C ++ project.
• 4. Ffmpeg Introduction — Introduction to FFMEPG, explain an official demo.

Have a chat

• 1. Some of you may find that two articles are missing from this topic. This problem is explained here. I dropped the article because of my personal reasons. I will revise the two articles at an appropriate time and then republish them. If you really need to talk with me in private, I will send it to you.
• 2. Friends of Jane Book may find that there is a label of Jane Book copyright on my personal interface, which indicates that I have signed a contract with Jane Book. After accumulating certain saved articles, I will cooperate with Jane Book to publish a book on Android+ audio and video technology + deep learning + short video. The book is expected to be written in half a year, so let’s warm up now, haha :).
• 3. Those who follow me on Github will see that I moved MyTikTok to TheGodsThemselves, where all the official projects on this topic will be posted, and perhaps the first student will join the organization before long. In addition, here are the ideal goals of this project: Complete copy of dacang’s project process, write a short video App(tentatively imitate Douyin), and use interesting technologies of various ends (including but not limited to Android, IOS, background, front end, algorithm, audio and video) in the project. Students who participate and follow this project can learn the technology stack they are interested in (the real development experience of Daco).

Second, audio and video pre-knowledge

In fact, I have explained roughly what needs to be learned in learning audio and video technology in my path of technical growth. In this section, I will talk about some specific things. Of course, it is only a superficial introduction, and readers still need to accumulate more in-depth knowledge.

1. Multimedia concepts

• 1. Video format: USB camera output formats are RGB24, YUV2, YV2, these are unencoded raw data, MJPEG is encoded data
• 2. Audio format: Many
• 3. Container and Protocol: Container refers to an audio/video file format such as.avi, and protocol refers to the way the data stored in an audio/video file is codec. A container can contain various codec types of data, each of which requires a different codec. MPEG, H.26X and other encoding methods are common. AVI, MPG, MP4 and other containers are common.
  • 1. Container file format: A container file usually consists of three parts: header, index, and multimedia data.
    • 1. File headers often indicate the resolution, frame rate, audio sampling rate, and other specifications of multimedia data
    • 2. The index section is used to record the location of multimedia data in the file, because multimedia data is not necessarily continuous, and there may be audio and video synchronization indexes, etc. When you play a video, you often read the entire index into memory.
    • 3. The multimedia data section is the storage of compressed video, audio, text data and so on.
  • 2. Protocol: video compression protocol h.26X, MPEG-X and so on. H. 265 is the latest compression protocol. Audio compression protocols are G. 7XX and so on
• 4. Common concepts:
  • 1. Hard decoding: Do not allow the CPU to participate in decoding, but use a specialized device for decoding, which is generally integrated in the GPU. The advantage of hard decoding is that it is much faster than CPU and saves CPU resources. The disadvantage is that it starts late and has little software support, is not compatible with a variety of different codec methods and file formats, does not have the benefits of image quality enhancement like soft solution, and is difficult to hard decode the GPU.
  • 2. Ibp frame: GOP is a group of frame. A video file is composed of N GOP. A GOP is divided into I, B and P frames.
    • 1. I is a full frame, which is equivalent to the compressed data of a picture. It can recover a display frame by itself, and the compression rate is about 7 times
    • 2. P is a forward prediction frame, it needs to rely on I frame to decode, it uses motion compensation to transmit the error with the previous I or P frame, and then reconstructed a display frame, I and P can be used as the preframe of P frame. Because the P-frame can be used as a reference for the post-frame, it can cause decoding errors to spread. The compression ratio is 20 to the left.
    • 3. B is a two-way predictive frame, which needs to rely on the I or P frame in front and the P frame in the back to decode. The compression rate is about 50 times, and the decoding is troublesome because of the high compression rate. You need to know the postframe in advance, so you need to preread the predecode.
    • 4. In MPEG4, each frame begins with 00 00 01 B6, and the following two bits indicate which frame the current frame belongs to. 00 is I, 01 is P, and 10 is B.

2.FFmpeg basic concepts

• 1. Module composition:
  • 1. Libavformat: parse audio and video files in various formats, read audio and video frames for decoding information, and provide independent audio and video streams for LibavCode
  • 2. Libavcodec: A codec for various codec protocols
  • 3. Libavdevice: Hardware collection, acceleration, display video.
  • 4. Libavfilter: Convert videos, such as clipping, scaling, aspect ratio, etc
  • 5. Libavutil: the tool library
  • 6. Libavresample:..
  • 7. Libswscale: Scale, color mapping conversion, image color space conversion
  • 8. Libpostproc: Audio and video post effects processing
  • 9. Ffmpeg: An externally exposed tool
  • 10. Ffplay: Simple player that uses the FFMPEG library for parsing and decoding
• 2. In general, FFmpeg is a c language written library. It consists of the modules above. It’s not a player, it’s the core component of the player. For example, I need to write a player on Windows, and we have an MP4 file, so the player will play the video by following these steps: FFmpeg parses file format — >FFmpeg reads file data — >FFmpeg decodes file data to restore data to image frames — > Windows Api displays image frames. If I wanted to write a player again on Android I would use the same three steps and replace the last one with Opengl ES + TextureSurfaceView to display the image frames. From this we can find that FFmpeg is cross-platform, the core logic of video playback as long as the use of FFmpeg then in each platform does not need a big change, the need to change is only each platform display picture frame logic.
• 3.FFmpeg has an FFMEpg module. When you have FFmpeg installed on your computer, you can use the command line to call the functions exposed by FFmpeg to process the video.

3, Cmake entry

Cmake is a tool for organizing C/Cpp projects, similar to gradle we use for Android. We’re going to write a larger tool, and a project management tool like Cmake is essential. In this section, we’ll take a look at Cmake. Note that the following tutorial is a translation of the official tutorial.

Here are the corresponding items for this chapter:Cmake_learning project

1. Prepare the compiler

I use CLion, which is also a member of the JetBrain family, because my main Mac is my IDE. If you are using Android Studio or IDEA, you can easily switch to this IDE. CLion is also a cross-platform IDE, meaning it can be used on Windows And Linux. Of course, Visual Studio will always be the strongest IDE(manual). It is important to note that CLion requires a fee to buy an activation code. It seems that there is no free version of CLion that can be used for a free trial for a month or so, so it is up to everyone to get the activation code.

2.Cmake

(1) the most basic Cmake program

• 1. When we entered the directory of one/ A in the project, we found two files: cmakelists. TXT and tutorial. CPP.

  • 1. We wrote a CPP code to calculate the square root, and then put it into the Tutorial project.
  • 2. We create a build directory in a, go to this directory on the command line, and run cmake.. With this command, we can see that several files are generated under build, which are the files we need to make.
  • 3. We finally run the make command in the build folder, which will generate a Tutorial executable file, this is the final Tutorial project product, we can type./Tutorial 3 to calculate the square root of 3.

  A cmake project has at least the following three lines of code
  cmake_minimum_required (VERSION 2.6) # indicates the minimum version of cmake
  project (Tutorial)# Create a new project named Tutorial
  add_executable(Tutorial tutorial.cpp) Add an executable file tutorial.the CPP to the Tutorial project
  # 1. Cmake syntax supports size, lowercase, and case mixing. For example, project above can be written as project
  Copy the code

  //
  // Created by
  // 
  #include <stdio.h>
  #include <stdlib.h>
  #include <math.h>
  int main (int argc, char *argv[]) {
      if (argc < 2) {
          fprintf(stdout."usage: %s number\n", argv[0]);
          return 1;
      }
      double inputValue = atof(argv[1]);
      double outputValue = sqrt(inputValue);
      fprintf(stdout."The square root of %g is %g\n", inputValue, outputValue);
      return 0;
  }
  Copy the code

• 2. When we entered the directory of one/b in the project, we found three files: cmakelists. TXT, tutorial.cpp, tutorialconfig.h.i.

  • 1. Declare two parameters in the Tutorial_A project and reference the two parameters in the tutorialconfig.h.id file. Cmake generates a file named tutorialconfig.h based on this file.
  • 2. We use tutorialconfig. h in tutorial.cpp, which uses the parameters defined in the cmake file. This is very similar to when we were developing Android and defining parameters in Gradle files and then using them in Java code.
  • 3. Let’s run cmake in the build file. , make,./Tutorial_A. You’ll see that the parameters we used are printed.

cmake_minimum_required (VERSION 2.6)
project (Tutorial_A)
We can add a set(KEY VALUE) to the cmake program. Here is how to set a KEY VALUE.
${KEY} = ${KEY}
set (Tutorial_VERSION_MAJOR 1)
set (Tutorial_VERSION_MINOR 0)

In tutorialconfig.h.id, we can set the key-value pairs set in set()
## PROJECT_SOURCE_DIR indicates the path to the source code
## PROJECT_BINARY_DIR indicates the path to the cmake build
configure_file (
        "${PROJECT_SOURCE_DIR}/TutorialConfig.h.in"
        "${PROJECT_BINARY_DIR}/TutorialConfig.h"
)

# Add cmake's build directory to the list of directories where cmake looks for include files, so that cmake can find the previously generated tutorialconfig. h configuration file
include_directories("${PROJECT_BINARY_DIR}")

add_executable(Tutorial_A tutorial.cpp)
Copy the code

// A simple program that computes the square root of a number
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
// include: cmake generates the configuration file
#include "TutorialConfig.h"

int main (int argc, char *argv[])
{
    if (argc < 2)
    {
        fprintf(stdout."%s Version %d.%d\n",
                argv[0].// Use the configuration parameters generated by cmake
                Tutorial_VERSION_MAJOR,
                Tutorial_VERSION_MINOR);
        fprintf(stdout."Usage: %s number\n",argv[0]);
        return 1;
    }
    double inputValue = atof(argv[1]);
    double outputValue = sqrt(inputValue);
    fprintf(stdout."The square root of %g is %g\n",
            inputValue, outputValue);
    return 0;
}
Copy the code

// This is the configuration file, and cmake will generate a tutorialconfig. h file based on his build directory in cmake
#define Tutorial_VERSION_MAJOR @Tutorial_VERSION_MAJOR@
#define Tutorial_VERSION_MINOR @Tutorial_VERSION_MINOR
Copy the code

(2). Add library dependencies

• 1. We will see three files cmakelists. TXT, mysqrt. CPP and mathfunctions. h in two/a/mylib of the project:
  • 1. Declare a library
  • 2. Define a function that calculates the square root and expose it using a header file

cmake_minimum_required (VERSION 2.6)
# declare a library named MathFunctions, which contains an executable file mysqrt.cpp
add_library(MathFunctions mysqrt.cpp)
Copy the code

#include "MathFunctions.h"
#include <stdio.h>

// a hack square root calculation using simple operations
double mysqrt(double x)
{
    if (x <= 0) {
        return 0;
    }

    double result;
    double delta;
    result = x;

    // do ten iterations
    int i;
    for (i = 0; i < 10; ++i) {
        if (result <= 0) {
            result = 0.1;
        }
        delta = x - (result * result);
        result = result + 0.5 * delta / result;
        fprintf(stdout."Computing sqrt of %g to be %g\n", x, result);
    }
    return result;
}
Copy the code

//
// Created by
//

#ifndef PROJECT_MATHFUNCTIONS_H
#define PROJECT_MATHFUNCTIONS_H
double mysqrt(double x);
#endif //PROJECT_MATHFUNCTIONS_H
Copy the code

• Cmakelists.txt, cmakelists.txt, cmakelists.txt, cmakelists.txt, cmakelists.txt, cmakelists.txt,Configure.h.in, mathfunctions.h, tutorial.cpp:
  • When the switch is ON, we will integrate the library we defined into the project. Otherwise, we will not integrate the library we defined into the project. We will only use the system’s own library. This is very useful when you’re cross-platform, for example, the log library in ios and Android is different, so I can define a switch to distinguish between the two platforms.
  • 2. Note that there is also a configure.h.id file defined as the configuration file. Cmake will create a configure.h file from this file, and then we can use the switch we defined in the Cpp file.
  • 3. We can run cmake in two/a/build. Make,./Tutorial_Mylib 3. If you change USE_MYMATH to OFF and delete the files in the build, you will find that the system functions are called.

cmake_minimum_required (VERSION 2.6)
project (Tutorial_Mylib)

set (Tutorial_VERSION_MAJOR 1)
set (Tutorial_VERSION_MINOR 0)

configure_file (
        "${PROJECT_SOURCE_DIR}/TutorialConfig.h.in"
        "${PROJECT_BINARY_DIR}/TutorialConfig.h"# Add a switch to use our own library, USE_MYMATH. This switch can be used directly in cmake using option (USE_MYMATH)"Use tutorial provided math implementation"ON) # Define a file to store USE_MYMATH so you can use configure_file() in the CPP file"${PROJECT_SOURCE_DIR}/Configure.h.in"
        "${PROJECT_BINARY_DIR}/Configure.h")

include_directories("${PROJECT_BINARY_DIR}"If we set the switch to ON, then we integrate Mylib into the compilation, otherwise we don't.if (USE_MYMATH)
    include_directories ("${PROJECT_SOURCE_DIR}/mylib")
    add_subdirectory (mylib)
    set(EXTRA_LIBS MathFunctions) endif (USE_MYMATH) add_executable (Tutorial_Mylib tutorial. CPP) Make library function target_link_libraries available for project calls (Tutorial_Mylib ${EXTRA_LIBS})Copy the code

#cmakedefine USE_MYMATH
Copy the code

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include "TutorialConfig.h"
#include "Configure.h"
#ifdef USE_MYMATH
#include "MathFunctions.h"
#endif

int main (int argc, char *argv[])
{
    if (argc < 2)
    {
        fprintf(stdout."%s Version %d.%d\n", argv[0],
                Tutorial_VERSION_MAJOR,
                Tutorial_VERSION_MINOR);
        fprintf(stdout."Usage: %s number\n",argv[0]);
        return 1;
    }

    double inputValue = atof(argv[1]);

#ifdef USE_MYMATH
    // If the switch is on, use my own library
    double outputValue = mysqrt(inputValue);
    fprintf(stdout."use my math");
#else
    double outputValue = sqrt(inputValue);
    fprintf(stdout."not use my math");
#endif

    fprintf(stdout."The square root of %g is %g\n",
            inputValue, outputValue);
    return 0;
}
Copy the code

(2). Add library dependencies

• 1. We will see three files cmakelists. TXT, mysqrt. CPP and mathfunctions. h in two/a/mylib of the project:
  • 1. Declare a library
  • 2. Define a function that calculates the square root and expose it using a header file

cmake_minimum_required (VERSION 2.6)
# declare a library named MathFunctions, which contains an executable file mysqrt.cpp
add_library(MathFunctions mysqrt.cpp)
Copy the code

#include "MathFunctions.h"
#include <stdio.h>

// a hack square root calculation using simple operations
double mysqrt(double x)
{
    if (x <= 0) {
        return 0;
    }

    double result;
    double delta;
    result = x;

    // do ten iterations
    int i;
    for (i = 0; i < 10; ++i) {
        if (result <= 0) {
            result = 0.1;
        }
        delta = x - (result * result);
        result = result + 0.5 * delta / result;
        fprintf(stdout."Computing sqrt of %g to be %g\n", x, result);
    }
    return result;
}
Copy the code

//
// Created by
//

#ifndef PROJECT_MATHFUNCTIONS_H
#define PROJECT_MATHFUNCTIONS_H
double mysqrt(double x);
#endif //PROJECT_MATHFUNCTIONS_H
Copy the code

• Cmakelists.txt, cmakelists.txt, cmakelists.txt, cmakelists.txt, cmakelists.txt, cmakelists.txt,Configure.h.in, mathfunctions.h, tutorial.cpp:
  • When the switch is ON, we will integrate the library we defined into the project. Otherwise, we will not integrate the library we defined into the project. We will only use the system’s own library. This is very useful when you’re cross-platform, for example, the log library in ios and Android is different, so I can define a switch to distinguish between the two platforms.
  • 2. Note that there is also a configure.h.id file defined as the configuration file. Cmake will create a configure.h file from this file, and then we can use the switch we defined in the Cpp file.
  • 3. We can run cmake in two/a/build. Make,./Tutorial_Mylib 3. If you change USE_MYMATH to OFF and delete the files in the build, you will find that the system functions are called.

cmake_minimum_required (VERSION 2.6)
project (Tutorial_Mylib)

set (Tutorial_VERSION_MAJOR 1)
set (Tutorial_VERSION_MINOR 0)

configure_file (
        "${PROJECT_SOURCE_DIR}/TutorialConfig.h.in"
        "${PROJECT_BINARY_DIR}/TutorialConfig.h"# Add a switch to use our own library, USE_MYMATH. This switch can be used directly in cmake using option (USE_MYMATH)"Use tutorial provided math implementation"ON) # Define a file to store USE_MYMATH so you can use configure_file() in the CPP file"${PROJECT_SOURCE_DIR}/Configure.h.in"
        "${PROJECT_BINARY_DIR}/Configure.h")

include_directories("${PROJECT_BINARY_DIR}"If we set the switch to ON, then we integrate Mylib into the compilation, otherwise we don't.if (USE_MYMATH)
    include_directories ("${PROJECT_SOURCE_DIR}/mylib")
    add_subdirectory (mylib)
    set(EXTRA_LIBS MathFunctions) endif (USE_MYMATH) add_executable (Tutorial_Mylib tutorial. CPP) Make library function target_link_libraries available for project calls (Tutorial_Mylib ${EXTRA_LIBS})Copy the code

#cmakedefine USE_MYMATH
Copy the code

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include "TutorialConfig.h"
#include "Configure.h"
#ifdef USE_MYMATH
#include "MathFunctions.h"
#endif

int main (int argc, char *argv[])
{
    if (argc < 2)
    {
        fprintf(stdout."%s Version %d.%d\n", argv[0],
                Tutorial_VERSION_MAJOR,
                Tutorial_VERSION_MINOR);
        fprintf(stdout."Usage: %s number\n",argv[0]);
        return 1;
    }

    double inputValue = atof(argv[1]);

#ifdef USE_MYMATH
    // If the switch is on, use my own library
    double outputValue = mysqrt(inputValue);
    fprintf(stdout."use my math");
#else
    double outputValue = sqrt(inputValue);
    fprintf(stdout."not use my math");
#endif

    fprintf(stdout."The square root of %g is %g\n",
            inputValue, outputValue);
    return 0;
}
Copy the code

(3). Install libraries and executable files

• 1. Let’s enter the three/a folder of the project, and all the files in this folder are copied from two/ A. I just list the added code mylib/ cMakelists. TXT, a/ cMakelists. TXT:
  • 1. Here is relatively simple, we just generated library and executable files to install to the computer
  • 2. Run cmake.. /usr/local/bin/Tutorial_Mylib_Install 3 /usr/local/bin/Tutorial_Mylib_Install 3 /usr/local/bin/Tutorial_Mylib_Install

Install the library and add the library and header files to the bin and include folders, respectively
# /usr/local/bin/libMathFunctions_Install.a
# /usr/local/include/MathFunctions.h
install (TARGETS MathFunctions_Install DESTINATION bin)
install (FILES MathFunctions.h DESTINATION include)
Copy the code

TARGETS: ARCHIVE, LIBRARY, RUNTIME, OBJECTS, FRAMEWORK, BUNDLE Note that Mathfunction_Install installs LIBRARY, and Tutorial_Mylib_Install is RUNTIME type.
# FILE Copies the given FILE to the specified directory. If no permission parameter is given, the default files installed by this form are OWNER_WRITE, OWNER_READ, GROUP_READ, and WORLD_READ.
TARGETS and FILE can be specified as relative and absolute directories.
DESTINATION is a relative path here, take the default value. C :/Program Files/${PROJECT_NAME} on Windows.
You can also set the installation path by setting the CMAKE_INSTALL_PREFIX variable, so that the installation location is not /usr/local, but the directory you specified.

Install the executable and add the executable and header files to the bin and include folders, respectively
# /usr/local/bin/Tutorial_Mylib_Install
# /usr/local/include/TutorialConfig.h
install (TARGETS Tutorial_Mylib_Install DESTINATION bin)
install (FILES "${PROJECT_BINARY_DIR}/TutorialConfig.h"
         DESTINATION include)

Copy the code

(4).Cmake generates the Cpp file

• Mylib/cmakelists.txt, mylib/ maketable.cpp, a/ configure.h.id:
  • 1. The main purpose here is to generate a table. h through the MakeTable project. Finally, mysqrt. CPP is used when log and exp are not available in the current system.
  • 2. We run cmake.. You will see that the build/mylib directory generates the table.h file

project(MakeTable)

add_executable(MakeTable MakeTable.cpp)

# 1. Output Table file
# 2. Pass the Table file as a parameter to the MakeTable project and run it
# 3.Table generation depends on the MakeTable project
# CMAKE_CURRENT_BINARY_DIR indicates the folder after the cmake file build, such as build/mylib
add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/Table.h
        COMMAND MakeTable ${CMAKE_CURRENT_BINARY_DIR}/Table.h
        DEPENDS MakeTable)

include_directories(${CMAKE_CURRENT_BINARY_DIR})
# Compile the generated tables into MathFunctions_Table
add_library(MathFunctions_Table mysqrt.cpp ${CMAKE_CURRENT_BINARY_DIR}/Table.h)
Copy the code

//
// Created by
//
#include <stdio.h>
#include <stdlib.h>
#include "math.h"

int main (int argc, char *argv[]) {
    double result;
    if (argc < 2) {
        return 1;
    }
    FILE *fout = fopen(argv[1]."w");
    if(! fout) {return 1;
    }
    fprintf(fout, "double sqrtTable[] = {\n");
    for (int j = 0; j < 10; ++j) {
        result = sqrt(static_cast<double>(j));
        fprintf(fout, "%g,\n", result);
    }
    fprintf(fout, "0}; \n");
    fclose(fout);
    return 0;
}
Copy the code

#cmakedefine USE_MYMATH
#cmakedefine HAVE_LOG
#cmakedefine HAVE_EXP
Copy the code

(5). CMake syntax

• 1. The required, the command can be filled, a | b
• 2. Cmake can organize files in three ways
• 3. Folder format: The root directory of Gradle needs to have a cmakelist.txt file as the entrance, if other directories need to have new subfolders to compile, the subfolders also need to have cMakelist.txt. Add add_subdirectory() to cmakelist.txt. In addition, each cmakelist.txt is processed with the folder called by the cmake command as the current working directory and output directory.
• 4. Set () and unset() are used to define and cancel variables. The defined variable is always a string type, and the variable name is case sensitive. The variable name with{${}}
• 5. Add_excutable () and add_library() are used to generate executable files and libraries, respectively. Add_library () is used to build the Android SO library. Target_link_libraries () is used to link n libraries that are dependent on each other
• 6. Message ([] “message to display”) This method is used to output logs

(6).CMake process statement

• 1. If: The usage is similar to C language. ${} does not need to be used for parameter values
• 2. Foreach: foreach(loop_var 1 2 3)… Endforeach (loop_var) or foreach(loop_var RANGE 4)… Endforeach (loop_var) or foreach(loop_var RANGE 0 3 1)… Endforeach (loop_var) goes from 0 to 3. 1 is the step
• 3. While: while (condition)… endwhile(condition)
• 4. Foreach and while can use break and continue, using ${} in the loop
• 5. Option and if can be used together. option(
  “description” [initial_var])
  

(7) macros and methods

• ${a1} = ${a1} = ${a1} = ${a1}
  • 1.ARGV#, # is a subscript and can be used to reference variables
  • 2.ARGV, which represents all passed variables
  • 3.ARGN, passed in more than the required parameters
  • 4.ARGC, the total number of arguments passed
  • 5. Macro is a character substitution, similar to c preprocessing, so when used in if, you need ${} to get the parameter
• 2. Function ([a1 [a2 [a3…])… endMacro () is similar to macro, but is not a character substitution.

Four, FFmpeg official demo explanation

The last project:FFmpeg-learingIn the future, the FFmpeg demo will be added to this project, we still need to look at the blog together with this project.

1. Project structure

• 1. Let’s first understand the structure of this project, as shown in Figure 1:
  • 1. Under the Java directory in Figure 1, I think you should know that you put the Java files for developing Android
  • 2. Then we look at the jni/ffmpeg directory which has three folders:
    • Armeabi: This is the so file, which I compiled from the ffMPEG source code. Each so file corresponds to an FFmpeg module code that we discussed in Chapter 2.
    • 2. Include: this includes the exposed.h files of each FFmepg module. This means that we need to invoke the function definitions in the.h file to implement the functions in the so file.
    • 3. My: This is the code I wrote.

• 2. Take a look at the Cmake file in the project, because Android Studio currently supports Cmake files to manage Cpp code in Android.
  • 1. If you read chapter 3 carefully, then this should be easy to understand. There are two major new cmake commands that we didn’t cover before:
    • 1. Find_library: This command is used to find the local path of the library. In this case, I look for the local path of the log library and assign it to the log-lib parameter. ${log-lib} is the liblog.so file in the Android NDK directory, which is mainly used for Android log output. In addition, you can use this command to find the various SO files that you have locally.
    • 2. Set_target_properties: This command converts the paths of various SO files to simple values. Such as: set_target_properties( postproc-54 PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR} / SRC/main/jni/ffmpeg/armeabi/libpostproc – 54. So here is so file address into postproc – 54 this short name for later use.
    • 3. The rest of the code is simple: The main thing is to declare each so file as a library, and then use the target_link_libraries command to link my code to the library of each so file. This eventually allows you to package all of your Cpp code into your Android app.

2.FFmpeg reads the video file information

** Let’s look at the first official document Demo: read video information from a video file. 支那

• 1. Firstly, according to the multimedia concept described in the second chapter, we can know that video data has many encapsulation formats, such as MP4, AVI, FLV and so on. Structs used in FFmpeg to process these video formats (since FFmpeg is written in C, there is no concept of classes inside). Is AVFormatContext. If you go into this struct you can see that the definition is really similar to a Class in Java. There are member variables, and there are function Pointers (used in place of member functions).
• 2. With the struct that parses the video data encapsulation format, we also need something that can read data from the file. In FFmpeg, this thing is the AVIOContext, which is a member variable of AVFormatContext and is used to read data from the file and send it to AVFormatContext for parsing.
• 3. After popularizing the two structs, we can explain the demo. The entry is the av_io_reading method in the following code. The input parameter to this method is argc, which represents the number of argv arrays. Argv has two arguments representing the input file and the output file.Note: The following FFmpeg method I explain in the article, students who have downloaded the project can directly go to the definition of the method to see, I have translated the method documents INTO easy to understand Chinese.
  • 1. A bunch of variables are defined first
    • 1. For example, the two structs we mentioned earlier.
    • 2. Then we have two Pointers that define unit8_t, which is actually an unsigned char and you can go in and see what that means. Now, if you’re familiar with C, you know that an unsigned char pointer is basically just a chunk of memory like a Java byte array.
    • 3. Then two size_t are defined to represent the size of memory pointed to by the two unit8_t Pointers defined in 2.
    • 4. Two char Pointers are defined to represent the input and output files.
    • 5. Finally, we define a RET to represent the return value of this method, and a struct of type Buffer_data. This is our own definition, which encapsulates the unit8_t pointer and size_t, to make it easier.
  • Av_file_map = av_file_map = av_file_map Input_filename The data in this file is mapped to memory using mmap(), then the buffer pointer points to the memory, and then the size of the memory is given to the BUFFer_size pointer
  • Avformat_alloc_context initializes an AVFormatContext **
  • 4. Next to the av_malloc method, we use this method to make the avio_ctx_buffer pointer point to a 4KB memory area, which will be used to continuously read 4KB of data from the buffer. The keyword is memory alignment, Resources.
  • 5. Next we go to the avio_alloc_context method, which is used to initialize an AVIOContext. Here we pass in a couple of parameters and LET me explain:
    • First, avio_CTX_buffer and its corresponding size. As we said in 4, all subsequent reads from buffer are done using this memory block, and AVIOContext is the object that calls the read.
    • 2. It then passes in an address of type buffer_data and the address of the function read_packet. This is actually very similar to the callback we use in Java. **AVIOContext is not responsible for actually fetching data from buffer to buffer_data. He just needs to call read_packet when appropriate, where we implement the logic to populate buffer_data. ** If you have the code on hand, look at the definition and you’ll see that the next NULL argument is a function that writes buffer_data somewhere.
  • 6. Next comes the avformat_open_INPUT method, which simply takes the AVFormatContext we built earlier and tells it to stream the AVIOContext we defined earlier. So what we’re going to do here is we’re going to read the header of the file which we talked about in Chapter 2, and that’s where we’re going to read all the information about the video file.
  • The last two methods, avformat_find_stream_info and av_dump_format, are relatively simple. One is to parse the information of the stream in 6, and the other is to output the information of the video wrapper file to the file.
  • 8. The following work is to free the previously applied memory space. Unlike Java, C does not have a garbage collection mechanism.
  • 9. Here I think many students may be confused, which is normal. After all, they just call a method without knowing how to achieve the internal, and they will certainly feel very empty. And some of the data structures don’t know what to do, what the internal implementation is. But don’t worry, this is just the beginning of audio and video, things have to be done step by step. I’ll also take you through the source code of FFmpeg and then write some enterprise-class code that mimics the company code.

struct buffer_data {
    uint8_t *ptr;
    size_t size; ///< size left in the buffer
};
static int read_packet(void *opaque, uint8_t *buf, int buf_size)
{
    struct buffer_data *bd = (struct buffer_data *)opaque;
    buf_size = FFMIN(buf_size, bd->size);

    if(! buf_size)return AVERROR_EOF;
    printf("ptr:%p size:%zu\n", bd->ptr, bd->size);

    /* copy internal buffer data to buf */
    memcpy(buf, bd->ptr, buf_size);
    bd->ptr  += buf_size;
    bd->size -= buf_size;

    return buf_size;
}

int av_io_reading(int argc, char *argv[])
{
    syslog_init();
    AVFormatContext *fmt_ctx = NULL;
    AVIOContext *avio_ctx = NULL;
    uint8_t *buffer = NULL, *avio_ctx_buffer = NULL;
    size_t buffer_size, avio_ctx_buffer_size = 4096;
    char *input_filename = NULL;
    char *output_filename = NULL;
    int ret = 0;
    struct buffer_data bd = { 0 };

    if(argc ! = 2) { fprintf(stderr,"usage: %s input_file\n"
                "API example program to show how to read from a custom buffer "
                "accessed through AVIOContext.\n", argv[0]);
        return1; } input_filename = argv[0]; output_filename = argv[1]; // Read the file that input_filename points to and point to it with the buffer pointer. Ret = AV_file_map (input_filename, &buffer, &buffer_size, 0, NULL);if (ret < 0)
        goto end;

    bd.ptr  = buffer;
    bd.size = buffer_size;

    if(! (fmt_ctx = avformat_alloc_context())) { ret = AVERROR(ENOMEM); goto end; } // Request a buffer size of 4 bytes and use avio_ctx_buffer = (uint8_t *) av_malloc(avio_ctx_buffer_size);if(! avio_ctx_buffer) { ret = AVERROR(ENOMEM); goto end; } avio_ctx = avio_alloc_context(avio_ctx_buffer, avio_ctx_buffer_size, 0, &bd, &read_packet, NULL, NULL);if(! avio_ctx) { ret = AVERROR(ENOMEM); goto end; } fmt_ctx->pb = avio_ctx; ret = avformat_open_input(&fmt_ctx, NULL, NULL, NULL);if (ret < 0) {
        fprintf(stderr, "Could not open input\n");
        goto end;
    }

    ret = avformat_find_stream_info(fmt_ctx, NULL);
    if (ret < 0) {
        fprintf(stderr, "Could not find stream information\n"); goto end; } av_dump_format(fmt_ctx, 0, output_filename , 0); end: avformat_close_input(&fmt_ctx); /* note: the internal buffer could have changed, and be ! = avio_ctx_buffer */if (avio_ctx) {
        av_freep(&avio_ctx->buffer);
        av_freep(&avio_ctx);
    }
    av_file_unmap(buffer, buffer_size);

    char buf2[500] = {0};
    av_strerror(ret, buf2, 1024);
    if (ret < 0) {
        fprintf(stderr, "Error occurred: %s\n", av_err2str(ret));
        return 1;
    }

    return 0;
}
Copy the code

3. The statement

I was going to do two official demos, but I’m going to stop there. In fact, I have integrated the encoding video and decoding video demo in the project. Each method definition also has Chinese explanation, interested students can check by themselves. One more thing to say is that due to the limited time, many things in the project are not guaranteed to run successfully. I will declare this problem in the COMMIT if all tests pass.

Five, the tail

The beginning of the video finally finished, there is a “great man” said: the more you know, the more you do not know – when the evening. I’ve been feeling a lot of shortcomings lately, too, and each morning on my bike to work I reflect on what WENT wrong the day before. I every three provinces my body, this sentence no matter in what years are not when ah, mutual encouragement!!

No anxiety peddling, no headlines. Share something interesting about the world. Topics include but are not limited to: science fiction, science, technology, Internet, Programmer, computer programming. Below is my wechat public number: the world’s interesting things, do a lot of goods waiting for you to see.