preface

Recently do some work in the field of audio and video, this field is not around the basic FFMPEG, so want to do some research on its source code, standing on the shoulders of the giant, learn about its design ideas and implementation ideas, FFMPEG many people first contact should be its command and FFPlay, In this article we will examine the implementation of the FFMPEG command.

For those who are interested, keep an eye on Nilstorm and Gezilinll

Start with the problem

  • What is the command structure of FFMPEG
  • How to achieve arbitrary function, configuration of random assembly
  • How do we deal with the non-linear case of inverting
  • When do not need to transcode similar to just need to transencapsulate words how to deal with

The command structure

ffmpeg [global_options] {[input_file_options] -i input_url} ... {[output_file_options] output_url} ...

The process that

  1. There should be a data layer that parses the data in terms of input parameters
  2. The decoding, transcoding, functional processing and coding modules are abstractly separated
  3. The above four modules are assembled according to the data analysis structure of the first link to form the effect of one or more responsibility chains
  4. Start processing

There should be a lot of branching, parameter setting, etc., which we will not consider here. Let’s just look at the main link.

Main process

The first entry to the ffmpeg command is in the main function of fftools/ffmpeg.c. Let’s look at the whole process from this function. The code uses the version 4.4 of ffmpeg and omits some trivial things:

Int main(int argc, char **argv) { */ ret = ffmpeg_parse_options(argc, argv); */ ret = ffmpeg_parse_options(argc, argv); /* some data judgement, start benchmarking etc. */... /* if (transcode() < 0)... */... */... }

From the above code, you can see that the core part is two parts: data parsing + file conversion, and we will take a closer look at these two parts.

The following commands are used in our analysis:

ffmpeg -i douyin_700x1240.mp4 -vf reverse -af areverse reversed.mp4

Douyin_700x1240. mp4 takes the video stream from the douyin_700x1240.mp4 and inverts it into the audio stream, creating reversed. Mp4.

Data parsing

Before we look at the FFMPEG_PARSE_OPTIONS function, let’s look at some of the main constructs associated with parameter configuration in the FFMPEG command:

OptionParseContext

typedef struct OptionParseContext { OptionGroup global_opts; OptionGroupList *groups; // Output/input configuration parameter set int nb_groups; // ExceptionGroup; parsing state */ index.php cur_group; } optionParseconText; optionParseconText (); optionParseconText ();

This object is used to store parsed data, global parameters, input and output parameters, etc.

OptionGroupList

/**
 * A list of option groups that all have the same group type
 * (e.g. input files or output files)
 */
typedef struct OptionGroupList {
    const OptionGroupDef *group_def;

    OptionGroup *groups;
    int       nb_groups;
} OptionGroupList;

The above notes have been explained quite clearly.

OptionGroup

typedef struct OptionGroup {
    const OptionGroupDef *group_def;
    const char *arg;

    Option *opts;
    int  nb_opts;

    AVDictionary *codec_opts;
    AVDictionary *format_opts;
    AVDictionary *resample_opts;
    AVDictionary *sws_dict;
    AVDictionary *swr_opts;
} OptionGroup;

This is where the various parameters are stored, and then below that is where the individual configuration parameters are stored.

Option

/**
 * An option extracted from the commandline.
 * Cannot use AVDictionary because of options like -map which can be
 * used multiple times.
 */
typedef struct Option {
    const OptionDef  *opt;
    const char       *key;
    const char       *val;
} Option;

This is the configuration of key and value pairs and the definition specification.

OptionDef / OptionGroupDef

Here are the canonical definitions for various parameters, such as optionGroupDef:

typedef struct OptionGroupDef { /**< group name */ const char *name; /** * Option to be used as group separator. Can be NULL for groups which * are terminated by a non-option argument (e.g.  ffmpeg output files) */ const char *sep; /** * Option flags that must be set on each option that is * applied to this group */ int flags; } OptionGroupDef;

Additional configurations supported by FFmpeg are listed in FFmpeg_opt.c:

#define OFFSET(x) offsetof(OptionsContext, x)
const OptionDef options[] = {
    /* main options */
    CMDUTILS_COMMON_OPTIONS
    { "f",              HAS_ARG | OPT_STRING | OPT_OFFSET |
                        OPT_INPUT | OPT_OUTPUT,                      { .off       = OFFSET(format) },
        "force format", "fmt" },
    { "y",              OPT_BOOL,                                    {              &file_overwrite },
    .......

ffmpeg_parse_options

Now let’s look at the analytic function:

int ffmpeg_parse_options(int argc, char **argv) { ...... /* Split the arguments on the command line, */ ret = split_commandLine (&octx, argc, argv, options, groups, FF_ARRAY_ELEMS(groups)); . */ ret = parse_optgroup(NULL, &octx.global_opts); */ ret = parse_optgroup(NULL, &octx.global_opts); if (ret < 0) { av_log(NULL, AV_LOG_FATAL, "Error splitting the argument list: "); goto fail; } /* Open all input file streams in the configuration */ /* Open_Input_File if there is an initial processing time for the configuration file */ ret = Open_Files (& Octx.groups [GROUP_INFILE]),  "input", open_input_file); if (ret < 0) { av_log(NULL, AV_LOG_FATAL, "Error opening input files: "); goto fail; */ ret = init_complex_filters(); */ ret = init_complex_filters(); */ ret = init_complex_filters(); if (ret < 0) { av_log(NULL, AV_LOG_FATAL, "Error initializing complex filters.\n"); goto fail; } /* Open output file stream */ ret = open_files(&octx.groups[GROUP_OUTFILE], "output", open_output_file); if (ret < 0) { av_log(NULL, AV_LOG_FATAL, "Error opening output files: "); goto fail; }... }

In the above code, we focus on split_commandLine, and the other functions are just as useful as the comments. Most of them are logical, and different business scenarios may have different design ideas, which does not affect our current analysis: split_commandLine

int split_commandline(OptionParseContext *octx, int argc, char *argv[], const OptionDef *options, const OptionGroupDef *groups, int nb_groups) { ... /* init_parse_context(octx, groups, nb_groups); /* init_parse_context(octx, groups, nb_groups); av_log(NULL, AV_LOG_DEBUG, "Splitting the commandline.\n"); / / while (optIndex < argc) {const char *opt = argv[optIndex+ +], *arg; / / while (optIndex < argc) {const char *opt = argv[optIndex+ +], *arg; const OptionDef *po; int ret; av_log(NULL, AV_LOG_DEBUG, "Reading option '%s' ..." , opt); / * - XXX skip * / if (opt [0] = = '-' && opt [1] = = '-' &&! opt[2]) { dashdash = optindex; continue; } /* if (opt[0]! * if (opt[0]!);} /* if (opt[0]!); = '-' | |! Opt [1] | | dashdash + 1 = = optindex) {/ * here is to find the corresponding OptionGroup parameters about the assignment * / finish_group (octx, 0, opt); av_log(NULL, AV_LOG_DEBUG, " matched as %s.\n", groups[0].name); continue; } opt++; . /* Matches whether the current parameter is -i configuration, and takes the following parameter such as douyin_700x1240.mp4, */ if ((ret = match_group_separator(groups, NB_groups, opt)) >= 0) {GET_ARG(arg); finish_group(octx, ret, arg); av_log(NULL, AV_LOG_DEBUG, " matched as %s with argument '%s'.\n", groups[ret].name, arg); continue; } /* matches if the current parameter is one of the options defined above, as in our example -vf */ Po = find_option(options, opt); If (pop-> flags & OPT_EXIT) {/* Optional argument if (pop-> flags & OPT_EXIT) {/* Optional argument if (pop-> flags & OPT_EXIT) { e.g. -h */ arg = argv[optindex++]; } else if (po->flags & HAS_ARG) { GET_ARG(arg); } else { arg = "1"; } /* Set the parsed parameter configuration to the input or output Group data structure */ /* Add the parameters to the octx->cur_group or global_opts after the pre-saved flag is determined in options. The cur_group is used to store the data for the non-attribution criteria above, and as a final warning, because it has no effect on the problem mentioned in the opening paragraph, I will skip it for now. */ add_opt(octx, po, opt, arg); av_log(NULL, AV_LOG_DEBUG, " matched as option '%s' (%s) with " "argument '%s'.\n", po->name, po->help, arg); continue; } /* All data types are matched, parsed, and set. */ av_log(NULL, AV_LOG_ERROR, "Unrecognized option '%s'.\n", opt); return AVERROR_OPTION_NOT_FOUND; }... }

Analytical results

MP4 – VF Reverse-af Areverse Reverse reversed. MP4 will store the data for a single command reversed. MP4:

The flow chart

File conversion

After parsing the arguments, let’s see how the file conversion is handled. Here we will focus on the logic of the function transcode, and extend the other relevant parts with this function:

/* * The following code is the main loop of the file converter */ static int transcode(void) { int ret, i; AVFormatContext *os; OutputStream *ost; InputStream *ist; int64_t timer_start; int64_t total_packets_written = 0; */ ret = transcode_init(); */ ret = transcode_init(); */ ret = transcode_init(); . /* If a stop signal is not received, continue the conversion process until it completes or a stop signal is received */ while (! received_sigterm) { ...... /* ret = transcode_step(); /* ret = transcode_step(); . }... /* for (I = 0; /* for (I = 0; i < nb_input_streams; i++) { ist = input_streams[i]; if (! input_files[ist->file_index]->eof_reached) { process_input_packet(ist, NULL, 0); } } flush_encoders(); term_exit(); /* for (I = 0; i < nb_output_files; i++) { os = output_files[i]->ctx; if (! output_files[i]->header_written) { av_log(NULL, AV_LOG_ERROR, "Nothing was written into output file %d (%s), because " "at least one of its streams received no packets.\n", i, os->url); continue; } if ((ret = av_write_trailer(os)) < 0) { av_log(NULL, AV_LOG_ERROR, "Error writing trailer of %s: %s\n", os->url, av_err2str(ret)); if (exit_on_error) exit_program(1); }} /* Close codec, close file */...... return ret; }

As you can see from the above code, the core is in transcode_step:

/** * Run a single step of transcoding. * * @return 0 for success, <0 for error */ static int transcode_step(void) { OutputStream *ost; InputStream *ist = NULL; int ret; */ ost = choose_output(); */ ost = choose_output(); . if (ost->filter && ost->filter->graph->graph) { ...... /* Process_input output frame (process_input output frame); /* Process_input output frame (process_input output frame); / if ((ret = transcode_from_filter(OST-> filter->graph, ret = transcode_from_filter(OST-> filter->graph), &ist)) < 0) return ret; if (! ist) return 0; } else if (ost->filter) { int i; for (i = 0; i < ost->filter->graph->nb_inputs; i++) { InputFilter *ifilter = ost->filter->graph->inputs[i]; if (! ifilter->ist->got_output && ! input_files[ifilter->ist->file_index]->eof_reached) { ist = ifilter->ist; break; } } if (! ist) { ost->inputs_done = 1; return 0; } } else { av_assert0(ost->source_index >= 0); ist = input_streams[ost->source_index]; } /* Loading an AVPACKET includes some PTS checks, etc. Process_Input_Packet --> DECODE_AUDIO/DECODE_VIDEO Process_Input_Packet --> DECODE_AUDIO/DECODE_VIDEO Process_Input_Packet --> DECODE_AUDIO/DECODE_VIDEO Process_Input_Packet --> DECODE_AUDIO/DECODE_VIDEO */ * Decoded data is initialized and configured for FilterGraph using send_frame_to_filters. */ FilterGraph is initialized and configured for send_frame_to_filters. / ret = process_input(ist->file_index); / / the AVFilterContext */ ret = process_input(ist->file_index); if (ret == AVERROR(EAGAIN)) { if (input_files[ist->file_index]->eagain) ost->unavailable = 1; return 0; } if (ret < 0) return ret == AVERROR_EOF ? 0 : ret; /* Filter the filter to the filter, Do_video_out /do_audio_out /do_audio_out/write_packet */ return reap_filters(0); }

The overall flow of transcode_step is described in the code above, but the example in our example is an inverted function, It doesn’t use the reap_filters at the end of the \ code but the reap_filters in the transcode_from_filter, because we use the -VF reverse filter to process the output. The logic processed by the FFMPEG filter waits for the decoded data to be fully ready, and then encodes it in reverse order, which can cause considerable memory pressure, especially on mobile terminals.

Ffmpeg – I douyin_700x1240.mp4 audio.aac, then the filters are reap_filters at the bottom. The FFMPEG command in process_intput_packet will call do_streamcopy directly if it doesn’t need to be encoded again:

static int process_input_packet(InputStream *ist, const AVPacket *pkt, int no_eof) { ...... for (i = 0; i < nb_output_streams; i++) { OutputStream *ost = output_streams[i]; if (! ost->pkt && ! (ost->pkt = av_packet_alloc())) exit_program(1); if (! check_output_constraints(ist, ost) || ost->encoding_needed) continue; do_streamcopy(ist, ost, pkt); } return ! eof_reached; }

And then the last thing that I want to think about in this example backwards, from the point of view of the code logic is that there are actually two conditions that allow you to start coding, one is OST-> filter->graph->graph is not null, The AVFilter_Graph_Request_oldest inside Transcode_From_Filter returns >=0. If you don’t want to use the AVFilter_Graph_Request_oldest, you can use the simple form to return >. But at present, I haven’t gone to the in-depth study, and I will finish it together when I study the AVFILTER module. It is necessary to know this logic first here.

The flow chart

conclusion

So far we have done an analysis of the main process of FFmpeg command implementation, the overall implementation of the idea with our initial prediction or some close, but the assembly of FFmpeg is mainly through its AVFilter and related modules to deal with, The FilterGraph is supposed to be a more complex chain of responsibility, but there are a lot of details and problems that need to be solved in the actual code, which will be analyzed further in the rest of the FFMPEG source world later.