preface

In Android audio and video development, online knowledge is too fragmented, self-learning is very difficult, but audio and video cattle Jhuster put forward “Android audio and video from the entry to improve – task list”. This article is the fourth Android audio and video task list, corresponding to the content to learn is: using the Android platform MediaExtractor and MediaMuxer API parsing and packaging MP4 files

Audio and video task list

Audio and video task list: Click here to jump to view

A, directories,

(a) MediaExtractor introduction

MediaExtractor is the role of audio and video data separation, that is, through MediaExtractor audio or video to separate out

Main API introduction:

SetDataSource (String Path) : you can set the local and network files
GetTrackCount () : Gets the number of channels in the source file
GetTrackFormat (int index) : Gets the channel format specified (index)
GetSampleTime () : Returns the current timestamp
ReadSampleData (ByteBuffer byteBuf, int offset) : Reads data from a specified channel into ByteBuffer at an offset;
Advance () : Reads the next frame
Release (): Releases resources after reading

MediaExtractor puts audio or video to separate extraction out of the steps: (1) create an instance

MediaExtractor mediaExtractor = new MediaExtractor();
Copy the code

(2) Set the data source

mediaExtractor.setDataSource(path);
Copy the code

(3) Obtain the track number of the data source and switch to the desired track

// Track index
int videoIndex = -1;
// Video track format information
MediaFormat mediaFormat = null;
// The number of tracks in the data source
int trackCount = mediaExtractor.getTrackCount();
for (int i = 0; i < trackCount; i++) {
    MediaFormat format = mediaExtractor.getTrackFormat(i);
    String mimeType = format.getString(MediaFormat.KEY_MIME);
    if (mimeType.startsWith("video/")) {
        videoIndex = i;
        mediaFormat = format;
        break; }}// Switch to the desired track
mediaExtractor.selectTrack(videoIndex);
Copy the code

(4) The required track data is read in a cycle and each frame is read for processing

while (true) {
    // Store the sample data in the byte cache
    int readSampleSize = mediaExtractor.readSampleData(byteBuffer, 0);
    // If no sample is available, exit the loop
    if (readSampleSize < 0) {
        mediaExtractor.unselectTrack(videoIndex);
        break; }... .// Read the next frame
    mediaExtractor.advance();
}
Copy the code

(5) Release resources after completion

mediaExtractor.release();
Copy the code

(2) Introduction to MediaMuxer

MediaMuxer is used to generate audio or video files; You can also mix audio and video into a single audio and video file

Related API introduction:

MediaMuxer(String path, int format) : path indicates the name of the output file, format indicates the format of the output file. Currently, only MP4 is supported.
AddTrack (MediaFormat format) : Adds channels. Us more is to use MediaCodec. GetOutpurForma () or Extractor. GetTrackFormat (int index) to obtain MediaFormat; You can also create your own;
Start () : Starts synthesizing files
WriteSampleData (int trackIndex, ByteBuffer byteBuf, mediacodec.bufferInfo BufferInfo) : Writes data from ByteBuffer to a file set in the constructor;
Stop () : Stops the synthesis file
Release () : Releases resources

MediaMuxer generates audio, video, and mixes audio and video into an audio and video file: (1) Create an instance

MediaMuxermediaMuxer = new MediaMuxer(path, format);
Copy the code

Path: the name of the output file. Format: Format of output files. Currently, only MP4 is supported.

(2) Add the audio track or video track to MediaMuxer and return to the new track

int trackIndex = mediaMuxer.addTrack(videoFormat);
Copy the code

(3) Start synthesis

mediaMuxer.start();
Copy the code

(4) Loop the audio track or video track data to the file

while (true) {
     // Store the sample data in the byte cache
     int readSampleSize = mediaExtractor.readSampleData(byteBuffer, 0);
     // If no sample is available, exit the loop
     if (readSampleSize < 0) {
         mediaExtractor.unselectTrack(videoIndex);
         break;
     }

     bufferInfo.size = readSampleSize;
     bufferInfo.flags = mediaExtractor.getSampleFlags();
     bufferInfo.offset = 0;
     bufferInfo.presentationTimeUs = mediaExtractor.getSampleTime();

     mediaMuxer.writeSampleData(trackIndex, byteBuffer, bufferInfo);

     // Read the next frame
     mediaExtractor.advance();
}
Copy the code

(5) Release resources after completion

mediaMuxer.stop();
mediaMuxer.release();
Copy the code

(three) through MediaExtractor to extract the video in mp4 files separately, through MediaMuxer synthesis of new video files

Example:

/** * split video */
private void extractorVideo(a) {
    // Create MediaExtractor instance
    MediaExtractor mediaExtractor = new MediaExtractor();

    // Initialize MediaMuxer
    MediaMuxer mediaMuxer = null;

    // Track index
    int videoIndex = -1;

    try {
        // Set the data source
        mediaExtractor.setDataSource(SDCARD_PATH + "/input.mp4");
        // The number of tracks in the data source
        int trackCount = mediaExtractor.getTrackCount();

        for (int i = 0; i < trackCount; i++) {
            // Video track format information
            MediaFormat trackFormat = mediaExtractor.getTrackFormat(i);
            String mimeType = trackFormat.getString(MediaFormat.KEY_MIME);
            if (mimeType.startsWith("video/")) {
                // This track is a video trackvideoIndex = i; }}// Switch to the desired track
        mediaExtractor.selectTrack(videoIndex);

        // Video track format information
        MediaFormat trackFormat = mediaExtractor.getTrackFormat(videoIndex);

        New MediaMuxer(String Path, int format) specifies the output path and format of the video file
        mediaMuxer = new MediaMuxer(SDCARD_PATH + "/output_video.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);

        // Add a media channel
        int trackIndex = mediaMuxer.addTrack(trackFormat);

        ByteBuffer byteBuffer = ByteBuffer.allocate(1024 * 500);
        MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();

        // After adding all tracks, call start method to start audio and video composition
        mediaMuxer.start();

        // Get the interval between frames
        long videoSampleTime;
        {
            // Store the sample data in the byte cache
            mediaExtractor.readSampleData(byteBuffer, 0);
            // skip first I frame
            if (mediaExtractor.getSampleFlags() == MediaExtractor.SAMPLE_FLAG_SYNC)
                // Read the next frame
                mediaExtractor.advance();

            mediaExtractor.readSampleData(byteBuffer, 0);
            long firstVideoPTS = mediaExtractor.getSampleTime();

            mediaExtractor.advance();
            mediaExtractor.readSampleData(byteBuffer, 0);
            long SecondVideoPTS = mediaExtractor.getSampleTime();


            videoSampleTime = Math.abs(SecondVideoPTS - firstVideoPTS);
            Log.d("fuck"."videoSampleTime is " + videoSampleTime);
        }

        mediaExtractor.unselectTrack(videoIndex);
        mediaExtractor.selectTrack(videoIndex);

        while (true) {
            // Store the sample data in the byte cache
            int readSampleSize = mediaExtractor.readSampleData(byteBuffer, 0);
            
            // Store the sample data in the byte cache
            if (readSampleSize < 0) {
                break;
            }

            // Read the next frame
            mediaExtractor.advance();

            bufferInfo.size = readSampleSize;
            bufferInfo.offset = 0;
            bufferInfo.flags = mediaExtractor.getSampleFlags();
            bufferInfo.presentationTimeUs += videoSampleTime;

            / / call MediaMuxer. WriteSampleData () to write data the mp4 files
            mediaMuxer.writeSampleData(trackIndex, byteBuffer, bufferInfo);
        }

        mediaMuxer.stop();
        mediaExtractor.release();
        mediaMuxer.release();

        Toast.makeText(this."Split video completed", Toast.LENGTH_LONG).show();
        Log.i("info"."Separation of video finish++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +");

    } catch (IOException e) {
        e.printStackTrace();
        Toast.makeText(this."Failed to separate video", Toast.LENGTH_LONG).show();
        Log.i("info"."Failed to separate video ++++++++++++++++++++++++++++++++++++++"+ e.toString()); }}Copy the code

(four) through MediaExtractor to extract the audio in MP4 files separately, through MediaMuxer synthesis of new audio files

Example:

/** * separate audio */
private void extractorAudio(a) {
    // Create MediaExtractor instance
    MediaExtractor mediaExtractor = new MediaExtractor();

    // Initialize MediaMuxer
    MediaMuxer mediaMuxer = null;

    // Track index
    int audioIndex = -1;

    try {
        mediaExtractor.setDataSource(SDCARD_PATH + "/input.mp4");
        int trackCount = mediaExtractor.getTrackCount();
        for (int i = 0; i < trackCount; i++) {
            MediaFormat trackFormat = mediaExtractor.getTrackFormat(i);
            if (trackFormat.getString(MediaFormat.KEY_MIME).startsWith("audio/")) {
                audioIndex = i;
            }
        }
        mediaExtractor.selectTrack(audioIndex);

        MediaFormat trackFormat = mediaExtractor.getTrackFormat(audioIndex);
        mediaMuxer = new MediaMuxer(SDCARD_PATH + "/output_audio.mp3", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
        int writeAudioIndex = mediaMuxer.addTrack(trackFormat);
        mediaMuxer.start();

        ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
        MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();

        long stampTime = 0;
        // Get the interval between frames
        {
            mediaExtractor.readSampleData(byteBuffer, 0);
            if (mediaExtractor.getSampleFlags() == MediaExtractor.SAMPLE_FLAG_SYNC) {
                mediaExtractor.advance();
            }

            mediaExtractor.readSampleData(byteBuffer, 0);
            long secondTime = mediaExtractor.getSampleTime();
            mediaExtractor.advance();

            mediaExtractor.readSampleData(byteBuffer, 0);
            long thirdTime = mediaExtractor.getSampleTime();
            stampTime = Math.abs(thirdTime - secondTime);
            Log.e("fuck", stampTime + "");
        }

        mediaExtractor.unselectTrack(audioIndex);
        mediaExtractor.selectTrack(audioIndex);

        while (true) {
            int readSampleSize = mediaExtractor.readSampleData(byteBuffer, 0);
            if (readSampleSize < 0) {
                break;
            }
            mediaExtractor.advance();

            bufferInfo.size = readSampleSize;
            bufferInfo.flags = mediaExtractor.getSampleFlags();
            bufferInfo.offset = 0;
            bufferInfo.presentationTimeUs += stampTime;

            mediaMuxer.writeSampleData(writeAudioIndex, byteBuffer, bufferInfo);
        }

        mediaMuxer.stop();
        mediaMuxer.release();
        mediaExtractor.release();

        Toast.makeText(this."Audio separation completed", Toast.LENGTH_LONG).show();
        Log.i("info"."Split audio complete ++++++++++++++++++++++++++++++++++++++");

    } catch (IOException e) {
        e.printStackTrace();
        Toast.makeText(this."Audio separation failed", Toast.LENGTH_LONG).show();
        Log.i("info"."Audio separation failed ++++++++++++++++++++++++++++++++++++++"+ e.toString()); }}Copy the code

(five) through MediaExtractor, respectively, the audio in mp3 files, MP4 files in the video extraction, through MediaMuxer synthesis of new video files

Example:

/** ** */
private void muxerVideoAudio(a) {
    try {
        // The following procedure is to find the video track in output_video.mp4
        MediaExtractor videoExtractor = new MediaExtractor();
        videoExtractor.setDataSource(SDCARD_PATH + "/output_video.mp4");
        MediaFormat videoFormat = null;
        int videoTrackIndex = -1;
        int videoTrackCount = videoExtractor.getTrackCount();
        for (int i = 0; i < videoTrackCount; i++) {
            videoFormat = videoExtractor.getTrackFormat(i);
            String mimeType = videoFormat.getString(MediaFormat.KEY_MIME);
            if (mimeType.startsWith("video/")) {
                videoTrackIndex = i;
                break; }}// The following procedure is to find the audio track in output_audio-mp3
        MediaExtractor audioExtractor = new MediaExtractor();
        audioExtractor.setDataSource(SDCARD_PATH + "/output_audio.mp3");
        MediaFormat audioFormat = null;
        int audioTrackIndex = -1;
        int audioTrackCount = audioExtractor.getTrackCount();
        for (int i = 0; i < audioTrackCount; i++) {
            audioFormat = audioExtractor.getTrackFormat(i);
            String mimeType = audioFormat.getString(MediaFormat.KEY_MIME);
            if (mimeType.startsWith("audio/")) {
                audioTrackIndex = i;
                break;
            }
        }

        videoExtractor.selectTrack(videoTrackIndex);
        audioExtractor.selectTrack(audioTrackIndex);

        MediaCodec.BufferInfo videoBufferInfo = new MediaCodec.BufferInfo();
        MediaCodec.BufferInfo audioBufferInfo = new MediaCodec.BufferInfo();


        // Use new MediaMuxer(String Path, int format) to specify the output path and file format of the video file
        MediaMuxer mediaMuxer = new MediaMuxer(SDCARD_PATH + "/output-composite.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
        // MediaMuxer add media channel
        int writeVideoTrackIndex = mediaMuxer.addTrack(videoFormat);
        // MediaMuxer add media channel (audio)
        int writeAudioTrackIndex = mediaMuxer.addTrack(audioFormat);
        // Start audio/video synthesis
        mediaMuxer.start();

        ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
        long sampleTime = 0;
        {
            videoExtractor.readSampleData(byteBuffer, 0);
            if (videoExtractor.getSampleFlags() == MediaExtractor.SAMPLE_FLAG_SYNC) {
                videoExtractor.advance();
            }

            videoExtractor.readSampleData(byteBuffer, 0);
            long secondTime = videoExtractor.getSampleTime();
            videoExtractor.advance();

            long thirdTime = videoExtractor.getSampleTime();
            sampleTime = Math.abs(thirdTime - secondTime);
        }

        videoExtractor.unselectTrack(videoTrackIndex);
        videoExtractor.selectTrack(videoTrackIndex);

        while (true) {
            int readVideoSampleSize = videoExtractor.readSampleData(byteBuffer, 0);
            if (readVideoSampleSize < 0) {
                break;
            }

            videoBufferInfo.size = readVideoSampleSize;
            videoBufferInfo.presentationTimeUs += sampleTime;
            videoBufferInfo.offset = 0;
            videoBufferInfo.flags = videoExtractor.getSampleFlags();

            mediaMuxer.writeSampleData(writeVideoTrackIndex, byteBuffer, videoBufferInfo);
            videoExtractor.advance();
        }

        while (true) {
            int readAudioSampleSize = audioExtractor.readSampleData(byteBuffer, 0);
            if (readAudioSampleSize < 0) {
                break;
            }

            audioBufferInfo.size = readAudioSampleSize;
            audioBufferInfo.presentationTimeUs += sampleTime;
            audioBufferInfo.offset = 0;
            audioBufferInfo.flags = videoExtractor.getSampleFlags();

            mediaMuxer.writeSampleData(writeAudioTrackIndex, byteBuffer, audioBufferInfo);
            audioExtractor.advance();
        }

        mediaMuxer.stop();
        mediaMuxer.release();
        videoExtractor.release();
        audioExtractor.release();

        Toast.makeText(this."Synthesizing audio and video completed.", Toast.LENGTH_LONG).show();
        Log.i("info"."Synthesizing audio and video complete ++++++++++++++++++++++++++++++++++++++");

    } catch (IOException e) {
        e.printStackTrace();
        Toast.makeText(this."Synthesizing audio and video failed.", Toast.LENGTH_LONG).show();
        Log.i("info"."Synthesizing audio and video failed ++++++++++++++++++++++++++++++++++++++"+ e.toString()); }}Copy the code

(vi) Complete code

(1) Layout

<? xml version="1.0" encoding="utf-8"? > <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    android:gravity="center">

    <Button
        android:id="@+id/btn_extractor_video"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Split video" />

    <Button
        android:id="@+id/btn_extractor_audio"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="40dp"
        android:text="Split audio" />

    <Button android:id="@+id/btn_muxer_videoaudio"
        android:layout_height="wrap_content"
        android:layout_width="wrap_content"
        android:layout_marginTop="40dp"
        android:text="Synthetic audio and video"/>

</LinearLayout>
Copy the code

(2) Code

package com.lzacking.mediaextractortest;

import androidx.appcompat.app.AppCompatActivity;
import androidx.core.app.ActivityCompat;
import androidx.core.content.ContextCompat;

import android.Manifest;
import android.app.Activity;
import android.content.pm.PackageManager;
import android.media.MediaCodec;
import android.media.MediaExtractor;
import android.media.MediaFormat;
import android.media.MediaMuxer;
import android.os.Bundle;
import android.os.Environment;
import android.util.Log;
import android.view.View;
import android.widget.Button;
import android.widget.Toast;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;

public class MainActivity extends AppCompatActivity {

    private static final String SDCARD_PATH = Environment.getExternalStorageDirectory().getPath();

    Button mExtractorVideo;
    Button mExtractorAudio;
    Button mMuxerVideoAudio;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        // Obtain permission
        verifyStoragePermissions(this);

        // Split the video
        mExtractorVideo = (Button) findViewById(R.id.btn_extractor_video);

        // Split audio
        mExtractorAudio = (Button) findViewById(R.id.btn_extractor_audio);

        // Mix audio and video
        mMuxerVideoAudio = (Button) findViewById(R.id.btn_muxer_videoaudio);

        mExtractorVideo.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) { extractorVideo(); }}); mExtractorAudio.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) { extractorAudio(); }}); mMuxerVideoAudio.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) { muxerVideoAudio(); }}); }/** * split video */
    private void extractorVideo(a) {
        // Create MediaExtractor instance
        MediaExtractor mediaExtractor = new MediaExtractor();
        // Initialize MediaMuxer
        MediaMuxer mediaMuxer = null;
        // Track index
        int videoIndex = -1;

        try {
            // Set the data source
            mediaExtractor.setDataSource(SDCARD_PATH + "/input.mp4");
            // The number of tracks in the data source
            int trackCount = mediaExtractor.getTrackCount();

            for (int i = 0; i < trackCount; i++) {
                // Video track format information
                MediaFormat trackFormat = mediaExtractor.getTrackFormat(i);
                String mimeType = trackFormat.getString(MediaFormat.KEY_MIME);
                if (mimeType.startsWith("video/")) {
                    // This track is a video trackvideoIndex = i; }}// Switch to the desired track
            mediaExtractor.selectTrack(videoIndex);

            // Video track format information
            MediaFormat trackFormat = mediaExtractor.getTrackFormat(videoIndex);

            New MediaMuxer(String Path, int format) specifies the output path and format of the video file
            mediaMuxer = new MediaMuxer(SDCARD_PATH + "/output_video.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
            // Add a media channel
            int trackIndex = mediaMuxer.addTrack(trackFormat);

            ByteBuffer byteBuffer = ByteBuffer.allocate(1024 * 500);
            MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();

            // After adding all tracks, call start method to start audio and video composition
            mediaMuxer.start();

            // Get the interval between frames
            long videoSampleTime;
            {
                // Store the sample data in the byte cache
                mediaExtractor.readSampleData(byteBuffer, 0);
                // skip first I frame
                if (mediaExtractor.getSampleFlags() == MediaExtractor.SAMPLE_FLAG_SYNC)
                    // Read the next frame
                    mediaExtractor.advance();

                mediaExtractor.readSampleData(byteBuffer, 0);
                long firstVideoPTS = mediaExtractor.getSampleTime();

                mediaExtractor.advance();
                mediaExtractor.readSampleData(byteBuffer, 0);
                long SecondVideoPTS = mediaExtractor.getSampleTime();

                videoSampleTime = Math.abs(SecondVideoPTS - firstVideoPTS);
                Log.d("fuck"."videoSampleTime is " + videoSampleTime);
            }

            mediaExtractor.unselectTrack(videoIndex);
            mediaExtractor.selectTrack(videoIndex);

            while (true) {
                // Store the sample data in the byte cache
                int readSampleSize = mediaExtractor.readSampleData(byteBuffer, 0);
                // Store the sample data in the byte cache
                if (readSampleSize < 0) {
                    break;
                }
                // Read the next frame
                mediaExtractor.advance();

                bufferInfo.size = readSampleSize;
                bufferInfo.offset = 0;
                bufferInfo.flags = mediaExtractor.getSampleFlags();
                bufferInfo.presentationTimeUs += videoSampleTime;

                / / call MediaMuxer. WriteSampleData () to write data the mp4 files
                mediaMuxer.writeSampleData(trackIndex, byteBuffer, bufferInfo);
            }

            mediaMuxer.stop();
            mediaExtractor.release();
            mediaMuxer.release();

            Toast.makeText(this."Split video completed", Toast.LENGTH_LONG).show();
            Log.i("info"."Separation of video finish++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +");
        } catch (IOException e) {
            e.printStackTrace();
            Toast.makeText(this."Failed to separate video", Toast.LENGTH_LONG).show();
            Log.i("info"."Failed to separate video ++++++++++++++++++++++++++++++++++++++"+ e.toString()); }}/**
     * 分离音频
     */
    private void extractorAudio(a) {
        // Create MediaExtractor instance
        MediaExtractor mediaExtractor = new MediaExtractor();
        // Initialize MediaMuxer
        MediaMuxer mediaMuxer = null;
        // Track index
        int audioIndex = -1;
        try {
            mediaExtractor.setDataSource(SDCARD_PATH + "/input.mp4");
            int trackCount = mediaExtractor.getTrackCount();
            for (int i = 0; i < trackCount; i++) {
                MediaFormat trackFormat = mediaExtractor.getTrackFormat(i);
                if (trackFormat.getString(MediaFormat.KEY_MIME).startsWith("audio/")) {
                    audioIndex = i;
                }
            }
            mediaExtractor.selectTrack(audioIndex);

            MediaFormat trackFormat = mediaExtractor.getTrackFormat(audioIndex);
            mediaMuxer = new MediaMuxer(SDCARD_PATH + "/output_audio.mp3", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
            int writeAudioIndex = mediaMuxer.addTrack(trackFormat);
            mediaMuxer.start();

            ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
            MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();

            long stampTime = 0;
            // Get the interval between frames
            {
                mediaExtractor.readSampleData(byteBuffer, 0);
                if (mediaExtractor.getSampleFlags() == MediaExtractor.SAMPLE_FLAG_SYNC) {
                    mediaExtractor.advance();
                }

                mediaExtractor.readSampleData(byteBuffer, 0);
                long secondTime = mediaExtractor.getSampleTime();
                mediaExtractor.advance();

                mediaExtractor.readSampleData(byteBuffer, 0);
                long thirdTime = mediaExtractor.getSampleTime();
                stampTime = Math.abs(thirdTime - secondTime);
                Log.e("fuck", stampTime + "");
            }

            mediaExtractor.unselectTrack(audioIndex);
            mediaExtractor.selectTrack(audioIndex);

            while (true) {
                int readSampleSize = mediaExtractor.readSampleData(byteBuffer, 0);
                if (readSampleSize < 0) {
                    break;
                }
                mediaExtractor.advance();

                bufferInfo.size = readSampleSize;
                bufferInfo.flags = mediaExtractor.getSampleFlags();
                bufferInfo.offset = 0;
                bufferInfo.presentationTimeUs += stampTime;

                mediaMuxer.writeSampleData(writeAudioIndex, byteBuffer, bufferInfo);
            }

            mediaMuxer.stop();
            mediaMuxer.release();
            mediaExtractor.release();

            Toast.makeText(this."Audio separation completed", Toast.LENGTH_LONG).show();
            Log.i("info"."Split audio complete ++++++++++++++++++++++++++++++++++++++");

        } catch (IOException e) {
            e.printStackTrace();
            Toast.makeText(this."Audio separation failed", Toast.LENGTH_LONG).show();
            Log.i("info"."Audio separation failed ++++++++++++++++++++++++++++++++++++++"+ e.toString()); }}/** ** */
    private void muxerVideoAudio(a) {
        try {
            // The following procedure is to find the video track in output_video.mp4
            MediaExtractor videoExtractor = new MediaExtractor();
            videoExtractor.setDataSource(SDCARD_PATH + "/output_video.mp4");
            MediaFormat videoFormat = null;
            int videoTrackIndex = -1;
            int videoTrackCount = videoExtractor.getTrackCount();
            for (int i = 0; i < videoTrackCount; i++) {
                videoFormat = videoExtractor.getTrackFormat(i);
                String mimeType = videoFormat.getString(MediaFormat.KEY_MIME);
                if (mimeType.startsWith("video/")) {
                    videoTrackIndex = i;
                    break; }}// The following procedure is to find the audio track in output_audio-mp3
            MediaExtractor audioExtractor = new MediaExtractor();
            audioExtractor.setDataSource(SDCARD_PATH + "/output_audio.mp3");
            MediaFormat audioFormat = null;
            int audioTrackIndex = -1;
            int audioTrackCount = audioExtractor.getTrackCount();
            for (int i = 0; i < audioTrackCount; i++) {
                audioFormat = audioExtractor.getTrackFormat(i);
                String mimeType = audioFormat.getString(MediaFormat.KEY_MIME);
                if (mimeType.startsWith("audio/")) {
                    audioTrackIndex = i;
                    break;
                }
            }

            videoExtractor.selectTrack(videoTrackIndex);
            audioExtractor.selectTrack(audioTrackIndex);

            MediaCodec.BufferInfo videoBufferInfo = new MediaCodec.BufferInfo();
            MediaCodec.BufferInfo audioBufferInfo = new MediaCodec.BufferInfo();

            // Use new MediaMuxer(String Path, int format) to specify the output path and file format of the video file
            MediaMuxer mediaMuxer = new MediaMuxer(SDCARD_PATH + "/output-composite.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
            // MediaMuxer add media channel
            int writeVideoTrackIndex = mediaMuxer.addTrack(videoFormat);
            // MediaMuxer add media channel (audio)
            int writeAudioTrackIndex = mediaMuxer.addTrack(audioFormat);
            // Start audio/video synthesis
            mediaMuxer.start();

            ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
            long sampleTime = 0;
            {
                videoExtractor.readSampleData(byteBuffer, 0);
                if (videoExtractor.getSampleFlags() == MediaExtractor.SAMPLE_FLAG_SYNC) {
                    videoExtractor.advance();
                }

                videoExtractor.readSampleData(byteBuffer, 0);
                long secondTime = videoExtractor.getSampleTime();
                videoExtractor.advance();

                long thirdTime = videoExtractor.getSampleTime();
                sampleTime = Math.abs(thirdTime - secondTime);
            }

            videoExtractor.unselectTrack(videoTrackIndex);
            videoExtractor.selectTrack(videoTrackIndex);

            while (true) {
                int readVideoSampleSize = videoExtractor.readSampleData(byteBuffer, 0);
                if (readVideoSampleSize < 0) {
                    break;
                }

                videoBufferInfo.size = readVideoSampleSize;
                videoBufferInfo.presentationTimeUs += sampleTime;
                videoBufferInfo.offset = 0;
                videoBufferInfo.flags = videoExtractor.getSampleFlags();

                mediaMuxer.writeSampleData(writeVideoTrackIndex, byteBuffer, videoBufferInfo);
                videoExtractor.advance();
            }

            while (true) {
                int readAudioSampleSize = audioExtractor.readSampleData(byteBuffer, 0);
                if (readAudioSampleSize < 0) {
                    break;
                }

                audioBufferInfo.size = readAudioSampleSize;
                audioBufferInfo.presentationTimeUs += sampleTime;
                audioBufferInfo.offset = 0;
                audioBufferInfo.flags = videoExtractor.getSampleFlags();

                mediaMuxer.writeSampleData(writeAudioTrackIndex, byteBuffer, audioBufferInfo);
                audioExtractor.advance();
            }

            mediaMuxer.stop();
            mediaMuxer.release();
            videoExtractor.release();
            audioExtractor.release();

            Toast.makeText(this."Synthesizing audio and video completed.", Toast.LENGTH_LONG).show();
            Log.i("info"."Synthesizing audio and video complete ++++++++++++++++++++++++++++++++++++++");
        } catch (IOException e) {
            e.printStackTrace();
            Toast.makeText(this."Synthesizing audio and video failed.", Toast.LENGTH_LONG).show();
            Log.i("info"."Synthesizing audio and video failed ++++++++++++++++++++++++++++++++++++++"+ e.toString()); }}private static final int REQUEST_EXTERNAL_STORAGE = 1;
    private static String[] PERMISSIONS_STORAGE = {
            "android.permission.READ_EXTERNAL_STORAGE"."android.permission.WRITE_EXTERNAL_STORAGE" };

    public static void verifyStoragePermissions(Activity activity) {
        try {
            // Check whether you have write permission
            int permission = ActivityCompat.checkSelfPermission(activity,
                    "android.permission.WRITE_EXTERNAL_STORAGE");
            if(permission ! = PackageManager.PERMISSION_GRANTED) {// There is no write permission. If you apply for the write permission, a dialog box will be displayedActivityCompat.requestPermissions(activity, PERMISSIONS_STORAGE, REQUEST_EXTERNAL_STORAGE); }}catch(Exception e) { e.printStackTrace(); }}}Copy the code

(3) Permission

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
Copy the code

(4) Documents

Put the input.mp4 file on your phone

(5) Results

Source code: Android audio and video development foundation (four) : using the Android platform MediaExtractor and MediaMuxer API parse and package MP4 files

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Android audio and video development foundation (four) : Using The Android platform of MediaExtractor and MediaMuxer API parsing and packaging MP4 files

preface

Audio and video task list

A, directories,

(a) MediaExtractor introduction

(2) Introduction to MediaMuxer

(three) through MediaExtractor to extract the video in mp4 files separately, through MediaMuxer synthesis of new video files

(four) through MediaExtractor to extract the audio in MP4 files separately, through MediaMuxer synthesis of new audio files

(five) through MediaExtractor, respectively, the audio in mp3 files, MP4 files in the video extraction, through MediaMuxer synthesis of new video files

(vi) Complete code

(1) Layout

(2) Code

(3) Permission

(4) Documents

(5) Results

Android audio and video development foundation (four) : Using The Android platform of MediaExtractor and MediaMuxer API parsing and packaging MP4 files

preface

Audio and video task list

A, directories,

(a) MediaExtractor introduction

(2) Introduction to MediaMuxer

(three) through MediaExtractor to extract the video in mp4 files separately, through MediaMuxer synthesis of new video files

(four) through MediaExtractor to extract the audio in MP4 files separately, through MediaMuxer synthesis of new audio files

(five) through MediaExtractor, respectively, the audio in mp3 files, MP4 files in the video extraction, through MediaMuxer synthesis of new video files

(vi) Complete code

(1) Layout

(2) Code

(3) Permission

(4) Documents

(5) Results

Related Posts

“BATcoder” has been working on Android for years and has not compiled the source code? A video to take you to play!

Kotlin Series 6: Lambda programming in Kotlin in terms of collections

Android: AAPT2 AAPT2-3.1-4818971-osx