I've found that there are a lot of misconceptions about video formats, and I wrote a previous post on the supported formats of AVPlayer, so HERE's a separate post on that, so hopefully you'll understand.
The basic concept
For an online video to play, it goes through the following steps:
Can be summarized as: pull data - protocol - decapsulation - audio and video decoding - audio and video synchronization - play. Here is an explanation of each of these concepts:
Generally, HTTP is used for vod, while RTMP or proprietary protocol is mostly used for live broadcast, because the delay is relatively small and RTMP itself is designed for live broadcast. Common streaming media protocols:
|shorthand||The full name||Launch institutions||Current Usage|
|HLS||HTTP Live Streaming||apple||Mostly used in Apple|
|RTP||Real-time transmission protocol||IETF||Has a wide range of|
|RTCP||Real-time transmission control protocol||IETF||Has a wide range of|
|RTSP||Real-time streaming protocol||RealNetworks etc.||Has a wide range of|
|RTMP||Real-time messaging protocol||Adobe||More popular|
|MMS||Streaming media protocol||Microsoft||Has a wide range of|
The difference between RTP, RTSP/RTCP
RTSP initiates/terminates streaming media, RTP transmits streaming media data, and RTCP controls and synchronizes RTP.
HLS(Http Live Streaming) HLS is an Http based Streaming protocol introduced by Apple. Advantages: 1. The M3U8 index file can be used to select the play source intelligentarily for the current browsing device; 2. The M3U8 index file can be used to add backup index files to prevent the server from crashing and video playing failure; 3. 2. Video processing is required. 3. The request times for index files (TS video files) are relatively large
The so-called video encoding means that through a specific compression technology, a video format file into another video format file way. It is a compression technology to reduce the video volume and ensure the picture quality at the same time. Common video coding includes:
|The name of the||Launch institutions||Launch time||Current Usage|
|HEVC(H.265)||MPEG/ITU-T||2013||In the research and development|
|VP9||2013||In the research and development|
H.264 is also called AVC, is the international Organization for Standardization (ISO) and the International Telecommunication Union (ITU) jointly proposed after the MPEG4 of a new generation of digital video compression format, it combines the advantages of H.263 and MPEG4, has a higher data compression ratio. Under the same image quality conditions, the data compression ratio of H.264 is 2 times higher than THAT of H.263 and 1.5 times higher than that of MPEG-4. It is the best coding format supported by Apple.
Similar to video coding, audio coding is used to reduce the redundancy in the audio and to compress the audio while ensuring the quality of the audio.
|The name of the||Launch institutions||Launch time||Current Usage|
|AC-3||Dolby Inc.||1992||DVD, digital TV|
|MP3||MPEG||1993||Various fields (old)|
AAC AAC (Advanced Audio Coding technology), appeared in 1997, is based on MPEG-2 Audio Coding technology. Developed by Fraunhofer IIS, Dolby, Apple, ATT, SONY and others to replace the MP3 format. In 2000, THE MPEG-4 standard was introduced, and AAC reintegrated its characteristics, so it is now called MPEG-4 AAC, namely M4A. Apple's CoreAudio has good support for AAC.
Our common video formats MP4, AVI, RMVB, 3GP, MKV all refer to the encapsulation format of video, most cases are the suffixes of video files. Common combinations are:
|The name of the||Introduction to the||Common encoding format||extension|
|Flash Video||An extension of Adobe Flash, a popular web video packaging format. With the rich video sites, this format has become very popular.||H.264+MP3||flv|
|AVI (Audio Video Interleave)||The earlier AVI was developed by Microsoft. It means Audio Video Interactive, which is a mix of Video and Audio encoded and stored. AVI is also the oldest format, having been around for over 10 years, and although there have been revisions (V2.0 was released in 1996), it is showing its age. AVI format is more limited, can only have one video track and one audio track (now there are non-standard plug-ins to add up to two audio tracks), and can have some additional tracks such as text. The AVI format does not provide any control functions.||Xvid+MP3||avi|
|WMV (Windows Media Video)||Also a group of digital video codec Format developed by Microsoft, ASF (Advanced Systems Format) is its packaging Format. WMV files encapsulated in ASF have digital copyright protection.||VC-1+WMA||wmv/asf|
|MPEG (Moving Picture Experts Group)||Is an INTERNATIONAL Organization for Standardization (ISO) approved form of media packaging supported by most machines. It can be stored in various ways and adapt to different application environments. The mPEG-4 file container format is specified in Parts 1 (MUX), 14 (ASP), 15 (AVC), etc. MPEG controls are rich, with multiple videos (i.e., angles), audio tracks, subtitles (bitmapped subtitles), and more. A simplified version of MPEG, 3GP, is also widely used on pre-3G phones.||H.264+AAC,H263+AAC||dat(VCD),vob(DVD), mp4, 3gp|
|Matroska||Matroska Media is a new multimedia package format that encapsulates multiple video encoding formats and 16 or more audio and language captions into one Matroska Media file. It is also one of the open source multimedia encapsulation formats. Matroska also offers great interaction and is more convenient and powerful than MPEG.||A combination of various encoding formats||mkv|
|Real Video Real Media (RM)||Is a file container developed by RealNetworks. It usually only holds Real Video and Real Audio encoded media. The file has some interactive features that allow scripting to control playback. RM, especially variable bit rate RMVB format, has no complex Profile/Level, is simpler to make than H.264 video format, and is very popular among network uploanders. In addition, many people still have the misconception that RMVB is small in size and high in quality. This incorrect perception also leads many people to use RMVB. In fact, at the same bit rate, RMVB coding is larger than h.264, which is a highly compressed video coding.||RealVideo+RealAudio||rm/rmvb|
|QuickTime File Format||It's a container developed by Apple. On February 11, 1998, the International Organization for Standardization (ISO) recognized the QuickTime file format as the basis for the MPEG-4 standard. In addition to video and audio, QuickTime can store images, text (text captions), and more.||H.264+AAC||mov, qt|
MP4 The MP4 format is the standard package format specified for use in H.264 encoding. 3GP is a simplified version of the MP4 format, reducing the storage space and lower bandwidth requirements, making the limited storage space available on the mobile phone.
In fact, the audio and video encoding formats corresponding to these package formats are not fixed, take MP4 for example, the common MP4 is h.264 +AAC package, but also Xvid+AAC encoding may be possible. If the decoder does not support Xvid, you may not be able to play, or you may be playing with sound and no pictures.
Which audio and video coding formats apple supports
We can find the supported video format iPhone-Compare Models-Apple in the phone introduction interface
Here we can find the corresponding video encoding format supported by iPhone7:
Video formats supported: HEVC, H.264, MPEG-4 Part 2, and Motion JPEG
HEVC is also called h.265, iOS11+A9 chip just started to support HEVC, iPhone6s and previous devices do not support HEVC decoding. Audio format with red box marked out, more content, can be compared.
Video formats supported by AVPlayer
Just because Apple devices support audio and video formats doesn't mean AVPlayer supports many. To determine which formats AVPlayer supports, we can look at an API in AVKit:
// Display the currently supported audio and video formats let asset = AVURLAsset.audiovisualTypes() // Print the asset to get it. asset type ( "audio/aacp"."video/3gpp2"."audio/mpeg3"."audio/mp3"."audio/x-caf"."audio/mpeg"."video/quicktime"."audio/x-mpeg3"."video/mp4"."audio/wav"."video/avi"."audio/scpls"."audio/mp4"."audio/x-mpg"."video/x-m4v"."audio/x-wav"."audio/x-aiff"."application/vnd.apple.mpegurl"."video/3gpp"."text/vtt"."audio/x-mpeg"."audio/wave"."audio/x-m4r"."audio/x-mp3"."audio/AMR"."audio/aiff"."audio/3gpp2"."audio/aac"."audio/mpg"."audio/mpegurl"."audio/x-m4b"."application/mp4"."audio/x-m4p"."audio/x-scpls"."audio/x-mpegurl"."audio/x-aac"."audio/3gpp"."audio/basic"."audio/x-m4a"."application/x-mpegurl" ) Copy the code
There is another way to determine if the current format is playable:
//An extended MIME type string such as video/3gpp2; Codecs = "mp4v. 20.9 mp4a. E1" or audio/aac; codecs="mp4a.E1". let playable: Bool = AVURLAsset.isPlayableExtendedMIMEType("video/3gpp2; Codecs = \ "mp4v. 20.9, mp4a E1 \" ") Copy the code
The MIMEType here can be found here system-declared Uniform Type Identifiers
AVPlayer supports video encoding formats: h.264, HEVC(iPhone7 and later), mpeg-4. Video format (package format) :.MP4,.mov,.m4V,.3GP,.avi, etc. If you want to support more video formats, you can use third-party frameworks. Common video encoding and decoding frameworks include VLC and FFMPEG.
Video File Formats - Wikipedia, Free Encyclopedia Audio encoding format comparison - Wikipedia, Free Encyclopedia