In recent years, live broadcasting has become a hot topic in the Internet industry. Live answering questions, live games and live contests have emerged in an endless stream. Live broadcasting has long been a familiar technology for people. As a matter of fact, the rise of live broadcasting is not only related to the psychology that people require to speak for themselves in the new era, but also benefits from the acceleration of bandwidth and the development of CDN technology. With the maturity of CDN technology, it is increasingly easy for enterprises to deploy cloud servers to do live broadcasting.

As the second part of this series, this article mainly talks about live broadcast protocol, video streaming and other technical content


Broadcast protocol

Streaming media is divided into live broadcast and on-demand. Generally speaking, vod uses HTTP protocol, while live broadcast mainly uses RTMP, HLS, HTTP-FLV and so on. There have been developments in live streaming protocols in recent years, such as DASH, but they are still in their infancy. The difference between live and on-demand protocols is rooted in their business differences.


On demand, commonly used in youku, iQIYI video sites such as drama, movies and other media resources, namely on demand are recorded video, one thousand people watching the same video, no matter any time point in access to the media data are the same, while live otherwise, the information in view of the different time point is not the same.


Generally speaking, live streaming and on-demand are not integrated with each other, but in recent years, some people have innovated and developed the time-shifting mode of live streaming, which is the combination of on-demand and live streaming. The idea is to record live streams as bite-sized on-demand files that can then be accessed from anywhere, on any terminal. For example, if you are watching a football match live and there is a wonderful scene and you want to watch it again, you can drag the progress bar to go back and play it back. After watching the replay, you can also go back to live with one click.


At present, live broadcast distribution mainly has the following characteristics:

1. FLV is in the majority and TS is in the minority, mainly because TS standard is too complicated. The standard open documentation for Flv is 11 pages and for TS it is 174 pages. For general live broadcast, FLV can basically meet the needs, so there are few TS applications. Of course, we could use FFmpeg, but it encapsulates what you expect and what you don’t expect in streaming media, and it’s not precise enough.


2. RTMP and HLS coexist. Generally speaking, RTMP is used on PC and is played in Flash. HLS is used on mobile phones and tablets.


3. RTMP is generally used for real-time streams. RTMP can achieve 1 to 3 seconds of latency, which is the lowest latency protocol in live broadcast after RTSP. PC supports direct playback, mobile terminal can use FFmpeg decoding playback. Are there any protocols other than RTMP suitable for real-time streaming?

In fact, HTTP-FLV is better for real-time streaming than RTMP. Both have the same latency and can be played directly on PC, while mobile requires ffMPEG, but http-FLV has the added benefit of being able to pass through walls. However, most CDNS do not support HTTP-FLV live streaming, because ordinary Web servers do not support HTTP-FLV, which is a streaming problem.


Live broadcast server

The transmission of streaming media data in live broadcast mainly depends on servers. At present, open source streaming media servers include RED5, CRTMPD, Nginx-rtmp and SRS.


RED5: Open source streaming server for the oldest Flash-based streaming service. Written in the Java language, it uses RTMP as the streaming media transfer protocol and is fully compatible with FMS. With streaming FLV and MP3 files, real-time recording client streams into FLV files, shared objects, real-time video playback, Remoting and other functions. However, due to its relatively backward technology, the newly entered live streaming platform has been abandoned.


CRTMPD: a streaming media server written in c++ that supports multiple RTMP protocols, iptv-related network protocols, and mobile devices. The use of single-threaded asynchronous sockets was a leader at the time, but faded with the advent of NGINX.


Nginx-rtmp: a streaming media server written in C language based on the NGINX module. It is also the most used streaming media server in the market. With the expansion of CDN services in 2012, the demand for live streaming services skyrocketed, because the live streaming on demand in Nginx-rtmp shared a set of servers, and users were familiar with and trusted NGINX. Nginx-rtmp is becoming an industry monopoly.


SRS (Simple Rtmp Sever) is a domestic streaming media server. The product is positioned as an operation-level Internet live broadcast server cluster, pursuing better concept integrity and the simplest code implementation. According to the official website, its efficiency is very high, can reach 3 times of nginx-RTMP, and both Chinese and English documents have a copy, more suitable for domestic programmers development environment.

 

Live on flow


The general process of live streaming is shown below




As shown in the figure above, after collecting relevant data sources from cameras and microphones, some pre-packaging processing, such as denoising, beauty and voice changing, is usually needed in live broadcast, and then audio and video coding is carried out, and then encapsulated with appropriate streaming media protocol, and then the bit rate self-adaptation is carried out, and then it can be submitted to relevant sites for display.


but
There are different ways to do live streams in different technical languages:


If you are an iOS or Android programmer, it is much easier to make a RTMP stream. You can just go to the database of the stream and give the video parameters and the final RTMP address to make a standard RTMP stream


If you are a C++ programmer, it will be a lot of trouble, you should at least master the acquisition, coding, write stream these three steps. Of course, there are libraries to call these steps, but even so, assuming you’re using the FFmpeg library, you’ll need about 100 lines of code to complete the above actions; The main code flow includes opening audio and video devices, creating codecs, setting encoding parameters, initializing network flow handles, writing protocol headers, cyclic data collection, decoding data, encoding data, formatting and encapsulation, and writing network flow.


Of course, you can use the FFmpeg command line to push streams with a single command, but this is only for testing or simple demos, not for real engineering environments because the single-command approach does not support many functions.


Conclusion:


In short, the difficulty of making a live broadcast depends on the function you want to implement. If you just want to test it yourself, download an open source server code, run the code, push the stream with FFmpeg one line command, and then play it with the player. But when it comes to commercializing and satisfying users’ diverse needs, such as echo suppression, live streaming, and beauty filters, the problem becomes exponentially more complex.