The text/red5pro
Original link: www.red5pro.com/blog/5-reas…
Real-time streaming/live streaming is a complex process involving many different methods and formats. One of these components is a codec for encoding and decoding media files. The codec also defines the types of tools that the codec can use to perform stream transfers.
To greatly simplify the process; In order for video to be transmitted over the Internet, it must first capture audio and video using a microphone and camera. The raw data must then be compressed (encoded) into a codec, broadcast over an Internet connection (using a transport protocol), sent to some sort of server-side solution (typically a CDN or cloud-based cluster like Red5 Pro), and then decompressed (decoded) for the subscriber to eventually watch the video.
Several different codecs are currently in use, including VP8/9, AVC (H.264), HEVC (H.265), and AV1. This article will focus specifically on why AVC (H.264) is a more efficient option for low-latency live streaming than building HEVC (H.265) streaming server solutions.
What is the difference between AVC and HEVC?
HEVC (Efficient Video Coding), also known as H.265, is the latest international standard for video compression. It is a successor to AVC (H.264), formerly known as MPEG-4. The HEVC standard was developed by ISO/IEC MPEG (Moving Image Expert Group) and ITU-T VCEG (Video Coding Expert Group) and was first published in 2013.
Logically, one might think that since HEVC is the most recently developed codec, it would be the most efficient. The reality, however, is more complicated. HEVC does offer an advantage in terms of compression efficiency, but HEVC files take up about half the storage space of ACV files. The smaller file size means that it requires less storage space and, more importantly, less bandwidth consumption for streaming. This advanced compression efficiency is due to how HEVC handles macroblocks. HEVC also features better variable block size segmentation, improved block segmentation and motion compensation filters, sample adaptive offset filtering, and better motion vector prediction and accuracy. This page from the X265 team provides a good explanation of these terms and how to improve HEVC efficiency.
With all these advantages, why isn’t HEVC a better real-time streaming codec?
1) Higher coding speed
A big reason for this is delay. Although HEVC ends up producing smaller videos and can send videos faster because there is less data to send, the presence of all this extra compression (compared to AVC) means that the actual compression will take longer to perform. As a result, it will take longer to send video. Sending may not require much bandwidth, but it takes more time to actually compress, and the time spent will eventually outweigh the time saved in streaming video.
While this does seem obvious to us in the streaming industry, it’s worth explaining why it’s so important. Latency is important because any kind of delay between broadcasters and subscribers can make the interactive live streaming experience very bad. Real-time streaming applications of any kind require low wait times to have natural streams. Sports events and event broadcasts need to avoid revealing the plot or triggering crowd reactions in real time, live auctions need to ensure that bids are registered at the right time and social platforms need to ensure a smooth dialogue before and after.
Because HEVC has more overhead, it can cause latency compared to encoding H.264.
2) Limit adoption
In order to use a codec, a supported hardware or software encoder is required. The adoption rate of HEVC is low, due in large part to patent licensing. Although AVC has only one, HEVC has four related patent pools: HEVC Advance, MPEG LA, Velos Media and Technicolor. This makes them more expensive and thus prevents their wider adoption. Therefore, you need to restrict it to specific hardware encoders and mobile chipsets. Additionally, in 2013, Cisco made its H.264 open source and released it as a free binary download. This has greatly promoted the wide application of AVC.
Although HEVC software encoders are available, using a software encoder can cause performance problems because it consumes more CPU. Real-time streaming is already a cpu-heavy process, so increasing already high usage is going to be a problem. Thus, as the wait time potentially increases, battery life decreases.
3) Browsers do not support it
In contrast to AVC, most Internet browsers do not support HEVC. Only Edge, Internet Explorer, and Safari support HEVC. Even then, devices running browsers will need to support HEVC hardware encoding. Even if HEVC is supported in the browser with the correct implementation, WebRTC still doesn’t work.
With mobile devices and portable computers widely used for Internet browsing, the inability to watch video in an Internet browser severely limits how users can access content. Supporting a variety of devices is the only way to ensure that a variety of users can watch the content they like.
4) Bandwidth reduction is more important for high resolution
Part of the advantage of higher compression rates and smaller file sizes is that it takes up less bandwidth, meaning users can still stream high-quality video at lower Internet speeds. But the reality is that the global average upload speed for fixed broadband connections is 42.63 Mbps, and the Internet is fast enough in most places to support 4K streaming over AVC. The average speed on mobile devices was much lower, at 10.93 Mbps. Even so, it’s not fast enough for 1080p.
This chart from Boxcast shows that the average connection speed across the globe definitely meets upload speed requirements at all resolution levels.
For users in countries with slow Internet speeds, adaptive bitrate support can adjust the bitrate to provide the best experience, while transcoding can divide the broadcast into multiple qualities based on available bandwidth for the best experience. Palm-sized devices don’t even need 1080p resolution to look good when streaming to mobile devices with 2 or 3G connections. 720P and even 480P will still display in high quality.
However, it must be acknowledged that smaller files will save companies bandwidth costs. Smaller files mean you don’t have to pay a hefty fee for more data streaming on a CDN or cloud network. Of course, while that’s nice, halving the data consumption only makes a big difference at really high resolution (4K, for example).
Of course, saving money is important regardless of size. This brings us to the next best of both worlds: better compression with the same performance.
5) LCEVC sidesteps the whole debate
LCEVC (Low Complexity Enhanced Video Encoding) adds an additional processing layer to improve compression rates by 40% using MPEG or other current and future versions of compression such as VP9 and AV1. As we introduced in our last article, LCEVC is a promising technology because it essentially turns current protocols already adopted in one form or another into more efficient protocols themselves.
It looks as if content providers will be able to use lcEVC-enabled software or hardware-based encoders and Red5 Pro across cloud platforms to facilitate real-time streaming video formats through a new generation of process-intensive devices. Depending on the core codec used, this will work not only for 4K and eventually 8K UHD, but also for formats designed for 360-degree viewing, virtual reality and other innovations.
That’s because virtually any device can support a thin LCEVC client embedded in a service provider’s application player or downloaded separately to a viewer device. In addition, LCEVC supports free browser plug-in functionality through the accompanying HTML5 JavaScript implementation. This means that broad implementation should be fairly simple.
Nevertheless, AV1 is expected to replace H.264 and H.265. Because the consortium behind it has all the major player credentials and is royalty-free. All that makes AV1 back now is the lack of a real-time encoder. Once widely available, AV1 (especially when paired with LCEVC) is the way forward.
Bottom line: H.264 / AVC works better
Although HEVC did make improvements in AVC, these improvements were not sufficient to overcome the shortcomings. HEVC did not meet expectations due to higher latency due to longer coding times, limited adoption due to patent licensing, lack of browser support, and the true viewing resolution of most live streams. In general, AVC streaming servers are much more efficient than HEVC streaming servers. More importantly, LCEVC is prepared to sidestep the entire codec issue, thereby invalidating all of it.