From Blu-ray to 4K, Tencent video high bit rate download behind the technology

Welcome to cloud + community, get more Tencent mass technology practice dry goods oh ~

by
Tencent technology engineering official numberPublish in the cloud + community

Blu-ray and 4k video are becoming increasingly popular, with peak bit rates of 4k video exceeding 10Mbit/s. The TVideo platform of the architecture platform department is optimized from the aspects of resources, links, caching and access to effectively solve the secondary buffering problem of 4K high bit rate video, and the playback experience is comprehensively ahead of competitive products.

background

With the development of audio and video encoding and decoding, super resolution, VR/AR and other technologies, iPhone8, 4K TV, VR glasses and other terminal equipment update, high bit rate and high resolution video sources (4K video, 360° panoramic video, etc.) emerge one after another, and the viewing experience of multimedia users is gradually upgrading. The whole industry ecological chain is rapidly switching to high quality content.

In terms of resolution, the 4K video mentioned above refers to video sources with a resolution of 4096*2160, compared with 2K (2560*1440) and 1080P (1920*1080, the current resolution of Tencent Video Blu-ray). In the second half of 2017, Tencent Video launched “Dolby Privilege” to further improve user experience, introducing 4K video sources (see Figure 1).

The improvement of resolution is inevitably accompanied by the improvement of bit rate. Compared with the 3Mbit/s bit rate of blu-ray video, the peak value of dolby privileged 4K video exceeds 10Mbit/s, which poses great challenges to background download (Tvideo platform). To provide at least 10Mbit/s download speed, network delay, cache I/o, and source back rate need to be optimized and upgraded.

1. Introduction to Tvideo platform

Tencent Video Tvideo platform operates various important audio and video businesses in the company. Besides Tencent Video, it also provides support for space video and QQ Music. With 16T+ self-built bandwidth, 100 million levels of video library.

Tvideo adopts multi-layer cache and hot and cold classification architecture. The audio and video files are uploaded to the Tvideo background source station. The storage module of the source station adds characteristic information for each file to prevent malicious users from stealing, analyzes the file popularity, and pushes the file to the intermediate source according to different priorities. The intermediate source cache is isolated by service software to ensure that each service has independent storage space. Metro point load edge acceleration ensures fast data transmission to user terminals through scheduling and link acceleration.

2. Revelation of high-bit-rate video background optimization

Currently, high bit rate video is faced with a series of problems, such as network delay, uneven CACHE I/O, and lag caused by access/source back time. For example, dolby Vision 4K program, the peak bit rate is up to 10Mbit/s, and a movie is stored up to 10GB. How to guarantee background service quality and improve user viewing experience?

In view of the above problems, Tvideo platform has made a number of optimization in the link, cache, access, etc., to solve the cache load, link lag, high playback delay and other problems.

2.1 Link Acceleration

If no delay occurs during media playback on the client, it is called no buffer, or secondary buffering. The higher the ratio of no cache is, the better the background service is. Link acceleration focuses on solving the problem of audio and video playback delay.

There are many factors causing the lag, such as background network adapter speed down, transmission network fluctuation, cross-provincial and carrier access speed can not keep up, small pieces of requests caused BY IO fragments, etc. Tvideo uses the following measures to resolve the link delay:

1. Speed distribution: multithreading 10MB/s speed, asynchronous down disk
2. Data merge: HLS multi – fragment merge straight out, pseudo – stream processing
3. Back to source acceleration: Based on the client scenario, the back end automatically accelerates
4. Slow speed correction: support cross-network correction, slow link protocol stack acceleration
5. Source return by bit rate: The system automatically selects the best source return path based on the source return speed of the live network

For the scenario where files are not matched on edge nodes, there are two types: hotspot failure and common file failure. The link acceleration policy uses extreme speed distribution and source acceleration respectively to achieve 100% hit on edge nodes.

Speed distribution refers to the documents from the source station quickly distribute to the edge nodes, solve the source station, intermediate source such as busy link congestion situation, speed distribution by P2P network, hierarchical peak clipping, intelligent identification rate, in the second stage will be a section of video pushed to the entire network, first looks hot in tencent video play, hot sudden, member security scenarios, such as play an important role. Because the nodes of the distribution network communicate with each other, the TOP(K) high-bit-rate hotspot files of the entire network are distributed to edge nodes to achieve edge matching of hotspot videos and reduce link delay.

For common scenarios where no match is made, the loopback acceleration policy is adopted. Each loopback record the source speed. For videos with high bit rate, the loopback IP address with the source speed three times higher than the bit rate is used.

Dolby Video 4K adopts FMP4 sharding. Because the FMP4 slice file is too small, the link speed cannot reach the expectation due to the small sharding of each return source request. Tvido adopts the strategy of combining multiple fragments to return source download to reduce the link delay.

The figure above tests the return speed of merging multiple fragments of audio small files. It can be seen that merging multiple fragments has a significant effect on the return speed when TCP is not established.

Of course, background link acceleration also includes prioritizing client access. When the client performs an emergency download, the backend uses multi-connection acceleration. For slow requests, protocol stack acceleration is selected to minimize time consumption and ensure link download speed.

2.2 Cache Optimization

Cache is the core module of Tvideo, including cache by service management strategy, hot file-fragment definition algorithm, cold file elimination, global load balancing algorithm and so on.

The figure above shows the cluster architecture of the Tvideo platform’s metropolitan point machine room, in which the central index module maintains the index information of all files in the cluster, the HTTP download module is responsible for the access of business requests, and the data storage module is responsible for hot and cold data management.

Hot and cold data storage modules and memory cache are collectively referred to as caches. Hotspot classification is the core of the cache module. For common video hotspot statistics, it is relatively simple: The HTTP module is downloaded, and about 70% of hotspot requests are quickly processed through edge calculation in the access stage. Cold requests are calculated by the central index module, and the hotspot statistics are calculated by the preceding hierarchical calculation to achieve rapid hotspot statistics.

For high bit rate videos, in addition to the above conventional methods, the cache is classified according to room dimension, access model, IO and hotspot diffusion according to bit rate.

2.2.1 Cache Grading Policies Based on equipment Room Dimensions:

Each room there is a difference of quality Tvideo platform, so that can be divided into high quality resources and common resources, the resources through the network access to water quality analysis for different room now, the entire platform cache resource pool by bit rate, for the average room, cache more low bit rate video, high-quality room used to cache the high bit rate video. This section describes how to complete high-quality equipment room service requests with high bit rate through equipment room scheduling to avoid the situation that high-bit rate video requests hit low-quality equipment rooms.

High-quality resources in the process of file elimination, ensure that the number of files with high bit rate is less than the standard elimination value (for example, 5% for high bit rate files and 10% for ordinary videos), ensure that more space for storing high bit rate videos in the machine room; For a common equipment room, the device is used only when the resources are insufficient. More storage can be reserved for common video services. Service quality is guaranteed through the classification of the above physical media.

2.2.2 Cache classification based on client access model

According to the client access model, the client download process can be divided into rapid stage, gradual stage and P2P stage. Since each phase of the client requires different download speeds, the back-end cache needs to be able to differentiate between phases for special processing.

The user watches a movie and, according to the client’s playback logic, the client enters the fast download phase at the beginning of the playback. At this stage, the player buffer of the client has no data. Therefore, the player buffer must be filled quickly to reduce the lag caused by network jitter. When the client buffer is full, the client enters the progressive P2P stage. In this stage, the client stops downloading from the background and adopts P2P downloading. If the P2P speed is too slow and the cache content is too small, the client downloads from the background Tvideo again, which is called progressive downloading. For example, when the client first plays, the buffer will be filled up in 180 seconds for quick download, and the player starts to play and enters P2P download at the same time. Because P2P download speed cannot meet the requirements of high bit rate, when the client buffer is less than 90 seconds, the client will continue to download to the background Tvideo platform until the buffer is filled up.

The above three stages have different requirements for Tvideo background speed. According to edge calculation, the background accelerates the cache memory after detecting the request in the fast download stage and pre-pulls part of the data to the memory. In the progressive download process, the background predicts the VID of the file and preheats the file to the cache medium. In the P2P hole filling phase, cache load balancing is implemented to ensure that the cache load in the equipment room is balanced.

2.2.3 I/O Classification based on the file bit rate

At present, the Tvideo platform machine room cluster adopts the mixed model of SATA and SSD storage to solve the problem of high return source. For example, SATA uniformly uses 4T disks to ensure that edge nodes have enough storage to block hot spots and gradually spread to high-speed SSD according to the access heat. If more than 12 users in an equipment room access a 4K file (10Mbit/s) at the same time, a SATA disk cannot support download for all users. Therefore, you need to quickly spread the file to SSDS for backup.

Ordinary video and high bit rate video have different requirements for hotspot diffusion in the equipment room cluster, especially 4K video, which is stored separately. High bit rate video is sensitive to download speed, and is classified by bit rate and I/O frequency to SSD and memory diffusion, which solves the problem caused by insufficient disk speed.

2.3 Access Optimization

Access optimization mainly solves the lag caused by inter-provincial, inter-operator and insufficient resources, and solves the lag caused by insufficient resources through reasonable allocation of resources.

Client before every play, through Tvideo scheduling platform interface to get the download address, dispatching platform according to the client’s information such as IP, bit rate, choose a suitable room, through the load balancing algorithm, choose room one can download the address returned to the client, the client after the access to download address, access to the computer room. In simple terms, a user of Shanghai Telecom requests to play, first visits Tvideo scheduling platform, and then Tvideo returns a download address of Room 0001 of Shanghai Telecom to the client, and the client requests data from room 0001 of Shanghai Telecom.

In view of resource shortage and high bit rate video lag rate in peak hours, and access quality cannot be guaranteed, scheduling and grading policies are implemented: VIP and high bit rate videos are covered locally as much as possible, while services such as low bit rate, audio, and offline download are scheduled to secondary resources or regional resources to ensure playback quality. As shown in the above table, in the evening peak, due to insufficient resources in downtown Shanghai, VIP with high bit rate will be assigned to Local Shanghai, while VIP with low bit rate will be assigned to secondary coverage, while offline download will be assigned to regional coverage such as east China where resources are idle.

In view of the problem of multi-exit of small operators, the second scheduling is carried out by scheduling characteristic string to solve the delay caused by cross-network. For example, a Shanghai Mobile user dispatts to room 0001 of Shanghai Telecom through the Tvideo scheduling platform, and the client accesses the download server of room 0001 of Shanghai Telecom. When the download server finds that the CLIENT IP address is not that of China Telecom, the download server returns 302 to redirect the user to the mobile IP address for access.

conclusion

It is optimized from the aspects of resources, links, cache and access, and optimized from the aspects of bit rate classification, IO classification and service classification to effectively solve the secondary buffering problem of Blu-ray and 4K high bit rate videos. The figure below shows the comparison of self-built and outsourced no-buffering rate. Through the above optimization, the no-buffering rate of high bit rate is increased by 1.5%.

reading

This article has been authorized by the author yunjia community published, reproduced please indicate the source of the original