At present, the camera has two encoding formats, H.265 and H.264. Our camera already supports the transmission of H265 encoded video. The H.264 encoding format can be directly decoded by H5 in the Web browser to play the video. However, for the H.265 encoded video, it cannot be directly decoded by H5 to play the video. The current scheme is to call the codec component through H5 to broadcast the video on the Web.

But the pain point of current playback is: only soft solution and single thread can be used for video playback processing.

Let’s test h.265 live stream decoding playback. After testing, using Chrome browser to watch live broadcast for a long time on MacBook Pro 2.2ghz Intel Core I7/16g notebook, the memory usage is stable between 270M and 320M, and the CPU usage is between 40% and 50%. The CPU usage is too high to allow multiple videos to be played under the same client.

Because the client is limited by the technical bottleneck in playing H.265 encoded video, the server transcodes THE H.265 encoding format into H.264 encoding, and the server performs video processing in a unified manner, while the client only plays H.264 encoded video.

Implementation scheme

Plan 1:

Directly to encapsulate H265 into custom FLV, RTMP, HLS release to the front-end broadcast, web front-end to support analytical H265 playback controls, if using this scheme the basic it is hard to find a suitable open source solution, and the back-end and front-end changes it was basically is over, so this scheme is difficult to achieve in a short period of time.

Scheme 2:

Do a transcoding service to decode H265 and then encode into H264, the Web front-end playback scheme does not need to do any changes, transcoding words have software transcoding and hardware transcoding two schemes, software transcoding CPU resources, there is to consider the hardware transcoding scheme, this article is to introduce the hardware transcoding scheme.

The implementation process

Based on the existing video service framework, transcoding function is added in the existing live broadcast. The platform uniformly connects cameras and determines whether the video stream of the connected cameras is H.265 encoded video stream by pulling the stream. If the video stream is determined to be H.265 encoded, the transcoding service is enabled to encode and decode the existing video stream and convert the video to H.264 encoded and push it to the streaming media for video distribution. The platform also synchronously adds whether to enable the transcoding function. After enabling the transcoding function, the channel device will output H.264 encoded video.

At present, CPU is used to transcode video. CPU will be added later for decoding (specific graphics card associated with the device).