There have been 9 updates in the Live broadcast Troubleshooting series. We have analyzed the problems in the live broadcast process for users and made troubleshooting one by one. This time, we focus on the problem of high power consumption in the process of live broadcasting.

Following the series of articles “Detailed Explanation of Live Broadcasting Technology”, we have launched this new series “Troubleshooting of Live Broadcasting Complicated Diseases” to share the experience of solving live broadcasting problems step by step. At the same time, some basic knowledge and optimization experience of audio and video development will be intermixed, hoping to help developers in the field of live broadcasting.

Topics covered in this series include, but are not limited to, the following:

  • Playback failed
  • Live caton
  • The first is slow
  • High latency
  • Out of sync
  • Severe Mosaic
  • Play black screen, splintered screen and green screen
  • Play noise, noise, echo
  • Drag on demand is not correct
  • Direct heat problem
  • Other questions (to be continued)

Phenomenon of the problem

During the live broadcast, the mobile phone heats badly and consumes power quickly.

Troubleshoot problems

The fundamental factor leading to high power consumption and serious heating of mobile phones is nothing more than a high CPU/GPU occupancy rate. Therefore, we should first analyze which factors lead to high CPU/GPU occupancy rate.

1. Too much data

Live broadcast mainly consists of: video collection -> video processing (clipping, beautifying and filtering) -> coding -> streaming.

What are the factors that determine the amount of data in this process?

  • The size of the video (e.g., 1280 x 720 images are significantly harder to process than 320 x 240 images)
  • Frame rate of video (e.g. 30 frames per second is obviously more laborious than 15 frames per second)

Therefore, appropriately reducing the size and frame rate of the video without affecting the business experience can significantly reduce the load of CPU/GPU in subsequent links, thus significantly reducing the power consumption.

2. Extensive format conversion

Different modules have different requirements for data formats. For example, data from Android cameras is mostly NV21, while encoders generally require data in I420 format. For example, FFMPEG decoded video is often in YUV format, while rendering display often requires RGB format, and so on.

We should reduce data conversion between different formats as much as possible, or use GPU to deal with some complex format conversion as much as possible. For example, it is a good choice to use OpenGL to directly render YUV format data, instead of using CPU to do YUV -> RGB conversion.

3. Enlarge the image

As mentioned in the article “Live Screening of Difficult and Miscellaneous Diseases (6) — Serious Mosaic”, it is not recommended to put a small size picture -> enlarge -> large size picture, which is easy to appear Mosaic.

In fact, such a design, not only easy to appear Mosaic, and in the process of image magnification, because of the complex interpolation operations involved, will also be very CPU consumption.

Similarly, zooming or clipping an image also consumes CPU, but it is better than zooming in.

Therefore, it is best to carefully choose the camera preview resolution and push stream size to be as consistent as possible, so as to maximize CPU consumption.

4. Soft editing/soft solving

This reason may be well understood, soft coding/soft decoding depends on the CPU, very performance consumption, and hard coding/hard decoding is the use of special hardware codec module, will significantly reduce the BURDEN of the CPU, relatively speaking, will save a lot of power.

You just need to be careful about compatibility issues with various Android models. For some weird devices, it’s better to join the hard-coded/hard-decompressed blacklist.

5. Others

Of course, there are many other factors leading to high power consumption, which are not explained here. They are listed as follows:

  • Face recognition/beauty/filter consumes a lot of CPU/GPU
  • Excessive memory copy operations in code logic
  • Background threads frequently wake up the phone to access the network or read and write SDCard
  • Some animation effects of App
  • Other, etc.

Activity recommendation: Seven cattle to open the history of the largest promotional activities oh, the first charge 1500, the highest send 8000!