Fang old wet write not to series one: JavaScript core encryption

Background of 0.

One of the most important improvements in HTML5 is plug-in free support for audio and Video. Whether it is the increasing support for various Video container formats by browsers on various ends, or the improvement of Video extensibility by Media Source Extension (MSE), We can now do more and more novel things with audio and video on the browser. But also because of the openness of Web technology, for domestic mobile browsers, many companies have carried out a series of private technical transformation of HTML5 Video capabilities, including (not limited to) :

  1. Video is a Native component by default. It has the highest component level and cannot be modified.
  2. Render independently and do not respond well to the browser’s own events
  3. The auto – play policy is stricter, and mute auto – play is not supported
  4. Cannot play inline properly (playsinline)
  5. Various advertisements are recommended before and after playing, which affects user experience
  6. .

As audio and video carry more information than text and pictures, and as Internet speeds improve, they have become an important entry point for content, this is understandable for commercial companies. However, for developers, the inconsistent behavior of browsers makes it extremely difficult to implement requirements and makes for a very poor user experience.

How to break the spell? Let’s start with a real development example.

1. The demand

Since The popularity of Kuaishou and Douyin, short videos have gradually become popular, and many companies have begun to try short video products. Of course, our company has also started an entrepreneurial project of “short video + e-commerce”, in which our products mainly demand “immersive short video” imitating Douyin.

The technical solution of “immersive short video” is relatively mature and systematic for Native end, but due to various technical reasons above, there is no systematic solution for Web end at present. At the same time, for the channel side, we hope our “immersive short video” scheme can be run in wechat WebView, which is convenient for product promotion and traffic expansion.

However, for wechat WebView, it is mainly the integrated X5 engine, which has made a lot of modifications to Video, which is not limited to the technical limitations mentioned at the beginning of the article, but there are two serious ones:

  1. Does not support auto – play, mute also cannot
  2. Playsinline doesn’t work well, and the official recommendation is to use something called the same player layer

Due to these two limitations, it was difficult to implement a solution like Douyin’s “immersive short video”.

2. Break the spell

In the national live broadcast, we used WebAssembly for H264 soft solution and wrote QMInlinePlayer, a FLV player capable of live broadcast. In the development and research process, we tested several mainstream machines at that time and got quite pleasing results, as shown in the figure:

Its writing idea is also very simple: since the video file is composed of image data and audio data, why don’t we get the original image data and audio data by soft solution, and draw and play by ourselves. The idea of the whole program is shown in the figure below:

Of course, we didn’t open source this solution at the time due to the chaos within the company, but fortunately there were many similar practices in the community later on, such as:

  • Taobao developed from the Web end H265 player
  • Community of individuals trying out the Web side H265 player
  • .

3. New WXInlinePlayer

This year, due to our demand, we started to dabbled in short video business gradually, and hoped to spread it in wechat WebView. Therefore, we have made a lot of modifications and improvements to QMInlinePlayer based on our experience in the nationwide live broadcast:

  1. H264 decoding performance improved
  2. YUV420 rendering performance improved
  3. Output smaller target file sizes
  4. Reduce memory footprint
  5. Added frame chasing strategy to low end machines

At the same time, we have expanded support for more browsers and improved compatibility. Current compatibility support includes (but is not limited to) :

  1. IOS 9+ (with Safari and Safari WebView)
  2. Android 5+ (some 4.4.2+ system browsers also support it)
  3. IE11+
  4. Chrome 24+
  5. Firefox 16+
  6. Safari 6.2 +
  7. Edge 15+

From QMInlinePlayer to WXInlinePlayer we started again and made it open source on Github. Of course, WXInlinePlayer is currently running stably in our own product Haohuimai, and the experience of the whole wechat terminal product is not inferior to Native experience (loading/second), which has achieved good results.

At the same time, we have also compared the performance and memory usage of similar existing H264 solutions, and WXInlinePlayer offers significant improvements in all aspects.

This test is multiple means, different devices will have different results, test video from WXInlinePlayer test video, BroadwayJS required MP4 with this test video transcoding ffMPEG.

4. Planning

At present, the overall function of WXInlinePlayer only covers the needs of our own products. More requirements and the realization of functions require more people to participate in WXInlinePlayer to help us better improve WXInlinePlayer. Of course, our own improvement plan also includes:

  1. Video streaming resolution, further improve the first frame display and memory usage
  2. Supports FLV live streaming
  3. Further improve H264 parsing performance

For more information you can view the documentation for WXInlinePlayer here.

Fang Laoshi, have you learned?