1, an overview of the


2. About the author

Rexchang (evergreen) :
Tencent technology sharing: wechat small program audio and video technology behind the story


4. Introduce small program audio and video and WebRTC respectively



What is applets audio and video?
















Tencent technology sharing: wechat small program audio and video technology behind the story






So what is WebRTC?
















Interview with the Father of WebRTC standards: WebRTC past, Present and Future


5, where is the difference between wechat small program audio and video and WebRTC?









If you are interested in technology, we can list the differences from multiple technical perspectives. Here is a detailed table for comparison:










Implementation principle:


















Underlying protocol:


















Fragmentation of mobile terminal:


















Extensibility:


















Desktop browser support:




















6, wechat small program audio and video and WebRTC is not a zero-sum game











7. Know yourself and your enemy and fully understand WebRTC















First, she’s not very pretty, but she’s got a lot going for her:












Secondly, she likes to accommodate others very much and can support all kinds of architectural schemes:








8. Establishment of communication scheme between micro program audio and video and WebRTC



After understanding these characteristics of WebRTC, our interworking scheme is relatively clear:





  • 1) First of all, the characteristics of small program audio and video interface is simple, fast start, which is the advantage of small program; This is exactly WebRTC’s disadvantage, so we don’t need to expose a dozen interface classes for WebRTC on the applet side, but continue to use the and tags for applet audio and video to solve the problem;
  • 2) Secondly, there is no official implementation of the background of WebRTC, which means there is a lot of room for play here. Tencent Video cloud can achieve a set of WebRTC background and get through it with the RTMP background used by small programs. To put it simply, Tencent video cloud to act as a matchmaker between small program audio and video and WebRTC (more specifically, it should be a translator) role.











9. Successful handshake between wechat mini program audio and video and WebRTC



The following figure shows the solution to this interworking problem:










As shown in the figure above, the principle of this interworking solution is as follows:





  • 1) Firstly, the small program on wechat terminal pushes audio and video streams to Tencent Cloud RTMP server through Tencent Video cloud SDK;
  • 2) Secondly, Tencent Cloud RTMP server will conduct preliminary conversion of audio and video data, and then pass it to Tencent Video Cloud’s real-time audio and video background cluster;
  • 3) Again, the real-time audio and video background gives the data to a module called WebrtC-proxy, where WebrtC-Proxy translates the audio and video data from the applets into a “language” that WebRTC understands;
  • 4) Finally, Chrome browser on PC can communicate with WebrTC-Proxy through the built-in WebRTC module of the browser, and then see the video image of the small program;
  • 5) If the above four processes are reversed, two-way video calls can be realized; Tencent video cloud as the star structure of the central node, multiple end (whether small programs or Chrome browser) access, it can form a multi-person audio and video solution.


10. Wechat mini program audio and video and WebRTC open up room logic





































As shown in the figure above, the principle is as follows:





  • 1) url interface no longer passes RTMP :// protocol push address, but passes room:// protocol push address Room :// For details on how to use the protocol, please refer to our documentation DOC. ;
  • During the video call, each member of the room in and out, will also be notified to your small program code through this event;
  • 3) Each item in the ROOM_USERLIST is a binary (if it’s a 1v1 video call, there’s only one person in the ROOM_USERLIST) : userID and playURL. Userid is the user, playURL is the user’s remote screen play address. All you need to do is use the tag to play the images and sounds of these remote screens;
  • 4) At the end of WebRTC, you can refer to our WebRTC API, which is more suitable for beginners than the native API of WebRTC.


11, to see the final access effect



<webrtc-room>
















Label Description:



DOC






Version requirements:









Effect demonstration:





  • 1) PC: Open the experience page with Chrome browser to experience the effect of desktop WebRTC;
  • 2) wechat terminal: found => small program => search “Tencent Video cloud”, click the WebRTC card, you can experience the effect of interworking with desktop Chrome.



Docking data:





  • 1) small program source code (including < webrTC-room > component source and demo source);
  • Component/Webrtcroom.js implements a simple room management function. Component /mainwindow.js contains code to use the WebRTC API));
  • 3) background source code (to achieve a simple room list function, while containing < webrtC-room > several required parameters of the generation code).