WebRTC communication related API is very much, in order to avoid too long, part of the article uses pseudo code to explain. Detailed code examples can be found on my Github page

Previous links:

WebRTC: session description protocol
WebRTC: Network penetration for the connection establishment process

API Introduction

In the previous chapters, we have introduced the important knowledge points related to WebRTC, including the network protocol, session description protocol, how to perform network penetration, etc. The rest is the API of WebRTC. There are many apis related to WebRTC communication, which mainly complete the following functions:

Signaling exchange
Communication candidate address exchange
Audio and video acquisition
Audio and video sending and receiving

There are too many related apis, so in order to avoid too much space, part of this article uses pseudocode to explain. The detailed code can be found at the end of this article or on Github.

1. Signaling exchange

Is the key link in WebRTC communication, the exchange of information including codec, network protocol, candidate address and so on. WebRTC does not specify how signaling is exchanged, leaving it up to the application to decide, such as using WebSocket. The sender pseudocode is as follows:

const pc = new RTCPeerConnection(iceConfig); const offer = await pc.createOffer(); await pc.setLocalDescription(offer); sendToPeerViaSignalingServer(SIGNALING_OFFER, offer); // The sender sends a signaling messageCopy the code

The pseudo-code of the receiver is as follows:

const pc = new RTCPeerConnection(iceConfig); await pc.setRemoteDescription(offer); const answer = await pc.createAnswer(); await pc.setLocalDescription(answer); sendToPeerViaSignalingServer(SIGNALING_ANSWER, answer); // The receiver sends a signaling messageCopy the code

2. Candidate address exchange service

When session description information is set locally and media streams are added, the ICE framework starts collecting candidate addresses. After collecting the candidate addresses, the two sides need to exchange the candidate addresses and know the appropriate candidate address pair.

The exchange of candidate addresses, again using the previously mentioned signaling service, has the following pseudo-code:

// Set the local session description to constlocalPeer = new RTCPeerConnection(iceConfig);
const offer = await pc.createOffer();
await localPeer.setLocalDescription(offer); // Collect audio and video locally constlocalVideo = document.getElementById('local-video');
const mediaStream = await navigator.mediaDevices.getUserMedia({ 
    video: true, 
    audio: true
});
localVideo.srcObject = mediaStream; // Add audio and video streams mediastream.getTracks ().foreach (track => {localPeer.addTrack(track, mediaStream); }); // Exchange candidate addresseslocalPeer.onicecandidate = function(evt) {
    if(evt.candidate) { sendToPeerViaSignalingServer(SIGNALING_CANDIDATE, evt.candidate); }}Copy the code

3. Audio and video collection

You can use the getUserMedia interface provided by the browser to collect local audio and video.

const localVideo = document.getElementById('local-video');
const mediaStream = await navigator.mediaDevices.getUserMedia({ 
    video: true, 
    audio: true
});
localVideo.srcObject = mediaStream;
Copy the code

4. Audio and video sending and receiving

Add the collected audio and video tracks through addTrack and send them to the remote end.

mediaStream.getTracks().forEach(track => {
    localPeer.addTrack(track, mediaStream);
});
Copy the code

The remote end can monitor the arrival of audio and video by listening onTrack and play them.

remotePeer.ontrack = function(evt) {
    const remoteVideo = document.getElementById('remote-video');
    remoteVideo.srcObject = evt.streams[0];
}
Copy the code

The complete code

Contains two parts: client code, server code.

1. Client code

const socket = io.connect('http://localhost:3000');

const CLIENT_RTC_EVENT = 'CLIENT_RTC_EVENT';
const SERVER_RTC_EVENT = 'SERVER_RTC_EVENT';

const CLIENT_USER_EVENT = 'CLIENT_USER_EVENT';
const SERVER_USER_EVENT = 'SERVER_USER_EVENT';

const CLIENT_USER_EVENT_LOGIN = 'CLIENT_USER_EVENT_LOGIN'; // Login const SERVER_USER_EVENT_UPDATE_USERS ='SERVER_USER_EVENT_UPDATE_USERS';

const SIGNALING_OFFER = 'SIGNALING_OFFER';
const SIGNALING_ANSWER = 'SIGNALING_ANSWER';
const SIGNALING_CANDIDATE = 'SIGNALING_CANDIDATE';

let remoteUser = ' '; // Remote userlet localUser = ' '; // Local login userfunction log(msg) {
    console.log(`[client] ${msg}`);
}

socket.on('connect'.function() {
    log('ws connect.');
});

socket.on('connect_error'.function() {
    log('ws connect_error.');
});

socket.on('error'.function(errorMessage) {
    log('ws error, ' + errorMessage);
});

socket.on(SERVER_USER_EVENT, function(msg) {
    const type = msg.type;
    const payload = msg.payload;

    switch(type) {
        case SERVER_USER_EVENT_UPDATE_USERS:
            updateUserList(payload);
            break;
    }
    log(` [${SERVER_USER_EVENT}] [${type}].${JSON.stringify(msg)}`);
});

socket.on(SERVER_RTC_EVENT, function(msg) {
    const {type} = msg;

    switch(type) {
        case SIGNALING_OFFER:
            handleReceiveOffer(msg);
            break;
        case SIGNALING_ANSWER:
            handleReceiveAnswer(msg);
            break;
        case SIGNALING_CANDIDATE:
            handleReceiveCandidate(msg);
            break; }}); asyncfunction handleReceiveOffer(msg) {
    log(`receive remote description from ${msg.payload.from}`); Const remoteDescription = new RTCSessionDescription(MSG. Payload. SDP); // set the remoteDescription const remoteDescription = new RTCSessionDescription(MSG. remoteUser = msg.payload.from; createPeerConnection(); await pc.setRemoteDescription(remoteDescription); // TODO error handling // local audio and video collection constlocalVideo = document.getElementById('local-video');
    const mediaStream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
    localVideo.srcObject = mediaStream; mediaStream.getTracks().forEach(track => { pc.addTrack(track, mediaStream); // pc.addTransceiver(track, {streams: [mediaStream]}); // This can also work}); // pc.addStream(mediaStream); Const answer = await pc.createAnswer(); const answer = await PC.createAnswer (); // TODO error handling await pc.setLocalDescription(answer); // TODO error handling await pc.setLocalDescription(answer); sendRTCEvent({type: SIGNALING_ANSWER,
        payload: {
            sdp: answer,
            from: localUser,
            target: remoteUser
        }
    });
}

async function handleReceiveAnswer(msg) {
    log(`receive remote answer from ${msg.payload.from}`); const remoteDescription = new RTCSessionDescription(msg.payload.sdp); remoteUser = msg.payload.from; await pc.setRemoteDescription(remoteDescription); // TODO error handling} asyncfunction handleReceiveCandidate(msg){
    log(`receive candidate from ${msg.payload.from}`); await pc.addIceCandidate(msg.payload.candidate); /** * send user related messages to the server * @param {Object}type: 'xx', payload: {} }
 */
functionsendUserEvent(msg) { socket.emit(CLIENT_USER_EVENT, JSON.stringify(msg)); } /** * Send RTC related messages to server * @param {Object} MSGtype: 'xx', payload: {} }
 */
function sendRTCEvent(msg) {
    socket.emit(CLIENT_RTC_EVENT, JSON.stringify(msg));
}

letpc = null; /** * Invite users to join video chat * 1, start local video collection * 2, exchange signaling */ asyncfunction startVideoTalk() {// Open local video constlocalVideo = document.getElementById('local-video');
    const mediaStream = await navigator.mediaDevices.getUserMedia({
        video: true, 
        audio: true
    });
    localVideo.srcObject = mediaStream; // createPeerConnection createPeerConnection(); // Add media streams to webrTC's audio and video transceiver mediastream.getTracks ().foreach (track => {pc.addtrack (track, mediaStream); // pc.addTransceiver(track, {streams: [mediaStream]}); }); // pc.addStream(mediaStream); // This will work for now, but the interface will be deprecated later}function createPeerConnection() {
    const iceConfig = {"iceServers": [
        {url: 'stun:stun.ekiga.net'},
        {url: 'turn:turnserver.com', username: 'user', credential: 'pass'}}; pc = new RTCPeerConnection(iceConfig); pc.onnegotiationneeded = onnegotiationneeded; pc.onicecandidate = onicecandidate; pc.onicegatheringstatechange = onicegatheringstatechange; pc.oniceconnectionstatechange = oniceconnectionstatechange; pc.onsignalingstatechange = onsignalingstatechange; pc.ontrack = ontrack;return pc;
}

async function onnegotiationneeded() {
    log(`onnegotiationneeded.`); const offer = await pc.createOffer(); await pc.setLocalDescription(offer); // TODO error handling sendRTCEvent({type: SIGNALING_OFFER,
        payload: {
            from: localUser, target: remoteUser, SDP: pc.localDescription }}); }function onicecandidate(evt) {
    if (evt.candidate) {
        log(`onicecandidate.`);

        sendRTCEvent({
            type: SIGNALING_CANDIDATE,            
            payload: {
                from: localUser, target: remoteUser, candidate: evt.candidate } }); }}function onicegatheringstatechange(evt) {
    log(`onicegatheringstatechange, pc.iceGatheringState is ${pc.iceGatheringState}. `); }function oniceconnectionstatechange(evt) {
    log(`oniceconnectionstatechange, pc.iceConnectionState is ${pc.iceConnectionState}. `); }function onsignalingstatechange(evt) {
    log(`onsignalingstatechange, pc.signalingstate is ${pc.signalingstate}. `); } // Call pc.addtrack (track, mediaStream), the remote peer onTrack will trigger twice Evt. Streams [0] point to the same mediaStream reference / / this behavior is a little strange, making issue has referred to https://github.com/meetecho/janus-gateway/issues/1313let stream;
function ontrack(evt) {
    // if(! stream) { // stream = evt.streams[0]; / /}else {
    //     console.log(`${stream === evt.streams[0]}`); / / here fortrue
    // }
    log(`ontrack.`);
    const remoteVideo = document.getElementById('remote-video'); remoteVideo.srcObject = evt.streams[0]; } // Click the user list asyncfunction handleUserClick(evt) {
    const target = evt.target;
    const userName = target.getAttribute('data-name').trim();

    if (userName === localUser) {
        alert('Can't have a video conversation with yourself');
        return;
    }

    log(`online user selected: ${userName}`); remoteUser = userName; await startVideoTalk(remoteUser); } /** * Update user list * @param {Array} users user list, for example, [{name:'Ming', name: 'jack'}] * /function updateUserList(users) {
    const fragment = document.createDocumentFragment();
    const userList = document.getElementById('login-users');
    userList.innerHTML = ' ';

    users.forEach(user => {
        const li = document.createElement('li');
        li.innerHTML = user.userName;
        li.setAttribute('data-name', user.userName);
        li.addEventListener('click', handleUserClick); fragment.appendChild(li); }); userList.appendChild(fragment); } /** * User login * @param {String} loginName User name */function login(loginName) {
    localUser = loginName;
    sendUserEvent({
        type: CLIENT_USER_EVENT_LOGIN, payload: { loginName: loginName } }); } // Process the loginfunction handleLogin(evt) {
    let loginName = document.getElementById('login-name').value.trim();
    if (loginName === ' ') {
        alert('User name empty! ');
        return;
    }
    login(loginName);
}

function init() {
    document.getElementById('login-btn').addEventListener('click', handleLogin);
}

init();
Copy the code

2. Server code

// add ws service const IO = require('socket.io')(server);
let connectionList = [];

const CLIENT_RTC_EVENT = 'CLIENT_RTC_EVENT';
const SERVER_RTC_EVENT = 'SERVER_RTC_EVENT';

const CLIENT_USER_EVENT = 'CLIENT_USER_EVENT';
const SERVER_USER_EVENT = 'SERVER_USER_EVENT';

const CLIENT_USER_EVENT_LOGIN = 'CLIENT_USER_EVENT_LOGIN';
const SERVER_USER_EVENT_UPDATE_USERS = 'SERVER_USER_EVENT_UPDATE_USERS';

function getOnlineUser() {
  return connectionList
  .filter(item => {
    returnitem.userName ! = =' ';
  })
  .map(item => {
    return {
      userName: item.userName
    };
  });
}

function setUserName(connection, userName) {
  connectionList.forEach(item => {
    if(item.connection.id === connection.id) { item.userName = userName; }}); }function updateUsers(connection) {
  connection.emit(SERVER_USER_EVENT, { type: SERVER_USER_EVENT_UPDATE_USERS, payload: getOnlineUser()});  
}

io.on('connection'.function (connection) {

  connectionList.push({
    connection: connection,
    userName: ' '}); Emit (SERVER_USER_EVENT, {// emit(SERVER_USER_EVENT, {// emit(SERVER_USER_EVENT, {// emit(SERVER_USER_EVENT, {// emit)type: SERVER_USER_EVENT_UPDATE_USERS, payload: getOnlineUser()});
  updateUsers(connection);

  connection.on(CLIENT_USER_EVENT, function(jsonString) {
    const msg = JSON.parse(jsonString);
    const {type, payload} = msg;

    if (type === CLIENT_USER_EVENT_LOGIN) {
      setUserName(connection, payload.loginName);
      connectionList.forEach(item => {
        // item.connection.emit(SERVER_USER_EVENT, { type: SERVER_USER_EVENT_UPDATE_USERS, payload: getOnlineUser()}); updateUsers(item.connection); }); }}); connection.on(CLIENT_RTC_EVENT,function(jsonString) {
    const msg = JSON.parse(jsonString);
    const {payload} = msg;
    const target = payload.target;

    const targetConn = connectionList.find(item => {
      return item.userName === target;
    });
    if(targetConn) { targetConn.connection.emit(SERVER_RTC_EVENT, msg); }}); connection.on('disconnect'.function () {
    connectionList = connectionList.filter(item => {
      returnitem.connection.id ! == connection.id; }); connectionList.forEach(item => { // item.connection.emit(SERVER_USER_EVENT, {type: SERVER_USER_EVENT_UPDATE_USERS, payload: getOnlineUser()});
      updateUsers(item.connection);
    });    
  });
});
Copy the code

Write in the back

WebRTC has a lot of apis, because WebRTC is inherently complex. Over time, some of WebRTC’s apis (including some protocol details) have been changed or deprecated, and there are backward compatibility complications, such as adding streams after local video capture, You could have used addStream or addTrack or addTransceiver, or for example, migrated the session description version from plan-b to unified plan.

It is suggested that you hand – off the code, deepen understanding.

A link to the

2019.08.02 – video – talk – using webrtc
Developer.mozilla.org/en-US/docs/…
onremotestream called twice for each remote stream

Follow [IVWEB community] public number to get the latest articles every week, leading to the top of life!

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

WebRTC: A simple example of video chat

Previous links:

API Introduction

1. Signaling exchange

2. Candidate address exchange service

3. Audio and video collection

4. Audio and video sending and receiving

The complete code

1. Client code

2. Server code

Write in the back

A link to the

WebRTC: A simple example of video chat

Previous links:

API Introduction

1. Signaling exchange

2. Candidate address exchange service

3. Audio and video collection

4. Audio and video sending and receiving

The complete code

1. Client code

2. Server code

Write in the back

A link to the

Related Posts

Do you know all these parseInt questions?

[luffie] _leetcode-589-n [luffie] _leetcode-589-n

Typescript makes a snake game