Food guidelines:

  1. If you are not familiar with WebRTC, you can first read the video from the front-end to understand the real-time Web communication;
  2. 6. We express a connection: The WebRTC Perfect negotiation pattern, this paper is using Google translation, modified into easy to understand statements, coupled with their own understanding, it is inevitable that there will be mistakes, I hope friends can correct in The comment area;
  3. Keep English if it is difficult to find similar words in Chinese. Such asA polite peerandA impolite peerThe literal translation isPolite equivalenceandImpolite equivalence, it is obviously difficult to understand what is being expressed, so there will be a note in the first place used, and the original text will be retained later;

The glossary

  • Peer: peers, peer connection, peer connection;
  • Signaling: To establish a P2P connection, the two parties need to exchange information. The exchange channel can be WebSocket, HTTPS request, or even datachannel channel. There are no clear restrictions on signaling channels, which are generally WebSocket connections.
  • Perfect negotiation: TODO:

Establish a connection: WebRTC perfect negotiation mode

This article introduces WebRTC perfect negotiation, describes how it works and why it is the recommended way to negotiate WebRTC connections between peers, and provides sample code to demonstrate the technique.

Because WebRTC does not mandate a specific signalling mechanism during the negotiation of a new peer connection, it is very flexible. However, despite this flexibility in signaling message transmission and communication, there is still a recommended design pattern that you should follow whenever possible, called perfect negotiation.

After browsers began to support WebRTC, people realized that some parts of the negotiation process were more complex than needed for a typical use case. This is due to a few issues with the API and some potential race conditions that need to be prevented. These issues have been resolved, allowing us to greatly simplify WebRTC negotiations. Perfect negotiation is an example of how negotiation has improved since the early days of WebRTC.

Concept of perfect negotiation

Perfect negotiation seamlessly separates the negotiation process completely from the rest of the application logic. Negotiation is essentially an asymmetric operation: one party needs to act as the “caller” and the other as the “called”. The perfect negotiation pattern eliminates differences by separating them into separate negotiation logic, so your application doesn’t need to care which end of the connection it is. As far as your application is concerned, it makes no difference whether you make or receive a call.

The biggest advantage of perfect negotiation is that both the caller and the called use the same code, so there is no need to write negotiation code that duplicates or otherwise adds levels.

// Comments start

P2p 1 p2p 2 p2p 2 p2p 2p

  1. P2p connections:

The code version:

(async() = > {const peer1 = new RTCPeerConnection();
    const peer2 = new RTCPeerConnection();
    const offer = await peer1.createOffer();
    await peer1.setLocalDescription(offer);
    await peer2.setRemoteDescription(offer);
    const answer = await peer2.createAnswer();
    awaitpeer2.setLocalDescription(asnwer); peer1.setRemoteDescription(answer); }) ();Copy the code

The image version:

The above process can be summarized as follows: the initiator creates the offer, sends it to the other party after setting it locally, the other party selects to set your offer, and then creates the corresponding answer. After setting it locally, the initiator returns it to the initiator, and then sets the answer to complete the process.

The signaling change of the initiator: have-local-offer -> stable The signaling change of the called party: have-remote-offer -> stable

  1. Signaling status:

As can be seen from the figure, there are many states of signaling, and only with strict flow control can stable state be reached

If both parties generate an offer locally and send it to the other party at the same time, the negotiation will fail

So perfect negotiation is a technical solution to this problem

// End of comment

Perfect negotiation works by assigning each of the two peers a role to play in the negotiation. , this role is completely separate from the WebRTC connection state:

  • A politePeer: If the signaling to be set conflicts with the signaling status of the peer, the peer uses rollback and the signaling status of the peer changes tostableThen set the signaling sent by the peer party.
  • A impolite peer. If the necessary signaling conflicts with its own signaling status, you should abandon setting the other party’s signaling (subject to my requirement).

This way, both partners know what will happen if there is a conflict between already sent offers. The response to error conditions becomes more predictable.

There is no restriction on who is a polite peer or a impolite peer, and it can be random.

Implement perfect negotiation (code implementation)

Let’s look at an example of implementing a perfect negotiation pattern. This code assumes that the SignalingChannel defines a class to communicate with the signaling server. Of course, your own code can use any signaling technology you like.

Note that this code is the same for both peers involved in the connection.

Create signaling and peer connections

The signaling channel needs to be opened and RTCPeerConnection needs to be created. The STUN servers listed here are clearly not real servers; You need stun.myServer. TLD to be replaced with the address of the real STUN server.

const config = {
  iceServers: [{ urls: "stun:stun.mystunserver.tld"}};const signaler = new SignalingChannel();
const pc = new RTCPeerConnection(config);
Copy the code

This code also

Connect to a remote peer

const constraints = { audio: true.video: true };
const selfVideo = document.querySelector("video.selfview");
const remoteVideo = document.querySelector("video.remoteview");


async function start() {
  try {
    const stream = await navigator.mediaDevices.getUserMedia(constraints);

    for (const track of stream.getTracks()) {
      pc.addTrack(track, stream);
    }
    selfVideo.srcObject = stream;
  } catch(err) {
    console.error(err); }}Copy the code

The function shown above by start() can be called by either of two endpoints that want to talk to each other. It doesn’t matter who goes first, negotiation will work.

This is not significantly different from the old WebRTC connection establishment code. The user’s camera and microphone are obtained by calling getUserMedia(). Then pass the generated media tracks RTCPeerConnection by passing them to addTrack(). Then, finally,

Handles tracks received

Next, we need to set up a handler for the track event to handle the inbound video and audio tracks that have been negotiated to receive over this peer connection. To this end, we implement the ontrack event handler of RTCPeerConnection.

pc.ontrack = ({track, streams}) = > {
  track.onunmute = () = > {
    if (remoteVideo.srcObject) {
      return;
    }
    remoteVideo.srcObject = streams[0];
  };
};
Copy the code

This handler executes when the track event occurs. Extract the RTCTrackEvent’strack and Streams properties using deconstruction. The former is the receiving video track or audio track. The latter is an array of MediaStream objects, each representing a stream containing the track (in rare cases, a track may belong to multiple streams at the same time). In our case, this will always contain a stream at index 0 because we passed it a stream addTrack() earlier.

Perfect negotiation logic

Now let’s move on to truly perfect negotiation logic, which functions completely independently of the rest of the application.

Handle events required for negotiation

First, we implement the RTCPeerConnection event handler onNegotiationNeeded to get a local description and send it to a remote peer using a signaling channel.

let makingOffer = false;

pc.onnegotiationneeded = async() = > {try {
    makingOffer = true;
    await pc.setLocalDescription();
    signaler.send({ description: pc.localDescription });
  } catch(err) {
    console.error(err);
  } finally {
    makingOffer = false; }};Copy the code

Note: OnNegotiationNeeded is some state change on the PC, and then a callback requires negotiation, perhaps adding a track to the PC or adding a Datachannel

Note that setLocalDescription() without arguments will be based on the current signalingState. The collection description is either the answer to the latest offer from the remote peer or a newly created offer if no negotiation has taken place. In this case, it will always be an offer, because the events required for negotiation are only fired in the stable state.

We set a Boolean variable and makingOffer marks with true that we are preparing the offer. To avoid contention, we will later use this value rather than the signal state to determine whether an offer is being processed, because the value signalingState changes asynchronously, thus introducing dazzling opportunities.

After the offer is created, set, and sent (or an error occurs), makingOffer is reset to false.

Dealing with ICE candidate

Next, we need to deal with RTCPeerConnectionevent ICecandiDate, which is how does the local ICE layer pass the candidate to us for passing to the remote peer over the signaling channel

pc.onicecandidate = ({candidate}) = > signaler.send({candidate});
Copy the code

This takes the candidate member of the ICE event and passes it to the signaling channel’s send() method for sending to the remote peer through the signaling server.

Processing signaling messages

The final piece of the puzzle is the code to process incoming messages from the signaling server. This is implemented here as an event handler on the OnMessage signal channel object. This method is called each time a message arrives from the signaling server.

let ignoreOffer = false;

signaler.onmessage = async ({ data: { description, candidate } }) => {
  try {
    if (description) {
      // If the description type sent by the peer end is Offer, the local signaling is generating Offers, or the local signaling status is not stable, signaling conflicts are considered
      const offerCollision = (description.type == "offer") && (makingOffer || pc.signalingState ! ="stable"); ignoreOffer = ! polite && offerCollision;if (ignoreOffer) {
        // If this is necessary and many signaling conflicts, then do not handle it directly
        return;
      }

      await pc.setRemoteDescription(description);
      if (description.type == "offer") {
        await pc.setLocalDescription();
        signaler.send({ description: pc.localDescription })
      }
    } else if (candidate) {
      try {
        await pc.addIceCandidate(candidate);
      } catch(err) {
        if(! ignoreOffer) {throwerr; }}}}catch(err) {
    console.error(err); }}Copy the code

After the SignalingChannel receives the incoming message from its onMessage event handler, the received JSON object is deconstructed to get the Description or candidate in it. If the incoming message has description, it is an offer or reply sent by another peer.

On the other hand, if a message has a candidate, it receives an ICE candidate from a remote peer as part of trickice. Candidates are destined by passing it to addIceCandidate().

After receiving description(offer or answer)

If we receive description, we are ready to respond to the incoming offer or answer. First, we check to make sure we are in a position to accept the offer. If the signaling state of the connection is not stable or if our connection starts its own offer process, then we need to be aware of offer conflicts.

If we are a necessary peer and we receive a conflicting offer, we will return without setting the description, but set ignoreOffer to true to ensure that we also ignore all candidates that the other party may have sent us on the signaling channel that belongs to this offer.

If we are a polite peer and we receive a conflicting offer, we don’t need to do anything special because our existing offer will be automatically rolled back in the next step.

After ensuring that we want to accept the offer, we set the remote description to the incoming offer setRemoteDescription() via a call. This lets WebRTC know what the recommended configurations of the other peers are. If we were a polite peer, we would drop our offer and accept the new offer.

If the newly set remote description is an offer, we ask WebRTC to select the appropriate local configuration setLocalDescription() by calling the RTCPeerConnection method with no arguments. This causes setLocalDescription() to automatically generate the appropriate answer for the offer received in response. We then send the answer back to the first peer over the signaling channel.

Improve the code

If you’re curious about what makes perfect negotiation so perfect, this section is for you. Here, we’ll look at each of the changes made to the WebRTC API and suggest best practices to make perfect negotiation possible.

Call setLocalDescription() with no arguments

In the past, a Negotiationneeded event was easy to handle in a way that made it easy to call setLocalDescription with arguments — that is, it was easy to conflict, in which case both sides might end up trying to make an offer at the same time, Causes one or another peer to receive an error and abort the connection attempt.

The old way
// bad case
pc.onnegotiationneeded = async() = > {try {
    await pc.setLocalDescription(await pc.createOffer());
    signaler.send({description: pc.localDescription});
  } catch(err) {
    console.error(err); }};Copy the code

Since the createOffer() method is asynchronous and takes some time to complete, the remote peer might try to send its own offer over a period of time, causing us to leave the stable state and go into the have-remote-offer state, This means we are now waiting for the offer. But once it receives the offer we just sent, so does the remote peer. This puts both parties in a state where the connection attempt cannot be completed.

Use the new invocation method

As shown in the Implementing Perfect Negotiation section, we can eliminate this problem by introducing a variable (called makingOffer here) that we use to indicate that we are sending an offer, using the updated setLocalDescription() method:

// good case let makingOffer = false; pc.onnegotiationneeded = async () => { try { makingOffer = true; await pc.setLocalDescription(); signaler.send({ description: pc.localDescription }); } catch(err) { console.error(err); } finally { makingOffer = false; }};Copy the code

We makingOffer set setLocalDescription() immediately before the call to prevent interference with sending the offer, and we do not clear it back to false until the offer has been sent to the signaling server (or an error occurs that prevents the offer from being made). In this way, we can avoid the risk of offer conflict.

Finally realize

let ignoreOffer = false;

signaler.onmessage = async ({ data: { description, candidate } }) => {
  try {
    if (description) {
      const offerCollision = (description.type == "offer") && (makingOffer || pc.signalingState ! ="stable"); ignoreOffer = ! polite && offerCollision;if (ignoreOffer) {
        return;
      }

      await pc.setRemoteDescription(description);
      if (description.type == "offer") {
        await pc.setLocalDescription();
        signaler.send({ description: pc.localDescription }); }}else if (candidate) {
      try {
        await pc.addIceCandidate(candidate);
      } catch(err) {
        if(! ignoreOffer) {throwerr; }}}}catch(err) {
    console.error(err); }}Copy the code