While choosing the right video codec for a WebRTC application can be tricky… Should you use VP8? H.264? VP9? Or AV1? How about HEVC?

The text/Tsahi Levent – Levi

Bloggeek. me/ WebrtC-vide…

WebRTC Video Codec – Tips

WebRTC used to be a very simple codec. You can have VP8, Opus and G.711. As for why G.711 was removed, it was mostly because I didn’t want you to use it, although there was really no reason to. Later, H.264 was added as a requirement for implementing video codecs. Everything is on track in the WebRTC space.

Next, Google decided to introduce VP9 in Chrome as an optional codec. Mozilla has also added VP9 to Firefox. What about Microsoft? When they switched Edge to Chromium, they got it “for free.” And Apple… Right… It’s Apple. VP9 should be in their Safari tech preview, but mainly because of the fact that Google Standia uses VP9, which may sound surprising.

Oh, and Apple decided to add HEVC to WebRTC as its own optional codec — just for good reason (layer groups). And make it more confusing for all of us.

And then we have AV1. Next generation best video codecs. For the moment. At least when it’s added to Chrome (version 90) for developers to use.

Support for video codecs across WebRTC browsers

I recently held the 2021 WebRTC Trends Seminar. It shows the current state of the supporting video codecs in the Web browser.

To summarize briefly:

  • VP8 and H.264 are ubiquitous in browsers, but as it turns out, there are problems with both

  • VP9 has not been widely adopted after years of development, but will appear in Safari “soon”

  • HEVC is “proprietary” to Apple

  • AV1, by contrast, is quite new

Video codec performance in WebRTC

Last week, I joined Philipp Hancke for this month’s WebRTC monthly “Gig.” In “Playing” this month, Philipp suggested that we look at the performance of video codecs, so he also wrote a… “Play.”

You can watch the entire play: measuring the performance of the video codec (click to view: webrtccourse.com/course/webr…

The results are interesting and sometimes surprising. But the nice thing about this is that you don’t have to stick to the information we’ve provided — you can take the code and use it yourself. Also, be sure to use it correctly in your scenario and not in the simple scenario we share, as everyone’s situation may be different.

Use VP8 or H.264 in WebRTC applications?

Today? You may be using VP8 or H.264 — or should be.

So what’s the real difference between the two? No, you can’t say that either. Rather, they produce similar video quality for a given bit rate.

But there are some subtle differences:

  • Google doesn’t really use H.264 in WebRTC. As a result, VP8 becomes the more maintained video codec of the two. For example, H.264 hasn’t supported syncing in Chrome for years (though it does now)

  • VP8 has little hardware acceleration, so in some cases it will consume more CPU.

  • H.264 has hardware acceleration. On Apple devices. Sometimes on a PC. Sometimes on Android. Sometimes, however, you won’t be able to install H.264 enabled in WebRTC because the hardware isn’t accessible and the software implementation isn’t there (royalties and the like)

  • Temporal scaling (time scalability) is only available in VP8. H.264 does not.

In our own quick test results, the H.264 decoder is superior to the VP8 decoder — regardless of whether there is hardware acceleration on H.264. It’s one thing to definitely consider.

Which one should you use? Roll the dice to decide…

VP8/H.264 or VP9 in WebRTC?

This is a real question to ponder. Should you use VP9? Last year, I suggested that this might be a good time to use VP9. But what has hardly changed is that I don’t see anyone actually using it.

No one uses it except Google.

In our tests, it had CPU utilization close to VP8. This is amazing. That’s probably why Google used it in the Google Meet.

What’s the best thing about VP9? It also supports SVC (in an unrecorded tuning manner)

What are the challenges? Apple. It still hasn’t. But it’s about time. But the question is when.

When to use HEVC in WebRTC?

The answer is simple – never.

That said, if you’re just talking between Apple devices, HEVC might be a good choice.

Is the timing right for AV1?

Yes and no.

Based on our own testing results, AV1 is significantly worse than all other codecs in terms of performance. AV1 requires twice or more CPU during encoding and decoding than any other video codec we’ve tried.

But AV1 should provide better quality than other codecs, so you might actually have to pay extra CPU. As far as I know, AV1 is used today for two reasons:

1. To deal with specific situations such as very low bit rates (CPU is not the bottleneck, but bandwidth is)

2. When you only need to decode and the encoder is in the cloud — a place where you control the hardware. However, you will pay for this in calculated costs

3. It’s also rumored to be good at decoding thumbnails

Welcome to the world of multi-codec WebRTC

WebRTC didn’t have many options when it started. Only VP8 and H.264. That’s the case. But what about now? We can provide 4-5 video codecs for your choice at any time.

There’s no reason why most of us end up using VP8. Others chose H.264, mostly for performance reasons. Most of the other codecs, while often talked about, are almost never used.

New video codecs are really promising — VP9, AV1 and even HEVC have real potential in WebRTC applications, for example. Still, they face some big challenges — primarily CPU and cross-browser usability.

To use them, a new approach is needed. That is, through one application it is sometimes possible to conduct exactly the same session using multiple video codecs.

Here are some suggestions for you to explore:

  • Higher complexity codecs are supported only on 1:1 calls, and can be dynamically switched to other video codecs when the number of calls exceeds two participants

  • Dynamically switch to a higher complexity codec at a lower bit rate

  • Enable decoding of as many codecs as possible in parallel on a single device, and then determine what the encoder should send based on its CPU power

  • Use multiple video codecs in co-broadcast — for example, AV1 with very low bit rate, followed by VP8 or VP9 with higher bit rate. Although this is not currently supported, you can open two independent peer connections using different codecs and bit rates to achieve similar results.

Is it worth a try? The answer is maybe. You can tell me if it’s important to improve the video quality in your application. In WebRTC, venturing into the multi-video codec space takes about 80% of the effort to produce the last 20% of improvements. After completing all the other simpler optimizations, go here.

WebRTC Trends in 2021

Last month, I hosted a workshop on WebRTC trends for 2021.

I introduced single video call, voice compression, background blur, E2EE and video coding optimizations. It also covers the challenges of choosing which video codec to use.