Reading time: 5~10min

Before in Tencent video binge-watching, accidentally saw a picture-in-picture function is very easy to use, do not understand the question: what is “picture-in-picture”? Talking about painting in painting, we have to mention a thing we often do, we want to watch TV on the PC browser at the same time shopping taobao, reading weibo, playing zhihu…… Nothing you can’t think of, nothing I can’t play! However, we only have one screen, so we need a secondary hero, picture-in-picture, to improve our desktop utilization and time efficiency.

What is a picture in a picture

First, let me introduce picture-in-picture: A picture-in-picture (PiP) is a Picture of a TELEVISION program (or other Picture) displayed on the entire Picture, while one or more other pictures are displayed in the corner, usually playing only the sound of the main window — from Wiki

A similar picture-in-picture effect appeared as early as 1976 in television coverage of the Montreal Olympics, which used Quantel digital frame storage to insert close-up photos of the Olympic flame during the opening ceremony.

Use picture-in-picture in your browser

As our daily social lives become more diverse, we expect to use picture-in-picture capabilities in browsers as well.

In the past two years, browser manufacturers have begun to support the function of picture in picture. Let’s take a look at the effect of picture in picture in browser:

Browser Support

The following is an introduction to the current status of picture-in-picture on mainstream browsers

  • Safari – Supported
  • Chrome – already supported
  • Firefox – Beta phase
  • IE – Planning stage

See Implementation Status for details

Safari

Back in September 2016, Safari added pick-in-Picture support via the WebKit API in macOS Sierra. In contrast to Chrome, Safari adds a picture-in-picture button to its own play control.

Chrome

  • Experimental function

In April 2017, Chrome released Android O using the native Android API to automatically play pict-in-picture videos on mobile devices. In October 2018, Chrome added the pin-in-Picture feature to PC client version 69, but this feature is disabled by default. To enable pin-in-Picture, you need to perform the following operations in your browser:

  1. The inputchrome://flagsAnd press enter
  2. Search for three keywords in turn in the search boxenable-experimental-web-platform-features(Enable experimental Web platform features under development),enable-Surfaces-for-videos(Enable VideoLayer compositing to Surface instead of video),enable-picture-in-picture(Enable picture in Picture feature for video) and set the option value toEnabled.
  3. Restart Chrome
  4. Right-click the video area on the page that contains the video and click the “Picture in Picture” option in the menu bar to watch the video
  • Officially open

In version 70, this feature is enabled by default. Install the Chrome extension to go to Picture-in-Picture. After entering picture-in-Picture, a blue icon will appear in the TAB to remind the user that the page is playing a video.

Support webRTC video streaming

Picture-in-picture in Chrome 71 supports playing videos of MediaStream objects (such as getUserMedia (), getDisplayMedia (), canvas.captureStream ()). This means that you can display a picture-in-picture window that contains the user’s webcam video stream, that is, webRTC video stream.

FireFox

To enable the Picture-in-Picture mode, use Firefox Advanced Preview edition Nightly. The details are as follows:

  1. Open theabout:configSettings page
  2. searchmedia.videocontrols.picture-in-picture.enabledClick toggle and set it totrue
  3. Restart the browser
  4. The page containing the video is displayed, and the video starts to play
  5. Right-click in the video area and the picture in Picture menu appears
  6. Turn on picture in picture

Note that the picturein mode of Chrome is slightly different from that of Firefox. When Chrome turns on The picturein mode, the video in the browser will no longer play, while Firefox has enabled the dual-screen mode of picturein and the original TAB to play simultaneously. And the current picture-in-Picture feature doesn’t even have the option to turn off picture-in-video, so you have to right-click again to turn it off.

How do developers use picture-in-picture

For developers, there is nothing more exciting than allowing users to experience the effects of picture-in-picture mode. Thanks to browser Dads for providing the corresponding API for us developers to call. Due to safari’s early implementation and Google’s use of its own API, the API has yet to be standardized (the good news is that the pictule-in-Picture Web API specification is in the DRAFT WICG stage, which is roughly the same as chrome’s API specification). I’ll take a look at the Web apis of the currently supported browsers (Chrome and Safari) :

Run it on Chrome

Let’s start with an example (the video in this example is from Tencent) :

Let’s look at the properties and methods used in the above example:

  • document.pictureInPictureEnabled

This property is used to determine whether the current document supports the picture-in-picture functionality. If the user disables pin-in-picture, it will return false if the user disables pin-in-picture.

  • document.pictureInPictureElement

This property returns the pin-in-picture element object that exists in the current document, or null if none exists, or video Element otherwise

  • video.requestPictureInPicture()

This API actually asks the video to go into picture-in-picture mode, and returns a Promise. In the Promise success callback, you get a pipWindow object that contains:

{ width, height, resize }
Copy the code

Width and height are respectively the width and height of the video entering the picture-in-Picture window, and resize can listen for an event callback that is triggered when the picture-in-picture window changes.

① Why is the API mounted on video instead of Document?

Because there can be multiple videos on a page, you need to specify the video element that triggers the picture-in-picture.

② When will the API call fail?

  1. The system does not support picture in picture
  2. Document is not allowed to use picture-in-picture due to restrictive functionality policies.
  3. Video metadata not loaded (VideoElement.readyState === 0)
  4. Video files are audio only
  5. The video element is setdisablePictureInPictureNew properties
  6. The picture-in-picture request invocation is not handled by user gesture events (such as button clicks), so be aware of the image heremouseover,mouseenter,mousemove,mouseleave,scroll,onloadSuch events do not count as user gesture events.
  • document.exitPictureInPicture()

When we want to actively exit picture-in-picture, we can call this API and the result will return a promise.

① Why is the API mounted on Document instead of Video?

At present, only one video can be displayed in the picture-in-picture window on a page. For developers, there is no need to exit which video is drawn, so just mount it to the document.

  • Onenterpictureinpicture and onleavepictureinpicture

When we want to monitor whether the video entry/exit the real picture in picture, sometimes entry/exit picture in picture is not our by calling requestPictureInPicture/exitPictureInPicture to trigger, such as user through chrome plug-ins make video into the picture in picture, At this time, these two events need to be monitored to obtain some useful information and report such behaviors.

Run it on Safari

Since Safari has native support for Pict-in-Picture since 2016, the API and Chrome are completely at odds. In Safari, the concept of “picture in picture” refers to the play mode. There are only three modes in Safari: inline, picture-in-picture and fullscreen.

Let’s start with a simple example:

As you can see from the example above, picture-in-picture properties and methods are mounted to specific video elements.

Let’s look at the properties and methods used in the above example:

  • video.webkitSupportsPresentationMode(mode)

This method checks the “presentation mode” supported by the video element. The mode parameter supports three valid values: picture-in-picture, inline, and fullscreen.

There are generally two ways to judge whether a picture within a picture is supported. The first is described in the above example, and the second is:

let supportsPiP = false;
try {
    supportsPiP = video.webkitSupportsPresentationMode('picture-in-picture');
} catch(e) {
    supportsPiP = false;
}
Copy the code

If webkitSupportsPresentationMode afferent invalid values, throws the error, as shown below:

  • video.webkitPresentationMode

This property returns one of the three valid values for the playback mode above. You can use this property to determine whether the current mode of video is picture in picture.

  • video.webkitSetPresentationMode(mode)

This method can set the current mode, when mode = ‘picture – in – the picture, is equivalent to the video in chrome. RequestPictureInPicture (). When mode = ‘inline’, is equivalent to the chrome in the document. The exitPictureInPicture (). Note that calling this method to enter or exit pin-in-picture in Safari does not return a value, nor does it return an error.

  • onwebkitpresentationmodechanged

This event can be used to listen when the current playback mode changes. Whether entering or exiting picture-in-picture, this event will trigger the listener callback. Note that the pin-in-picture window does not return the object information (including window width, height, etc.).

In Safari or Chrome, if a real-time audio and video stream is playing in Pictoin, the browser will pause the video when exiting pictoin, and you need to manually trigger the video stream to resume playing after exiting Pictoin.

Polyfill compatible

Based on current pin-in-Picture support, there is a Pin-in-Picture Polyfill library that is compatible with Chrome and Safari

Automatic picture-in-picture

The picture-in-picture feature is mentioned in the WICG picture-in-Picture draft:

“The video elements of some pages want to automatically enter/exit picture-in-picture, for example, video Web applications will benefit from some of the automatic picture-in-picture behavior when a user switches back and forth between a Web application and another application TAB. Unfortunately, user gestures aren’t currently available, so Auto picture-in-picture is needed!”

We just need to add a new attribute to the video element, autopictureinPicture:

<video autopictureinpicture></video>
Copy the code

This means that there is no need for the developer to manually invoke methods to perform the picture-in-picture behavior. The video element with the recently set Autopictureinpicture property goes picture-in-picture when the page document is hidden (if picture-in-picture is allowed), and when the page document is visible, The video elements in the picture will leave automatically.

Reference documentation

Let Chrome and Firefox achieve system-level picture-in-picture video effects

Picture-in-Picture

Github.com/WICG/pictur…

Developers.google.com/web/updates…

Pay attention to [IVWEB community] public number to get the latest weekly articles, leading to the top of life!