Real-time screen sharing has been widely used in video conferences, live games, online education and other scenarios. The recent acquisition of Screen sharing social app Squad by Twitter shows how real-time screen sharing can be incorporated into more industries, opening the way for richer gameplay.

As the first step of real-time screen sharing, the realization of screen recording is different in different terminals and systems.

Here is the third part of this series, how to achieve iOS screen sharing recording collection **. 支那

On iOS, the scope of screen recording is mainly divided into two types: the current App screen and the entire mobile phone screen. The following will describe in detail the usage scenarios of screen sharing, the implementation of iOS screen sharing, and the limitations of iOS system versions in the implementation process.

iOS 9

function

Starting with iOS 9, Apple provides replayKit. framework for developers to implement on-screen recording. Because the recording function involves user privacy, a dialog box is displayed asking the user whether to allow recording.

When recording, the developer can record the picture and sound in the current App screen, and also turn on the microphone to record the external sound of the device. For example, during the live broadcast of mobile games, the developer can explain while playing.

Once you’ve recorded it, you can preview it, edit it, or share it in some way.

RPScreenRecoder recording process for iOS 9

limit

  • It can only be started in the App, and recording cannot be started through the control center of the system.

  • Only the contents on the main screen of the current App can be recorded, such as notifications and keyboards.

  • It can not live broadcast audio and video, but only record the contents of the main screen into MP4 files;

  • Recorded MP4 files are not directly accessible, but can only be previewed and shared through the RPPreviewViewController.

RPScreenRecoder recording limits for iOS 9

iOS 10

function

Starting with iOS 10, Apple added Live Broadcast on top of iOS 9.

Live Broadcast uses the Broadcast Upload Extension and Broadcast Setup UI Extension to achieve screen recording and real-time audio and video data acquisition. The obtained data can be directly transmitted to the third-party streaming media service through the mobile phone to achieve the effect of real-time Broadcast. Therefore, the App containing the two extensions has the capability of Broadcast Services.

Broadcast Setup UI Extension is after the user successfully in RPBroadcastActivityViewController loading and display, select a has Broadcast Services App, The Extension UI is displayed. The main function of this Extension is to customize the recording configuration interface.

Broadcast Upload Extension triggers event callbacks and recorded audio and video data callbacks during recording after the recording configuration interface is complete. Developers can process logic in this callback, such as transferring the obtained audio and video data to third-party streaming services.

Broadcast Service workflow of iOS 10

RPBroadcastController can be used to start, pause, resume, and finish a recording. When recording is interrupted by another process (such as making a phone call), the recording is paused by default. When entering the foreground, the system will pop up to prompt the user whether to resume recording. If the recording is resumed, the recording is resumed; otherwise, the recording is completed.

RPScreenRecorder added CameraEnable. During recording, you can enable or disable the front camera. After the front camera is enabled, the images captured by the camera are displayed on the home screen. In addition, the Camera ReViewView is added, allowing you to customize the position of the camera’s preview view after capturing videos. When this function is applied in scenes such as game live broadcast, the head picture of the anchor can be displayed in the live broadcast screen.

IOS 10’s RPScreenRecoder supports camera capture

limit

Only the contents on the main screen of the current App can be recorded, such as notifications and keyboards.

iOS 11

function

Starting from iOS 11, Apple provides an upgraded version of ReplayKit 2. Once an App with Broadcast Services is installed on the phone, it can record the system through the control center of the iOS system and appear in the selection list of available apps. Meanwhile, the recording scope will be the entire phone screen.

(iOS 11 starts system-level recording from the control Center)

(iOS 11 Use the control center to hold down and select Broadcasr Service to record)

In terms of cameras, ReplayKit 2 also adds the ability to quickly switch between front and rear via RPScreenRecorder’s cameraPosition Settings.

When collecting screen data and audio data in App, RPScreenRecorder can directly obtain audio and video data collected in real time in callback through Capture. Developers can do their own business processing logic for these data, such as synthesise a video file.

Due to priority reasons, recording or broadcasting through ReplayKit in App will be interrupted by the system’s screen recording, and the App will be notified in the form of RPScreenRecorderDelegate. In this case, the recording in App will be ignored, and the App should update the UI to remind users.

If you don’t want to own App content were recorded and public radio and other media streaming service, ReplayKit 2 provides Broadcast Pairing, specify when loading BroadcastActivityViewContronller PreferredExtension, That is, the Bundle ID of Extension can be Broadcast in the specified Broadcast Service. Other Broadcast Service apps will not display the Bundle ID.

(Broadcast Pairing Workflow Diagram 1 for iOS 11)

(Broadcast Pairing Workflow Diagram 2 for iOS 11)

limit

Recording the entire screen of the phone can only be done through the control center system.

iOS 12

function

Since iOS 12 ReplayKit 2 in order to solve the record (recording system level) when the phone’s screen can only start from the system control center, adds RFSystemBroadcastPickerView, this view and in the control center long recording pop-up view, You can optionally use the Broadcast Service to Broadcast.

(ReplayKit and ReplayKit2 recordings before iOS 12)

(After iOS 12, ReplayKit2 starts system-level recording in App)

Although the recording is started in the App, it is recorded in the whole phone screen, and the recording started by the PickerView can be closed through the system control center, and the recording started by the system control center can be closed through the PickerView.

RFSystemBroadcastPickerView also like BroadcastActivityViewContronller, specify PreferredExtension, implement Broadcast ability of Pairing.

(by RPSystemBroadcastPickerView start recording system level)

(after RPSystemBroadcastPickerView start shows Broadcast Pairing option)

If only allow login successful cases to use your App’s Broadcast Service, can do it in BroadcastStartedWithSetupInfo login to check, if there is no login successful can call FinishBroadcastWithError, A dialog box will pop up asking the user to choose whether to log in or not.

(Login verification of Broadcast Service)

To prevent your App content from being recorded or broadcast, such as sensitive audio or video, and other sensitive information, Can use UIScreen. IsCaptured or listening UIScreenCapturedDidChangeNotification to judge whether the current is using the screen recording, if is to play audio or hidden, stopped playing video.

Airplay Screen Mirroring is a special case. After evaluating Uiscreen. isCaptured, you need to determine whether uiscreen. screens. Count is equal to 1. According to the need to take measures to protect App content.

conclusion

From iOS 9 to 12, different system capabilities affect the realization of screen recording acquisition. In summary, the main ways to achieve recording within the current App screen and the whole mobile phone screen are as follows:

First, to record the content on the current App screen, there are two methods: one is to obtain the collected data through RPScreenRecorder, the other is to implement or use other apps with broadcast Service capabilities to collect and record the screen.

Second, if it is to record the content of a phone’s screen, there are two ways, one is through RPSystemBroadcastPickerView arousal system recorded within the App, one is the control center by mobile phones to record button arousal system to record, before starting the recording system, You need to select an App that has the broadcast Service capability. The App will stream the collected audio and video data to the specified streaming media server.

If you want to implement broadcast Service by yourself, you can customize the UI to implement personalized functions such as information acquisition and verification based on your own requirements and make full use of the convenience brought by the function to enrich and enhance your App.