Flutter is Google’s mobile UI framework that allows you to quickly build high quality native user interfaces on iOS and Android, and is being used by more and more developers. Pale Cloud also provides the powerful Pano Flutter SDK, which is stable and easy to use and covers various functions such as voice calling, video calling, interactive whiteboard, interactive live broadcasting, and cloud recording. In the previous post “Pano Flutter SDK New Release”, we gave you a detailed access process for the SDK. Today we will continue to talk about our design ideas and practical experience with the Pano Flutter SDK.


The overall structure

Pano provides a complete high-performance SDK for native application development, so the Pano Flutter SDK encapsulates our SDK in the form of a plug-in package. Similarly, in RN we use the NativeModule to implement the Pano RN SDK. The overall structure of the cross-platform SDK of Pano mobile terminal is shown in the figure below:

SDK is divided into three layers, the bottom layer is Pano native SDK (IOS &Android). The bridge layer is based on the native SDK. Since the communication between Flutter and RN with the native layer is asynchronous communication, and a specific communication mode is required (Flutter uses the platform channel scheme, while RN uses the native module scheme), cross-platform calls need to be transformed before native SDK methods can be invoked. Therefore, the bridge layer will be divided into two parts, the native SDK bridge and the cross-platform (Flutter&RN) bridge, to achieve the purpose of maximizing code reuse. The native SDK interface is repackaged into a general asynchronous interface, on which the communication interfaces of Flutter and RN are respectively connected. The top layer of SDK is the cross-platform layer, which encapsulates the functional interface of Flutter or RN platform with the communication interface of the native layer.

Although the final structure is relatively simple and clear, due to the great differences between Flutter or RN in view update mechanism and native development, as well as the different data structures between cross-platform layer and native layer, there are many difficulties or pitches in the design and implementation of SDK involving data conversion, object mapping, memory management and other problems. Next, I will combine the design ideas and practical experience of SDK, and talk about the solutions or areas needing attention for several typical problems.


The working process

The API provided by the Pano Flutter SDK basically maintains a one-to-one correspondence with the native SDK, so that developers can easily apply the experience of native SDK development to Flutter. However, due to Flutter’s special Platform Channel scheme and view update mechanism, the native SDK interface is not simply encapsulated through transmission. The SDK invocation process is shown in the figure below:

The SDK uses MethodChannel and EventChannel in Flutter platform channel to realize the communication between Flutter layer and native layer, where MethodChannel is used by Flutter to call native layer methods. EventChannel is used to communicate from the native layer to the Flutter layer, where native layer callback messages are passed. When a developer calls the Flutter Layer interface, the SDK uses the corresponding MethodChannel to pass the method name and parameters to the Native layer, where the SDK implements the Flutter Native Bridge to handle these calls.

Recommendation: When the native layer receives a method call from MethodChannel (e.g., iOS is -[FlutterPlugin HandleMethodCall: Result :]), use a reflection call (e.g. : IOS NSSelectorFromString used in obtaining the selector, then through [NSObject performSelector: withObject:] calls) Native SDK Bridge method, In this way, the logic of Flutter can be isolated from the logic of the native bridge layer as much as possible. On the one hand, the logic of the native bridge layer can be thinly connected with the logic of the Flutter layer. On the other hand, the logic of the native bridge layer, which often needs to follow the changes of the native SDK, can be reused with other cross-platform frameworks (such as RN) to reduce the maintenance cost. Note: There is no ready-made binary data type in Flutter, which is usually replaced by Uint8List. However, after conversion through the platform channel, it will be converted to FlutterStandardtypedData type in iOS terminal, which cannot be automatically converted to NSData type. The actual NSData object needs to be retrieved from its property data. However, when you call the Flutter layer from the native layer, you can pass the NSData object directly, which will be automatically converted to a Uint8List in the Flutter layer. The platform channel in Flutter actually encodes the passing data into a message that is sent across threads to the host native layer where the application is located. In addition, the Native SDK Bridge connects with the Native SDK, and when the return value after the implementation of the Native SDK method is returned through callback, the data is also encoded into a message and returned to the Flutter layer in the same way. Messages and responses throughout the process are asynchronous, which is why the Flutter layer interface is designed to be asynchronous.

Note: In the MethodChannel type, calling native methods uses Future


(String method, [dynamic arguments]), there is no problem with direct calls when the SDK returns basic types of data supported by Flutter, for example when the interface for getting SDK version number returns String type, Then the Flutter layer interface can be implemented as: Static Future

getSdkVersion() {// java.lang.String in IOS and java.lang.String in AndroID can be automatically turned into Flutter’s String return _methodChannel.invokeMethod(‘getSdkVersion’); } However, when a non-basic type is returned, the return value needs to be converted. For example, when the audio interface is opened, the return value is an enumeration type ResultCode. If it is implemented directly as follows, an error will be generated: Future

startAudio() { return _methodChannel.invokeMethod(‘startAudio’); // error: return int does not automatically convert Flutter enumeration types} need to add conversion logic, e.g. Future

startAudio() { return _methodChannel.invokeMethod(‘startAudio’).then((value) { return ResultCodeConverter.fromValue(value).e as T; // ResultCodeConverter is a tool class that converts int to ResultCode}); } Suggestions: Since there are a large number of methods to return ResultCode in the SDK, and the conversion code is cumbersome and redundant in each interface implementation, we can extract a common template method for this situation, which can greatly improve the code simplicity. For example: Future


(String method, [Map

arguments]) {if (T == resultCode) {// When the current pattern is resultCode, Increase the transformation logic return _methodChannel. InvokeMethod (method, arguments).then((value) { return ResultCodeConverter.fromValue(value).e as T; }); } else {/ / other can automatic conversion is returning the results return _methodChannel. InvokeMethod () method, the arguments; }} This is how the Flutter calls the native layer. What should we do when the native layer needs to call back events to the Flutter layer? This is where EventChannel comes in. Let’s look at the basic flow of EventChannel:

The native layer calls setStreamHandler (for iOS -[flutterEventChannel setStreamHandler:]) to register the Handler implementation; Once the EventChannel is initialized, Through StreamHandler onListen (iOS – [FlutterStreamHandler onListenWithArguments: eventSink:]) callback interface to get eventSink reference and save; The Flutter layer calls EventChannel’s receiveBroadcastStream to register listening; The native layer sends event messages by calling EventSink. Recommendation: EventChannel is a data flow channel. Unlike MethodChannel, it does not encapsulate a model for method callbacks. However, in the current SDK, both the native layer and the Flutter layer are method callbacks, so we assemble the callback data into keyvalue pairs in a specific format, such as: {” methodName “: XXXX, / / callback method name” data “: [XXXX, XXXX…]. Void setEventHandler(RTCEngineEventHandler) {_handler = handler; void setEventHandler(RTCEngineEventHandler) {_handler = handler; . _eventChannel.receiveBroadcastStream().listen((event) { final eventMap = Map

.from(event); final methodName = eventMap[‘methodName’] as String; final data = List

.from(eventMap[‘data’]); _handler? .process(methodName, data); }); } With the above scheme, most of the functionality of the native SDK can be packaged and provided in the form of the Flutter SDK. However, one important issue remains: how to set up the logic of the native tier view.


Setting the native view

Since the platform channel scheme provided by Flutter essentially passes data between threads in the form of a byte stream, it is not supported for non-serialized objects such as native layer views. Flutter provides a platform-views solution on how to embed native views. Developers can create mappings of native views in Flutter’s layer (UIKitView for iOS, AndroidView for Android). It is embedded in the Widget.

So how do you pass the generated native view objects to the native layer SDK? After the native view is created by Flutter, it will return the corresponding unique ID of the view, so the most intuitive method is to generate corresponding MethodChannel on the native layer and Flutter layer respectively after the ID is returned, and form the key-value pair cache. MethodChannel is looked up by ID at invocation time, and method invocation messages are passed through MethodChannel. But there are two obvious drawbacks:

MethodChannel is not directly associated with the Widget, so you need to manually clear MethodChannel from the key-value pair when the Widget is destroyed; Using an ID as the identity of the native view may cause a call to an invalid MethodChannel to throw an exception due to a lack of validity checking. In general, native view is required to be passed in as a parameter in native SDK methods. However, since the corresponding native view object can only be accessed in native layer through the corresponding methodChannel of view, it is impossible to directly design methods similar to native SDK in Flutter layer.

Suggestions: In Pano Flutter SDK, in order to keep the interface consistency with the native SDK as much as possible, we adopted a curving solution to save the country. After creating the render view RTCSurfaceView (StatefulWidget), the callback returns the ViewModel object that holds MethodChannel: class RtcSurfaceViewModel { final MethodChannel _methodChannel;



(String method, [Map

arguments]) { if (T == ResultCode) { return _methodChannel.invokeMethod(method, arguments).then((value) { return ResultCodeConverter.fromValue(value).e as T; }); } else { return _methodChannel.invokeMethod(method, arguments); }}

RtcSurfaceViewModel(this._methodChannel); } Then define the corresponding Flutter layer interface according to the SDK method that requires the native view, accept the ViewModel as the parameter, and the method implementation calls the ViewModel’s methodChannel to pass the method message, such as calling StartVideo when the video is opened. The interface is defined as follows: Future

startVideo(RtcSurfaceViewModel viewModel, {RtcRenderConfig config}) { config ?? = RtcRenderConfig(); return viewModel.invokeMethod(‘startVideo’, {‘config’: config.toJson()}); } A method call is received on the corresponding MethodChannel in the native layer view. The corresponding SDK method (such as startVideo) is called through the Engine object cached in the native layer view, and the interface call is completed by passing it to the native layer view. Doing so, on the one hand, associates MethodChannel with the Widget, and on the other hand, uses the ViewModel object on the interface calls to ensure the validity of the pass-through. In addition, the interface is basically consistent with the native SDK, which reduces the understanding cost of the developers docking with SDK, and also takes into account the maintenance cost of the code.



Nowadays, developers often encounter various needs or problems of cross-platform development, while Pale Cloud has always insisted on putting developers first and working with users. Pano Flutter all open source SDK, you can by making (https://github.com/PanoVideo/)… Or Gitee (https://gitee.com/pano-video/…) See the complete source code. This article introduces the cross-platform SDK design and practice experience of Pano Flutter SDK, hoping to bring some help and inspiration to everyone. Bold bold