Google, Dart language, Flutter Engine, Responsive design mode, native rendering.

Flutter is a cross-platform mobile UI framework released by Google in 2018. Unlike React Native, which is developed in Javascript, Flutter is programmed in Dart, so no Javascript engine is required for implementation, but the actual effect is ultimately rendered in native.

Graphics performance is comparable to native apps

While other frameworks are only OEM packaged, the Flutter can be drawn directly with Skia.

Cross-platform: mobile, Web, desktop, embedded

Frame structure

According to the architecture diagram, the Flutter Framework can be divided into Framework layer and Engine layer.

The Framework section is written in the Dart language and is the focus of this series of articles.

The Engine part is implemented in C++. The engine supports the framework and acts as a bridge between the framework and the system (Android/iOS).

The Flutter Framework: The entire Framework layer is implemented in the Dart language. This layer provides a base library for handling animation, drawing, gestures, etc. It also encapsulates a library of UI components based on drawing and subdivides them into two styles of components.

Foundation, Animation, Painting, and Gestures are the UI layers of Dart implementations that provide Animation, gesture, and Painting.

At runtime, the Rendering layer builds a Widget tree, and when the Rendering layer changes, the Widget tree is updated by calculating the changed part based on an algorithm.

The Widgets layer is the base library provided by Flutter. On top of the base library, Flutter also provides libraries of Components in the Material and Cupertino visual styles.

Materail: Android-style widgets

Cupertino: IOS style Widget

Flutter Engine

Skia is an open source two-dimensional graphics library that provides a variety of commonly used apis and runs on a variety of hardware and software platforms. Google Chrome, Chrome OS, Android, Firefox, Firefox OS, and many other products use it as a graphics engine.

Skia is funded and managed by Google and can be used by anyone under the BSD freeware license. The Skia development team is committed to developing the core of Skia and has adopted a wide range of open source contributions to Skia.

Because the native UI and drawing framework are not used, the high-performance experience of Flutter is guaranteed.

The Flutter Engine is a pure C++ SDK that includes the Skia Engine, Dart runtime, text typography Engine, and more. It is simply a Dart runtime that can run Dart code in JIT, JIT Snapshot, or AOT mode. Dart: UI library Native Binding implementation is provided when code calls dart: UI library. Don’t forget, however, that this runtime also controls VSync signal passing, GPU data filling, and so on, and is responsible for passing client-side events to the code in the runtime.

The SDK source code

… /sdk/flutter/packages/flutter/lib

Flutter for Web

Dart can compile Dart code into Js code using the Dart 2JS compiler. Most native App elements can be implemented through DOM, and elements that can’t be implemented through DOM can be implemented through Canvas.

communication

Thanks to the Engine layer, Flutter doesn’t even use the mobile platform’s native controls. Instead, Flutter uses its own Engine to draw widgets (the display unit of Flutter). The Dart code is AOT compiled into platform native code. So the Flutter can communicate directly with the platform, without the need for a JS engine bridge. The only thing that Flutter requires is a canvas to draw the UI.

For Android, the C/C++ code for the Flutter engine is compiled by NDK, and for iOS, it is compiled by LLVM. The Dart code for both platforms is AOT compiled to native code, and the Flutter application runs using the native instruction set. Flutter builds iOS and Android apps simultaneously by using the same renderer, framework and set of widgets, without having to maintain two separate code libraries.

Communicate locally through platform channels.

Through a MethodChannel, Native can also invoke the methods of a Flutter, which is a two-way channel.

About the VSync

The Android display runs at around 60 frames per second, which is about 16.6ms per frame.

Stalling occurs when the drawing interval is larger than 16ms.

Graphics rendering

Rendering Pipeline

There is a Rendering pipline within the Flutter framework. The rendering pipeline is driven by the Vsync signal provided by the system. The Vsync signal is the familiar Android Vsync signal if your Flutter app is running on Android.

When the Vsync signal arrives, the Flutter framework will perform a series of actions in the sequence shown in the diagram: Animate, Build, Layout and Paint. Finally, a Scene will be generated and sent to the underlying layer to be drawn on the screen by the GPU.

Animate stage: Because the animation changes State with each Vsync signal, the Animate stage is the first stage of the pipeline.

Build in this phase of Flutter, widgets that need to be rebuilt in this phase will be rebuilt at this time. This is when the familiar statelessWidget.build () or state.build () is called.

In the Layout stage, the position and size of each display element will be determined. At this point is RenderObject performLayout () is called.

The Paint phase, when renderObject.paint () is called.

The above is a rough working process of the entire rendering pipeline.

The Flutter app only needs to trigger the rendering pipeline when the state changes. There is no need to re-render the page when your app is doing nothing. Therefore, Vsync signals need to be scheduled by the Flutter app. For example, we all know that state.setState () may be called when your page needs to be changed. This call to the Flutter framework will eventually initiate a request to schedule the Vsync signal to the underlying layer. The bottom layer then drives the rendering pipeline when the Vsync signal arrives and finally displays the new page on the screen.

Graphics Pipeline

A diagram of the Rendering mechanism of the Flutter framework.

The entire rendering pipeline runs in the UI thread, driven by Vsync signals, and outputs the Layer Tree after the frame is rendered.

The Layer tree is sent to engine, which schedules the Layer tree to GPU thread, synthesizes (compsite) layer tree in GPU thread, and then sends the layer tree to GPU for display after rendering by Skia 2D rendering engine.

Layer Tree is mentioned here because the final output of the rendering pipeline phase we are going to analyze is this layer tree.

So instead of just calling paint(), there’s a lot of layer Tree management involved.

Flutter only cares about providing view data to the GPU. The GPU’s VSync signal is synchronized to the UI thread. The UI thread uses Dart to construct an abstract view structure. This data is fed to the GPU via OpenGL or Vulkan.

Widget

In Flutter, most things are widgets. Widgets are immutable, support only one frame, and do not update directly on each frame. Updates must be made using the state of the widget. The core features of stateless and stateful widgets are the same; each frame they are rebuilt, with a State object that stores State data and restores it across frames. Skia is a 2D drawing engine library that is cross-platform and can be embedded into the iOS SDK of Flutter. This makes the Flutter Android SDK much smaller than the iOS SDK.

Widget lifecycle

A StatelessWidget cannot be changed, such as Icon, Text, etc.

If you don’t need to make any changes once your control is displayed, you should use the StatelessWidget.

A StatefulWidget is stateful and mutable.

It can change its appearance in response to user actions or changes in data.

For example: CheckBox, Switch..

State lifecycle