Author: Idle fish technology — pray for sunshine
1. Xianyu IM status quo
Xianyu IM framework was built from 2016 to 2017, during which many iterations and upgrades resulted in the accumulation of historical burdens. Later, IM interface was heavenized, resulting in more complex architecture. At the development level, xianyu’s current architecture mainly has the following problems:
• Low r&d efficiency: Current architecture development requirements involve the dual logic codes of Android/iOS and the UI code of Flutter. Locating problems can only be traced to Native logic vulnerabilities from the Flutter UI table. • Poor hierarchy of architecture: the hierarchy of architecture design is not clear, and the business logic is mixed in the core logic layer, resulting in high risk of code change; • Slightly poor performance test: core data sources store Native memory, which needs to be serialized onto the Flutter side by the Flutter Plugin. Poor performance in the case of a large number of data sources;
From the perspective of public opinion, the main problems of the current structure of Xianyu IM are summarized as follows:
• Difficulty in locating problems: Online public opinion feedback is very strange, and relevant scenes cannot be reproduced in the test, so we can only guess the essence by phenomena in many cases; • Many difficult problems: recurring problems caused by instability of architecture. The current difficult problems mainly include unread red dot count, iPhone5C low-end machine architecture, and multimedia sending and other problems; • Large differences in problems: There are large differences between the logic codes on both ends of Android and iOS, including the logic of existing buried points. Therefore, there are different root causes on both ends and different solutions to the problems when troubleshooting the root causes.
2. Cross-terminal solution
In order to solve the current IM pain point, Xianyu will start the IM architecture upgrade project this year, focusing on solving the pain point of the consistency between the two ends of the client. The initial plan is to realize the cross-terminal unified Android/iOS logical architecture. Currently, cross-end solutions in the industry can be initially classified into the architecture shown in the following figure. Cross-end solutions at the GUI level include Weex,ReactNative,H5, uni-APP, etc. Most of their memory models need to be connected to Native mode storage by bridging. At the logical level, there are roughly C/C++ and other virtual machine-independent languages to achieve cross-end, of course, assembly language is also feasible; Two other structures that are independent of this system are Flutter and KMM (Google implemented a similar Flutter architecture based on Kotlin), where Flutter runs a specific DartVM and mounts memory data into its own ISOLATE.
Considering that idle fish is the frontier explorer of Flutter, Flutter is preferred in the scheme. However, Flutter’s ISOLATE is more like the concept of a process (the underlying implementation does not use the process mode). Compared to Android, multiple threads running on Android’s Dalvik VM share a memory Heap, while DartVM’s ISOLATE their Heap. Therefore, communication between isolates is cumbersome (serialization and deserialization are required). The whole model is shown in the figure below:
If according to the official system Flutter mixed architecture application, open multiple FlutterAcitivty/FlutterController, underlying will generate multiple Engine, corresponding will have multiple isolate, and isolate communication (similar to a socket or AIDL) similar to the process communication, Here we refer to the design concept of Idle Fish FlutterBoost. The Architecture of FlutterIM shares engines of multiple pages, so the memory model naturally supports shared reading. The schematic diagram is as follows:
3. Design of Flutter IM architecture
3.1 Comparison of old and new architectures
The picture below is oneOld Architecture SolutionThe core problems mainly focus on the poor abstraction of Native logic, in which the design of multithreading concurrency at the logical level multiplies the problems, the interaction of Android/iOS/Flutter is complicated, the development and maintenance cost is high, the coupling of the core layer is serious, and there is no plug and pull concept.
Considering the historical architecture, the evolution is as followsNew Architecture Design:
The architecture consists of business layer, distribution layer, logical layer and data source layer from top to bottom. The data source layer comes from push or network request and is encapsulated in Native layer. The message protocol data is thrown up to the core logical layer on the Flutter side through the Flutter plug-in, and becomes the Enitity entity of the Flutter DB after processing. Some messaging protocol entities are mounted in the entity; In the core logic layer, the complex data is packaged flat and mounted to the session memory model data or message memory model data in the distribution layer, and finally distributed to the business logic through the subscription of observer mode. Flutter IM focuses on transforming the logical layer and the distribution layer, encapsulating and isolating the IM core logic and business-level data model. After the core logic layer interacts with the database, data is encapsulated into the moduleData of the distribution layer and distributed to the business-layer data model by subscription. In addition, DB is also a key dependency in IM model. I can fully encapsulate and solve DB database management to achieve a lightweight and high-performance Flutter DB management framework.
3.2 DB storage model
The DB storage of the Flutter IM architecture depends on the database pluginSqflite, its storage model is as follows:
3.3 ORM Database Scheme
IM architecture relies heavily on DB database, and currently there is no complete one in the industryDatabase ORM management schemeBased on Android’s OrmLite/GreenDao, I designed a Flutter ORM database management solution by myself. The core ideas are as follows:
3.4 IM memory data model
The in-memory data model of FlutterIM architecture is mainly divided into session and message granularity. The session in-memory data model is entrusted to SessionModuleData, and the message in-memory data model is entrusted to MessageModuleData. Session memory data has a root node RootNotice, which then mounts PSessionMessageNotice (PSessionMessageNotice is the ORM mapped session DB table model) child node set; Message memory data is managed by a MessageConatiner container, which internally mounts a collection of pMessages (orM-mapped message DB table model) messages in this session;
According to the previous chapter, PSessionMessageNotice designed an OrmEnitity Cache. Considering that the number of IM sessions is limited, PSessionMessageNotice is directly cached in the Cache. The advantage of this method is that the same object is stored in the cache when the session data element is fetched from all places, which ensures the consistency of data read and written repeatedly. PSessionMessageNotice is mounted to MessageContainer’s memory management in consideration of its infinite number of particulates. When quitting the session, the number of PMessage sets in the container will be checked. Appropriate reduction can reduce the memory overhead, as shown in the following figure:
3.5 Status Management Scheme
The Flutter IM state management scheme is relatively simple. It uses the observer mode subscription distribution for the data source Session/Message dimension. Its architecture is similar to EventBus mode. ScopeModel or provider has little impact, and it is more important to retain a plug-and-pull abstraction at the core. The architecture is shown below:
3.6 IM Synchronization Model Solution
The following is the current status of the message synchronization model, there are multi-threaded concurrent scenarios such as ACCS Thread/Main Thread/Region Thread in the model, which leads to the problem of high multi-threaded concurrency. The isolation scheme of synchronization between push and network request of Native adopts the locking mechanism of Lock, and is processed by queue downfrequencing, which is cumbersome and error-prone. On the whole, the Region Version Gap is used to determine whether there is a Region void, and then the Region synchronization is performed to supplement data.
4. Progress and performance comparison
• For architecture: In FlutterIM architecture, the dual-end logical differences are unified into the same Dart code, completely eliminating the problems caused by Android/iOS code differences, reducing half of the cost of development and maintenance, test regression and visual acceptance, and greatly improving r&d efficiency; The architecture is reconstructed and layered to achieve a decoupled, pluggable IM architecture. At the same time, a large amount of data Native to the Flutter side is thrown into the serialization process, which transforms the Flutter reference transmission to solve the private chat lag problem in the ultimate test scenarios. • For online public opinion: the group log of UT and TLog can be tracked and checked; In addition, special solutions were focused on many existing difficult and complicated diseases. For example, the architecture of iphone5C was uniformly planned on Flutter side, and the counting of unread red dots and other problems were also fixed in the upgrade of the architecture model. In addition, the multimedia audio and video transmission module was reformed and upgraded. • Performance data comparison: When the logical layer and UI layer of IM architecture are switched to Flutter, the overall memory water level is unchanged compared with the original architecture mode. In private chat scenario, the memory of Mi9 test structure is reduced by 40M, power consumption is reduced by 4mah, and CPU is reduced by 1%; In the extreme test scenario, the memory data of the new architecture is significantly improved compared with that of the old architecture, mainly because the overhead of page switching is greatly reduced in the Flutter scenario where both interfaces use Flutter.
5. Look forward to
JS cross-ends are not safe, C++ cross-ends are a bit expensive, Flutter would be a better choice; At that time, the fundamental purpose of the architecture upgrade of Xianfish FlutterIM was never to Flutter because of Flutter, but we had to explore new schemes due to the heavy historical burden, high maintenance cost at the code level, poor scalability of new business, uncoordinated manpower ratio, and continuous public opinion feedback of difficult and complicated diseases. Through super complex business scenarios of Xianyu IM to verify the logical cross-terminal feasibility of Flutter mode, Xianyu will always keep frontier exploration on the road of Flutter and finally feed back to the ecosystem. To sum up, exploration is about taking the first step, and the surprises will follow.