Following the “mPaaS Server Core Components” series, this article focuses on the architecture of Mobile Sync Service. MSS is one of the core basic service components of mobile development platform mPaaS, which is derived from ant Financial Group’s full-link solution for mobile applications to push massive data from server to client.
Please refer to the recommendations at the end of this article.
Core concept interpretation: Mobile synchronization service MSS
The core concepts of MSS are:
- Through a secure data channel TCP+SSL, the server-side business data can be synchronized to the client App in a timely, accurate and orderly manner, which can be defined as a reliable message middleware between the client and the server.
Traditional RPC has been based on the Internet industry for decades and can meet most of the business scenarios and functional requirements. However, with the comprehensive popularization and sublimation of mobile Internet, both the scale of App and users’ requirements for real-time response of App have entered a new stage.
Traditional request response has many shortcomings due to the characteristics of RPC itself, typical examples are:
-
The client App needs to call RPC request to obtain the latest data in specific scenarios, while the server (cloud) actually has no or only a small amount of data changes;
-
When the client is started, different service modules and service functions are independent in design, so they need to make RPC requests respectively to complete the data pulling of their respective services.
-
When service data changes on the server (cloud), the client cannot sense the changes in real time or can only poll the RPC interface periodically to check the changes.
-
From the technical point of view, traditional RPC mostly carries out data exchange based on HTTP (S) short connection. Even if the keepalive feature is used, the connection cannot be maintained for a long time and the link cannot be continuously reused. The request for connection creation, certificate exchange, encryption and decryption will bring great loss to the network time and performance.
Therefore, in order to solve or optimize these problems, MSS came into being and its core features are as follows:
-
Reliable synchronization: Reliability refers to Service scenarios where the Quality of Service (QoS) level required by services is mandatory. SYNC ensures that as long as users are active within the data validity period and match the requirements for Service push (such as the client version number and operating system type), The client must synchronize the data pushed by the business.
-
Incremental order: MSS ensures that the order of messages arriving at the client in the same channel is the same as the order in which the business server invokes the MSS server, and that all messages are incrementally synchronized to the client.
-
High real-time: When the network connected to the client is in good condition, the real-time performance of MSS push is very high, and the time of message push is almost the time of pure network transmission (delivery within 1s). When the network connected by the client is interfered by irresistible factors such as backbone network fluctuation, router failure, weak base station signal, and full client storage, data synchronization will be carried out after the TCP long connection is rebuilt.
Fundamentals of MSS for mobile synchronization service
Similar to MySQL’s binlog mechanism, the basic data unit transferred between MSS server and client SDK is Oplog. When a business needs to synchronize a changed data to a specified user or device, the business invokes the MSS interface. The MSS server wraps the data changes that the business needs to synchronize as an Oplog persistence to the database, and then pushes the Oplog to the client when the client is online. Each Oplog has a unique Oplog ID, which is guaranteed to be unique and monotonically increasing within a certain range of users and services (in order of invocation). The MSS server pushes each Oplog to the client in ascending order of Oplog ids.
Both the MSS server and the client record the maximum Oplog ID that the client has received, known as the synchronization point (also known as the data version number).
- Typical scenario 1: Backlog data is synchronized during the establishment of a long-term connection
Assume that the client has three businesses: Biz1,Biz2,Biz3. Their respective synchronization points are: 8,17,21.
The MSS server checks each of the three services for oplog backlogs after the synchronization point. As shown in the figure, Biz1 has two Oplog backlogs whose ids are 23 and 26, Biz2 has three Oplog backlogs whose ids are 24,25 and 29, and Biz3 has one Oplog backlogs whose IDS are 30.
The server pushes the service backlog data detected in the previous step to the client.
After receiving the data, the client reports the largest Oplog ID of each service to the server. The server confirms that the client has received the data. At this point, the synchronization points of Biz1,Biz2,Biz3 are 26,29,30 respectively. The server verifies that there is no backlog and the synchronization reaches a stable state for the time being. Meanwhile, on the client side, the MSS SDK notifies Biz1,Biz2, and Biz3 of the new data it receives.
- Scenario 2: When the client is online, data is pushed in real time
Assuming that the client is online and the TCP long connection is still alive, Biz3 business server requests to synchronize a data to the client, MSS server gives it oplog number 35, and persists to the database first.
The server detects that the long connection between the client and the server is online and pushes oplog 35 to the client.
After receiving oplog 35, the client sends an ACK reply to the server to report that oplog 35 is received. At this point synchronization temporarily reaches a stable state again. Meanwhile, on the client side, the SYNC SDK notifies the Biz3 business of the new data it receives.
MSS architecture advantages of mobile synchronization service
In terms of its characteristics and principles, MSS has unique advantages:
-
Incremental push: Incremental push of service data, which effectively reduces the transmission of redundant data and greatly reduces network transmission costs.
-
Merge push: When the client is successfully initialized, the server can push multiple service data at a time, avoiding RPC requests made by different service modules.
-
Request reduction: When the server has no incremental service data, it will not consume any client RPC request cost and reduce redundant RPCS of services.
-
Improved timeliness: When data changes on the server, data can be directly pushed to the client in the shortest time, without waiting for the client to respond to RPC requests.
-
Improved experience: Data is pushed without users’ direct perception, and the data is in place before rendering the client interface, reducing users’ waiting time.
Mobile synchronization Service MSS architecture analysis
- Business &MSS& Client:
Business services only need to interact with the MSS server. After the interface is successfully invoked, the subsequent data synchronization process is fully taken over by MSS, and the cost of business system access is very low.
There is also a similar mechanism between the MSS client SDK and the business module to ensure that the data is reliable and unique, and that the business module can be received.
- MSS and Access gateway:
The Ant Unified Access Gateway (Accgw) was mentioned in a previous article:
Accgw is responsible for TLS uninstallation, configuration control, dynamic UPSTREAM routing, MMTP protocol parsing, packet compression, connection retention, traffic control, and load balancing. MSS (SYNC) is one of the UPSTREAM routes. Accgw’s superpowers are fully applied by MSS (SYNC).
- MSS and MMTP&HTTP 2:
MMTP, short for Mayi Mobile Transport Protocol, is a private Protocol developed based on TCP. The protocol divides network packets into two types of frames: instruction frames and data frames.
The command frame mainly realizes the internal control of the network system, including intelligent heartbeat designed to reduce the power consumption of the device, control command frame for convenient link scheduling, client network configuration frame, etc. Data frames are used to transmit real business data. MMTP has minimal data size, high security, high scalability and extreme performance. MSS (SYNC) data transmission is also based on MMTP.
In addition, to accommodate industry standards, MSS also supports HTTP2 links based on the reuse of MMTP data protocols.
conclusion
MSS (SYNC) has been used in several super apps in Ant Group, including Alipay client, Ant Fortune, Hong Kong version of Alipay, Word-of-mouth independent client, word-of-mouth merchant client and so on. It has also been used in several international sites of Ant International, such as Indonesia, Philippines, Malaysia and Macau Star Exchange Bank. The business scope supported by ant has almost covered all sectors, from payment, social contact, marketing, intelligent customer service to wealth, credit, insurance, and then to intelligent release, intelligent operation and maintenance.
SYNC, meanwhile, was also involved in the group for many years in a double tenth, promoting activities, 12-12, big New Year red envelopes, after years of optimization and refining, has so far had the class online, online ability to push millions of TPS, within the group, push message volume reached billions of level of large scale, has become indispensable to group App core foundation ability, It is also a blow to the core component of mPaaS server.
| mobile development platform blockbuster online ant gold mPaaS three components in open platform:
- Open beta period, welcome to experience the trial. Alipay scan code orLogging in to the Open Platform
Past reading
The opening | ant gold clothes mPaaS core components of a service system overview”
Summary of Ant Financial mPaaS Server Core Component System: Mobile API Gateway MGS
Core Components of Ant Financial mPaaS Server: Analysis of Mobile End-to-end Network Access Architecture under Hundred-million-level Concurrency
Core Components of mPaaS Server: Architecture and Process Design of MPS for Message Push
MPaaS Core Components: How does Alipay build public Opinion Analysis System for Mobile Terminal Products?
“MPaaS Server Core Components: Mobile Analysis Service MAS Architecture Analysis”
Follow our official account for first-hand mPaaS technology practices
Nail Group: Search group number “23124039” by nail group
Looking forward to your joining us