1. Overview of Realtime DB

1.1 Introduction to RealTime DB



RealTime DB is a database hosted in the cloud, where data is stored in JSON format and synchronized in Realtime to each client it connects to. It has the following characteristics:

  • Instead of using the usual HTTP requests, you use data synchronization. Whenever data changes, any connected device receives updates in real time
  • Provides a flexible expression-based rules language that allows users to customize data structures and when data can be read or written
  • NoSQL database based on MongoDB, so it has different optimization direction and functional characteristics from the relational database. The server-side API is designed to support only operations that can be performed quickly, so users need to think carefully about the data structure they store.

1.2 Generation background and application scenarios of RealTime DB

The development of cloud native APP based on BaaS needs a set of database middleware for information storage, real-time synchronization, atomic modification, offline caching, and remote modification of the Serverless database to achieve the purpose of disconnecting from the server interface. In order to meet the above requirements, a middleware such as RealTime DB is designed and implemented. Usually used in the following scenarios:

  • Instant messaging
  • State synchronization
  • Real-time dynamic
  • Team collaboration
  • other

2. Development history of Realtime DB technology



• Local DB: Data Local persistence advantage: stable and reliable, Local persistence disadvantage: can not share data multi-terminal, low playability

• Server Interface: data persistence through Server Interface communication advantages: multi-terminal data sharing, cloud controllable disadvantages: Interface needs customization, usually requires client + Server + operation and maintenance intervention, can not automatically scale and expand capacity, more dependent on network availability, etc

• Remote DB: Remote real-time database, data cloud persistence Advantages: Multi-end data sharing, real-time synchronization, high playability, Serverless, end-to-side free customization of data structure, no need to deploy and maintain servers Disadvantages: Relying on network availability

3. Research status of RealTime DB

3.1 Industry Analysis

Firebase, a real-time back-end database startup acquired by Google in 2014, allows users to combine Firebase with Google’s cloud services in a more convenient way. Firebase currently offers two Cloud database solutions that support real-time data synchronization and can be accessed through the client, Live Database and Cloud Firestore.

product The principle of advantages disadvantages
Real-time database Store the data as a large JSON tree • Simple data is easy to store

• Support online status detection		 • Complex hierarchical data is difficult to organize on a large scale

• The query and sorting function is weak
Cloud Firestore Stores data as a collection of documents • Simple data is easily stored in a document, much like JSON

• Complex hierarchical data is easier to organize on a large scale by using subcollections in documents

• Less need for de-normalization and data flattening		 It’s complicated to use

3.2 Functional requirements and challenges

Preliminary planning Realtime DB to meet the following needs,

  • Add, delete, change and check statement support
  • Multi-terminal real-time synchronization data support
  • Public or private cloud implementation
  • Offline Cache Support
  • Automatic scaling and expansion of data, Serverless
  • Data security means
  • Custom add delete check change protocol, later extensible function

1. Cloud database selection to realize Serverless relational database VS non-relational database; based on the company’s self-developed Serverless cloud MongoDB, to realize server-side cloud database storage

2. The NoSQL database is adopted in the multi-terminal real-time synchronization scheme, and the data is stored in the form of multi-tree (K-V), namely JSON. The problem of node data merging is not concerned, and the data can be covered by the server based on the operation time

3. Data security means: data node defines read and write permissions, and the cloud is empowered [according to the account] to generate permission configuration table

Transaction support and atomic operations custom remote database operation protocol, reserved operator “. , for later implementation of atomic operations and other instructions

5. The real-time guarantee of the database adopts long connection, based on MDP self-developed long connection SDK-TapConnect, to realize stable sending and receiving and millisecond-level response

3.3 Solutions

The client-side architecture design can be roughly divided into the following four layers

Data flattening simplifies JSON



Leaf node basic data types: Boolean String/Long/Double

Key :must not contain: ‘/’ ‘. ‘ ‘#’ ‘$” [‘] ‘

Value limit :cannot be: ‘NaN’ ‘Inf’ ‘-inf’

Client-server communication model



Multi-terminal synchronization scheme



Multi-terminal adaptation scheme

The development of Realtime DB needs to adapt to Android, IOS, Flutter and other platforms

Option 1: Develop RealTime DB SDK separately for Android, iOS, Flutter, etc. Advantage: Native level performance disadvantage: high maintenance cost

Advantages: develop a set of code, which can be used by multiple terminals. Disadvantages: obvious performance problem, additional layer of Native communication, and inconvenient to use. API adaptation is required for each end

Advantage of scheme 3 is that it adopts Kotlin Multiplatform framework for development: a set of code, running multi-terminal, and the code is compiled and processed by the framework, automatically compiling multi-terminal products, naturally supporting the performance of the native level, adopting Kotlin, and easy to write

Cons: Currently the framework is in alpha, there are unknown risks

Write once, adapt to the multi-terminal Common Code: use Kotlin to realize the Common logic, can rely on the official existing multiplatform support library, network, IO, database and other operations

IOS Code: you can rely on the existing IOS native library through CocoaPods to call the principle: during the framework compilation, it automatically generates the corresponding KT class function according to the external interface of Objective-C/Swift and provides external call

Android Code: You can rely on existing JAR/AAR libraries through Gradle. The principle of calling: interoperability between Kotlin and Java

If there is platform-differentiated Code that cannot be implemented directly in Common Code, you can implement Actual class A by defining the Expect interface class A in Common Code, and Actual class A in iOS Code and Android Code, respectively

3.4 Prospects for Realtime DB in the future

The Realtime DB SDK is built into a middleware that covers iOS, Android, Flutter and even JS. The Kotlin Multiplatform Mobile framework is adopted to realize a set of codes and run everywhere for the purpose of bringing continuous gain in the later stage.

4. To summarize

RealTime DB allows secure access to the database directly from the client code. With this, you can build feature-rich, collaborative applications where data remains permanently on the client and real-time events continue to be triggered even when offline, providing a good real-time experience for the end user. Realtime DB is still in the research and development stage, for the program selection and implementation of the shortcomings, welcome to communicate or put forward optimization opinions or suggestions.

Author’s brief introduction

Yijun Oppo application engineers focus on mobile platform development, real-time database, Severless, QUIC and other technologies.

For more exciting content, please scan the code to pay attention to [OPPO Internet Technology] public number