Flow of data, using RxJS to construct the data logic of complex single page applications

Xu Fei, front-end architect of Teambition, has been working on enterprise application front-end architecture for a long time, and has in-depth research on single-page application implementation solutions. There are some differences between this field and the traditional front end of the Internet, which values the data layer and requires more componentization. Teambition is a team collaboration tool with high requirements for real-time data.

RxJS profile

RxJS is the ReactiveX library for JavaScript. In general, it describes the relationship between data changes and time and is a Lodash that can be used for asynchronous operations.

This sentence is difficult to understand and can be understood by looking at the situations in which it is used.

We often see these scenes:

List page of weibo
Task Kanban for various collaboration tools, such as Teambition

Review images

A common feature of these scenarios is:

It consists of a number of small squares
Each cube needs to be dominated by a business entity (a tweet, a task) and aggregate some other associated information (participants, tags, etc.)

For such an interface, given its full display, there are two possible scenarios:

The server renders, queries all the data, generates HTML and sends it to the browser
Front-end rendering, which queries all data and sends it to the browser to generate HTML display

Microblogging uses the former one with bigpipe mechanism to generate interface while Teambition uses the latter one. The main difference is due to the product form.

Teambition business scenario

Complex single page applications
The main functions are all in one page
Multiple copies of the same data exist
All services have real-time data synchronization with the server

Business problems

The same data is used in multiple places in the view
Extensive use of local data caching
Each business scenario has a server-side WebSocket push to update data
WebSocket push can cause many views to be updated

Let’s start with an example. We now have projects and tasks. The task is hung under the project. If you need to query A task, you should first query the project (A) to which it belongs, then switch to another project (B) and click on one of the tasks. At this time, you can switch back. We can know that we have requested the data of project A just now. Do we need to request it again when we switch back now? You don’t have to, but what’s the problem if you don’t ask? After all, I do not know whether project A was modified in the process just now.

Therefore, our data query is required to be discretized. Task information and additional associated information are separately queried, and then assembled by the front end. In this way, on the one hand, the amount of transmitted data can be reduced; on the other hand, the relationship between data can be analyzed and easily tracked when updating.

In addition, the operation of Teambition uses WebSocket to push updates in the whole business dimension. For example, when something changes in the current task kanpan (someone else creates a task, modifies a field), the server pushes a message to push the front end to update the interface.

Discrete data would require caching. For example, once the interface is set up, if someone creates a task on the other side, the local kanban simply receives the task information and creates a view, without having to query for people, tags, and other associated information because it has already been retrieved. So, it would look something like this:

The display of a view component needs to aggregate three entities ABC, among which, if any entity exists in the cache, the server does not pull the entity without cache, but only the entity without cache.

This process presents the first challenge:

Querying the same kind of data can be synchronous (fetched in cache) or asynchronous (fetched in AJAX), and business code needs to consider both cases.

WebSocket push is used to ensure the correctness of the front-end cache. However, it is important to note that WebSocket programming is different from AJAX. WebSocket is a subscription that is difficult to integrate with the main process, whereas AJAX can be organized and included in the main process.

For example, for different originators of the same update (they modify one thing, others modify this thing), the two kinds of follow-up is actually the same, but the code is not the same, need to write two copies of the business code.

Which brings us to the second challenge:

Retrieving data and notification of updates to data are written differently, adding complexity to business code.

Data is so discrete that, from the perspective of views, the data required by each view may be combined with a relatively long and complex combination to meet the needs of presentation.

So, the third challenge:

Each render data is combined by several query processes (combined synchronous asynchronism as mentioned earlier). How do you clearly define this combination relationship?

In addition, you may face scenarios like this:

After a set of data passes multiple rules (filtering, sorting), new data needs to be inserted (a new one is actively added, WebSocket pushes a new one created by others). These new data cannot be added directly, but must also go through these rules, and then merged into the result.

This brings us to the fourth challenge:

How to simplify code complexity by applying the same rules to existing and future data.

Start today’s thought process with these questions in mind.

1. Centralization of data cache

The first point is to centralize the data cache. In the front end, the cache of all data is treated as one center, but there are two situations:

The cache may not exist when the upper view is first called. After requesting the data from the server, the data is disassembled and cached, which is an asynchronous loading process.
In the presence of a cache, data is directly assembled from the cache to the interface display, which is a synchronous process.

Then the problem comes, the same request for a project data, may be synchronous, may be asynchronous, how to write the code, because synchronous and asynchronous code writing is different. Another problem is that WebSocket messages are merged directly into the cache. How does that affect the previous situation?

1. Synchronization and asynchrony

On the front end, you often encounter synchronization and asynchronous code unification. Suppose you want to implement a method that returns a value when it is available and otherwise retrieves it from the server.

A common approach is to use promises:

function getDataP() {
  if (a) {
    return Promise.resolve(a)
  } else {
    return AJAX.get('a')
  }
}
Copy the code

So, the way to handle this is, if you’re not sure whether it’s synchronous or asynchronous, take asynchronous, because it’s synchronous compatible. Resolve is to force a synchronous Promise to be an asynchronous compatible Promise.

Promises alone work, of course, but an Observable in RxJS can do the same:

function getDataO() {
  if (a) {
    return Observable.of(a)
  } else {
    return Observable.fromPromise(AJAX.get('a'))
  }
}
Copy the code

2. The difference from Promise

Somebody said, you know, this code isn’t as good as Promise, because you still have to roll over from it, so what’s the advantage?

Let’s take a look at the encapsulated methods. How are they used?

GetDataP ().then(data => {// Promise returns only one value, Console.log (data)}) getDataO().subscribe(data => {// Observable can have multiple returns, console.log(data)})Copy the code

In this section, instead of comparing the advantages of the two, we will only look at how to solve the problem:

GetData (): T{}, can only do synchronization things
GetDataP (): Promise {} can do both synchronous and asynchronous things
GetDataO (): Observable {}, which does both synchronous and asynchronous things

The conclusion is that both promises and Observables can encapsulate synchronously and asynchronously.

Reactive APIS for the data layer

Mainly to complete the following tasks:

To get the data
Subscribe and keep responding
Changes to data are pushed to associated subscriptions

Alignment of fetch and subscription

Typically, we implement things like custom events on the front end using the observer or subscribe publishing model, which is essentially a subscription.

From a view point of view, what it actually faces is:

I got a new mission data and I'm going to show it

It doesn’t matter how it got it, whether it was actively queried or pushed by someone else. That’s not its job. It just displays. So, it encapsulates two things:

Actively queried data
Passively pushed data

And then, it becomes something like this:

On ('task', data => {// render console.log(data)})Copy the code

This way, the view can handle two different sources of data in the same way, and the service can unify the two by firing the custom event task in its own callback.

But what we seem to be missing is that in addition to responding to such events, the view also needs to actively trigger the initial query request:

On ('task', data => {// render console.log(data)}) service.getData() Add this sentence to actively trigger the request aloneCopy the code

This still looks awkward, but back to the Observable example from the previous section:

getDataO().subscribe(data => {
  // render
  console.log(data)
})
Copy the code

Using RxJS, you can call the method directly and then immediately subscribe to the result of the method. This code does two things, first requesting data and then subscribing to data. In other words, if this data changes later, it will continue to get the changed data as the program executes. That seems to have done everything we’ve asked for. One way to think about it is this:

GetDataO is a business process
The resulting data of business processes can be subscribed

This way, you can merge the fetch and subscribe things together, making the focus of the view layer much easier.

3. State control of components

1. Streaming encapsulation of data

Because the React began to popular over the past two years, so people began to pay attention to the relationship between the view and its state – a state of time may be through the first data with the combination of some sort of relationship between the second data, and then to combination with another data, the result was a data, take the data to the view on display, Basically, this is the relationship.

Based on the ideas in the previous section, you can abstract the query process and the WebSocket response process into one. It’s easy to say, but if you look at the implementation, you’ll see that there are many steps involved. For example:

data1      data2      data3
  |          |          |
  ------------          |
        |               |
        -----------------
                |
              stateCopy the code

The data required for a view might look like this:

Data1 and data2, by some combination, get a result
This result is then combined with data3 to get the final result

How do you abstract this process?

Note that data1, data2, and data3, all of which may have been mentioned earlier, contain synchronization and asynchronous encapsulation processes, specifically, an RxJS Observable.

Each Observable can be thought of as a pipeline of data streams. All we need to do is assemble the pipelines based on their relationships, so that data comes in at one end of the pipeline and gets the final result at the end, and the data flows through the pipeline.

const A$ = Observable.interval(1000)
const B$ = Observable.of(3)
const C$ = Observable.from([5, 6, 7])Copy the code

2. Combinable data channels

RxJS gives us a bunch of operators to handle relationships between observables, for example, pipes can be connected in some way (operators)

const D$ = C$.toArray()
  .map(arr => arr.reduce((a, b) => a + b), 0)
const E$ = Observable.combineLatest(A$, B$, D$)
   .map(arr => arr.reduce((a, b) => a + b), 0)Copy the code

The above D is the data pipeline obtained through A transformation of C, while E is the data pipeline obtained after assembling A, B and D

A ------> |
B ------> | -> E
C -> D -> |
Copy the code

From the diagram above, we can see the composition relationship between them. In this way, we can describe the composition relationship of business logic, encapsulate each small-grained business into a data pipeline, and then assemble them to assemble the whole logic. So the data that you end up with is always the data that you combine through this relationship.

3. Unity of present and future

In business development, we often encounter a scenario like this:

Add a new piece of data to the filtered sorted list, follow this rule again.

I use a simple analogy to describe this:

Every student who comes into the classroom can get a candy

This sentence conveys two meanings:

Before that claim is made, everyone already in the room should be given a candy
After this assertion is made, everyone who enters the classroom again deserves a candy

Here, the first sentence expresses the present, the second sentence expresses the future. When we write business programs, we often think of the present and the future separately, ignoring the deep congruence that exists between them.

After thinking about this, and thinking about the problem in reverse, one can come to the conclusion:

All data entering this list should go through some filtering and some sorting

This is a proper business abstraction, and then code:

const final$ = source$.map(filterA).map(sorterA)
Copy the code

Where source represents the source and final represents the result. Source after filterA transformation, sorterA transformation, get the result.

Then, consider the definition of source:

const source$ = start$.merge(patch$)
Copy the code

The source is the combination of the initial data and the new data.

Then, implementing filterA and sorterA completes the abstract definition of the entire business logic. Define start and patch separately. For example, if start is a query and patch is a push, it will be run. Finally, add a subscription to Final, and the whole process maps perfectly to the interface.

A lot of times, we write code with appropriate abstractions in mind, but the two words don’t mean the same thing in many scenarios.

Many people think that dividing code into methods, types, and components abstracts the entire business process, but it doesn’t.

The abstraction of business logic is the way in which it differs from the business unit, the blood and nerves of the former, the limbs and organs of the latter, which need to be brought together to make a living whole.

General scenarios, business unit of abstract difficulty is relatively low, it is easy to understand, also easy to gain attention, so can usually do pretty well, such as the last two years, topics, such as modular are able to talk about, but for the business logic of abstract, most of the project is done very not enough, is worth thinking about.

4. Data and view binding

What’s interesting about using RxJS to organize data capture and change encapsulation from a business logic perspective? Ultimately, these things need to be reflected in views.

There’s been a lot of new stuff coming out of the view layer in recent years, which isn’t like jQuery manipulating arrays directly. Examples include Angular, React, and vue.js. From these mainstream frameworks, one idea emerges – MDV (Model-driven View). That is, everything is changed after the data, the view layer framework itself according to the defined rules to change the view. You don’t need to manually change the view every time you modify the data. So under this concept, all changes to the view should first be changes to the model, and then automatically synchronized through the mapping between the model and the view.

But think about it a little bit, what exactly is driving the view change? Different frameworks use different approaches. Angular, vue.js, and React use different approaches.

Angular compares the current data with the previous data after each asynchronous event, and if not, redraws the previous view.
Vue.js makes the data get and set do one thing, such as a.b = 1, which internally monitors the immediate assignment of a.b and updates the interface when assigning to a.B
React is a thing that converges back, because it’s one-way data, which means you output a piece of data, redirect it somewhere, and then converge back, and change the view that way.

If you have a situation where a Property is Computed depending on something else, for example, you have three input fields on the interface, and the value of the third input field is always equal to the sum of the first and the second, if you define a Computed Property on vue.js, A property is calculated depending on some other data. The calculation is synchronous, and if the data is passed in every few seconds, the relationship is hard to define. Since properties can only be evaluated with current data and not asynchronously, we can use the example above, which is to define a stream and then assign values from that stream to it.

Again, consider the purpose of these front-end MV* frameworks: to define the relationship between data and views, and to automatically update views as data changes. If the following data changes are implemented with RxJS, that is, become a data pipeline, data flow in the pipeline, the upper layer only need to subscribe to the data, and then update the interface state.

1. Combination of data flow and view

React or vue. js: set the data to state or data manually. Angular makes it even easier to bind an Observable directly to the view with an Async pipe. In these systems, using an RxJS Observable is very simple:

Data $.subscribe(data => {// React or Vue, set this to state or data manually // Angular 2 does not need to use this step, Observable async pipe directly binds Observable to view with Async pipe. })Copy the code

Along the way, you may need to define the relationship in several ways, such as templates in Angular and Vue, JSX in React, and so on.

Here are a few points to make:

Angular2 uses RxJS very easily, like: Let todo of todos $| async this code, can be directly binding a observables to view, automatically subscribe and destroyed, more simple and elegant way to solve the “wait for the data”, “data is not null,” “data results is empty” differences of these three status. A similar effect can be achieved with plug-ins for Vue.

CycleJS is unique in that the entire process is based on rXJs-like mechanisms, including views. See the official Demo:

import {run} from '@cycle/xstream-run';
import {div, label, input, hr, h1, makeDOMDriver} from '@cycle/dom';

function main(sources) {
  const sinks = {
    DOM: sources.DOM.select('.field').events('input')
      .map(ev => ev.target.value)
      .startWith('')
      .map(name =>
        div([
          label('Name:'),
          input('.field', {attrs: {type: 'text'}}),
          hr(),
          h1('Hello ' + name),
        ])
      )
  };
  return sinks;
}

run(main, { DOM: makeDOMDriver('#app-container') });
Copy the code

In this case, notice the DOM. Select section. In this case, you’re obviously starting to select before the interface even exists, and you’re starting to add event listeners, which is the predefined rule THAT I mentioned, to unify the present and the future; If the interface has.field, add listeners immediately; if not, add listeners as soon as they do.

2. Overall state

The structure of the application is shown below, with the server at the bottom, and a data cache at the front. On top of that cache is a Reactive API. Note that the API is not called once, but constantly returns data and then cuts it to the view.

Review images

So, what else can you think about RxJS from a view perspective?

Asynchronous computing properties can be implemented.

It was briefly mentioned in the data association calculation last time. In fact, the whole article is foreshadowing this article.

Have you considered how to organize these data flows from a view point of view?

An analysis process might look like this:

Review a view and find that it requires data A, B, and C
Their sources are defined as data streams A, B, and C respectively
By analyzing the sources of A, B and C, it is found that A comes from D and E. B comes from E and F; C comes from G
Define these sources separately and merge the same parts, resulting in multiple pipe flows that go straight to the view
Then define the composition process of these pipeline flows, making appropriate abstractions

5. How to understand the whole mechanism?

To understand this set of mechanisms, imagine this picture:

Review images

Think of Teambition SDK as a CPU. The API is the pins it provides externally. View components are connected to these pins. See the SDK design documentation for details.

Also, see this article for the RxJS data flow combination. You might click on the link and wonder: What’s the relationship between the two?

If you flip to the last figure and see multiple waves stacked on top of each other from the side, imagine that the state of a view can be understood as a stream on a timeline, which can be viewed as a superposition of several other streams. When these streams are stacked together, the values at the current moment represent all the state data of the view.

Doesn’t it make a lot of sense to think about it that way?

Teambition SDK

The new data layer of Teambition is built with RxJS, independent of any presentation framework, and can be used by any presentation framework, even in NodeJS. It provides a full set of API of Reactive. You can check the documentation and code to learn the detailed implementation mechanism.

With this mechanic, it is easy to implement a standalone view based on the Teambition platform, which allows third-party developers to build interesting things with their imaginations. We’ll also add some examples step by step.

Six, the summary

With RxJS, you can:

The unification of synchronous and asynchronous
Unification of fetch and subscription
The unity of present and future
Composable data change process
Precise binding of data to view
Automatic recalculation after a condition change

Flow of data, using RxJS to construct the data logic of complex single page applications

1. Centralization of data cache

Reactive APIS for the data layer

3. State control of components

4. Data and view binding

5. How to understand the whole mechanism?

Six, the summary

Related Posts

Jsonp – Reverse proxy -CORS personal summary of JS cross-domain problems

A large number of pictures on the Web side at the same time loading the problem of the optimization of the trip

How to make your GitHub project look professional