Creditease officially opened the nextSystem4 (hereinafter referred to as “NS4”) series modules on March 29, 2019.

This open source NS4 series module is a distributed business system solution generated by the cumbersome current payment system, high code coupling degree and high maintenance cost. NS4 series frameworks allow the creation of complex processes/business flows, and the implementation of business service nodes can be connected in series and distributed. It is simple and lightweight, realizing “container free” (independent of tomcat, Jetty and other containers). The design concept of NS4 series framework is to separate business and logic, and developers can achieve a business system with complex logic, efficient performance and stable function only through simple configuration and business implementation.

NS4 series includes 4 open source modules, which are: Ns4_frame Distributed service framework, NS4_gear_idgen ID generator component (NS4 framework Demo example), and NS4_gear_watchdog Monitoring system components (service daemon, application performance monitoring, data acquisition, automatic alarm system) and NS4_chatbot communication components.

The core advantages of NS4 series modules are mainly reflected in the following aspects:

  • Very scalable, can be gracefully expanded and degraded;

  • Centralized management, the message of each node for centralized management and distribution;

  • Easy to maintain, the complex process business is divided into multiple module systems for interaction, reduce code coupling;

  • Perfect call link, for the link complex system can accurately locate the fault link.

  • Real-time automatic reminders can be achieved through online chat bots.

Open source project address: github.com/newsettle

A, ns4_frame

Open source: github.com/newsettle/n…

Ns4_frame is a high-performance distributed service framework that allows the creation of complex processes/business flows. The implementation of business service nodes can be connected in series and distributed. It is simple and lightweight, realizing “container free” (independent of tomcat, Jetty and other containers). Ns4_frame separates business and logic, enabling developers to achieve a business system with complex logic, efficient performance and stable functions through simple configuration and service implementation.

The project structure

Ns4_frame is a set of MAVEN parent-child projects, consisting of five sub-projects:

  • NS_MQ: Is responsible for communicating with the underlying message queue and provides an API to operate on the message queue. At present, NS4 supports REDis as a message-oriented middleware and provides a common interface that can be extended to a variety of message-oriented middleware. The operation of message-oriented middleware is encapsulated in NS_MQ project.

  • NS_TRANSPORTER: Collects, processes, and forwards service messages by invoking apis provided by NS_MQ. It is essentially a set of message receiving and receiving processing framework, mainly responsible for receiving the message back to the business code, and send the message to the business layer for processing, when the business layer is finished processing, then return the processed message to Redis.

  • NS_CHAIN: An optional development framework responsible for chain integration of business process steps within the same JVM into the business process flow of the current business module.

  • NS_CONTROLLER: a service message forwarding application that forwards received messages to corresponding service modules for processing and associates service modules based on the overall service. NS_CONTROLLER is essentially an independent application system built on NS_TRANPORTOR and NS_CHAIN.

  • NS_DISPATCHER: The message entry specified by NS4 architecture. Based on the NETTY framework, NS_DISPATCHER accepts HTTP requests outside the boundaries of the business system through the HTTP service provided, and converts the requests into the message protocol format used by the internal communication of the business system.

The figure above shows the hierarchy of each system in NS4.

Run the process

In essence, the whole system of NS4 is a set of message-oriented middleware services and development framework. The overall structure is shown as follows:

The figure above shows the running flow of an NS4 overall distributed project. A message flows in the following order:

  • NS_DISPATCHER receives the HTTP request and converts it to the internal message protocol to put it in the specified message queue (according to the configuration file).

  • NS_CONTORLLER receives messages from the queue specified in Step 1 and sends them in sequence to the corresponding message queue for each business system step, starting with the configured service choreography.

  • The service system receives the message queue specified by NS_CONTROLLER in Step 2 and processes the message queue. After the processing is complete, the service system returns the result.

  • NS_CONTROLLER receives the response from the service system and sends the message result to the message queue corresponding to the next service system according to the configured service.

  • After the message is processed by all business systems, NS_CONTROLLER puts the message processing result into the specified message queue, and NS_DISPATCHER retrieves the message result from the corresponding message queue and responds to the HTTP caller.

Second, the ns4_gear_idgen

Open source: github.com/newsettle/n…

Ns4_gear_idgen (ID generator) is implemented based on the NS4 framework. It supports distributed deployment and generates globally unique ids. The length, prefix, suffix, step, and base can also be configured based on your own services.

advantages

  • Easy linear scaling that can support most business scenarios.

  • The generated ID rules are diverse and can be configured based on service requirements. The GENERATED ID rules can be based on 10, 36, and 62 bases.

  • Service ids are isolated from each other and do not affect each other.

  • It is not necessary to operate the database frequently to obtain the ID. The database will be operated only when the ID in the number segment is nearly exhausted, which reduces the pressure of the database.

  • The ids in the number segment are initialized in advance to ensure that the initialization is complete before all ids in each number segment are used up, avoiding the impact caused by service initialization after all ids are used up.

  • The size of key_value can be customized, and services can be easily migrated from the original ID mode.

  • High DISASTER recovery (Dr) and internal number segment caching enable services to be provided within a short period of time even if the DB is down.

Third, ns4_gear_watchdog

Open source: github.com/newsettle/n…

Ns4_gear_watchdog is the parent process of the NS4_frame process. It guards and manages the NS4_frame process. Its responsibilities include the following:

  • Remote start and stop of NS4_frame, real-time monitoring of ns4_frame process health status, memory consumption, CPU usage, internal threads;

  • Collect service log collection implemented by NS4_frame and service data for real-time internal service flow to monitor the online running status of the NS4_frame process, realized services, and flow status of service data in real time, and accurately, quickly, and conveniently locate exceptions, CPU, and thread running status.

Ns4_gear_watchdog functions as the parent process. The parent process starts the target project (child process) and monitors the environment factors (including memory consumption, CPU usage, load, and threads), code factors (code health), and service factors of the child process in real time. The parent and child processes communicate with each other through JMX, collect the above factor data, and save the data in ElasticSearch. Further, by analyzing the data and combining the indicators developed by the summary of the actual operation, the above factor data will be notified to the relevant person in real time through wechat robot.

[ns4_gear_watchdog]

Four, ns4_chatbot

Open source: github.com/newsettle/n…

Ns4_chatbot is a robot chat framework that integrates QQbot, WxChat, RASA and Web services. Ns4_chatbot provides wechat and QQ chat interfaces, and can send system monitoring messages to a group. It can also send ns4_gear_watchdog monitoring information to the corresponding group.

Ns4_chatbot implements the following functions:

  • Receive system calls from internal systems (such as monitoring systems) to push messages to QQ or wechat users. When an internal system invokes a service, the following information is required:

    • To which group?

    • To which user in this group

    • The content of the sent message

  • Can accept QQ and wechat users’ conversations, understand their intentions, and respond to users.

future

Creditease has always been practicing the technical belief that science and technology promote financial development, and is willing to open source share the results of technical practice, in order to promote the development and innovation of the fintech industry through the practical experience of CreditEase.

At present, CreditEase Institute of Technology has opened a number of creditEase’s technical achievements and research and development practices, sharing creditEase’s technical ideas for the software research and development industry. The open source of NS4 series modules will maintain long-term update and maintenance, and it is also hoped that more technical partners will join the open source project to jointly maintain and develop the open source achievements.

Source: Creditease Institute of Technology