In the CITIC Cloud Prototype Industry Conference in 2021, Changjietong won the Excellent Case of Cloud Prototype in 2021.

Changjietong Co., Ltd. is a member of Yonyou Group, focusing on the financial and management services for domestic small and micro enterprises. On the one hand, Changjietong makes its products, business and technical architecture internet-based. On the other hand, Changjietong has launched the Changjietong one-stop cloud service platform, which provides platform services, application services, business services and data value-added services centered on Shuizheng finance and taxation and Shuizheng business for small and micro enterprises, and is committed to establishing a “service ecosystem for small and micro enterprises”.

According to the report of Analysys International, Changjietong currently maintains the first coverage rate in the domestic small and micro enterprise cloud service market. More than 1.3 million enterprises and organizations have realized cost reduction, efficiency improvement, continuous innovation and digital-intelligence transformation through the use of ChangJietong’s software and services.

Early, chang access business group business applications are built in the company independent research and development of CSP platforms, including customer housekeeper, easy generation zhang, good accounting and good business, etc., each enterprise distribution, an independent virtual machine resources completely isolated, early the scheme greatly simplifies the complexity of the development platform, improve the development efficiency, meet the company to the initial needs of the development of cloud services.

New requirements

As the number of users increases, the existing scheme of assigning a separate virtual machine to each user causes significant problems. First of all, the increasing number of virtual machines, in the case of low customer conversion rate, easy to cause a waste of resources, while the cost of operation and maintenance is also high; Secondly, different user software iteration versions are not the same, but also need to specially design and maintain multiple versions of the patch update system; Third, the product needs to stop service and go online, and the business will be temporarily unavailable. Finally, if a number of users of the business peak, can not quickly and flexibly expand capacity, the utilization of resources is not high.

Under this background, the chang access business group research and development team decided to try to compare the current industry general cloud native architecture design scheme of using the cloud infrastructure, Shared computing resources and storage resources, the scheme of container, rapid expansion and elastic, micro service architecture, according to the application to update products, completely solve the current operational cost is high, the lack of elasticity, the resources use imbalance problem.

Small and micro enterprises are characterized by large numbers, relatively small business volume of a single enterprise and limited IT capabilities. Taking Changjietong good business products as an example, the adoption of the existing cloud native architecture is convenient for Changjietong to quickly develop a flexible and extensible SaaS application for large and small users. At the same time, through service arrangement, other product lines can be quickly built, such as Zhi +. At present, more than 200,000 paying users are using the cloud native architecture enterprise cloud service provided by Changjietong, generating more than 100 gigabytes of business data every day.

Driving the cloud native application architecture

The rapid development of domestic cloud computing products, the obvious trend of enterprise application migration to the cloud, coupled with the government departments to encourage enterprises to launch subsidy policies, the cloud has become the general trend of enterprises.

Especially in the epidemic stage, with the change of business model and consumption mode, only the enterprise on the cloud can be more conducive to accelerate the transformation of digitization and intelligence, and more effectively help the enterprise to achieve “customer online, business online, personnel online and management online”. Now the value brought by the cloud native technology can better help enterprises to carry out digital intelligence transformation.

1. Live online

Using intelligent access gateway, access to the nearest cloud enterprise network, between the VPC, VPC and set up private network communication channel between the local data center, through the automatic routing distribution and learning, improve the quality of the network of rapid convergence and across a network communication and security, ensure real-time online users worldwide, accelerate the realization of the integration of online of the customers.

2. Number of intellectualization

With cheap storage in the cloud, powerful computing resources and numerous algorithm models, Changjietong stores massive data at a very low cost and conducts real-time online analysis, thus providing users with more business decisions.

3. Quick response to market demand

Using DDD design idea and micro-service architecture, high cohesion and low coupling applications can be developed rapidly. Through service arrangement, these applications can quickly combine more applications to meet customer groups in different industries and fields, and achieve the effect of fast online and iterative optimization.

4. Stable and reliable

Container and microservice architectures allow you to quickly build and run resiliently scalable applications. In case of system failure or performance bottleneck, mirroring can restore damaged applications in seconds, ensuring high availability of the system. By taking advantage of the dividends of cloud native technology, Changjietong can only focus on business development. On the one hand, it can accelerate the construction of new applications; on the other hand, it can also optimize existing applications and integrate them into the cloud native architecture, so as to achieve the effect of changing wheels while running, and make it more convenient for customers with historical stock to upgrade to the cloud.

Cloud native application architecture design

Cloud native application architecture design route

The original product is deployed in the physical IDC, through the CloudFoundry cloud platform two open, to achieve virtual machine isolation between each tenant. However, because each tenant only enjoys the container + database, when the number of users reaches hundreds of thousands of levels, the efficiency of database upgrade, the cost of container deployment and the complexity of hardware operation and maintenance are all significantly improved. Through the application of micro-service, cloud to solve the problem of cost reduction and efficiency is imminent.

Through the reconstruction of cloud on infrastructure, cloud on database, technical framework and business framework, Changjietong has realized container deployment, application sharing and DB sharing among multi-tenants. The products are based on EDAS and the Alibaba cloud container service Kubernetes version ACK integrated on it. It is hoped that the current problems of high operation and maintenance cost, insufficient system flexibility and long product iteration and delivery cycle can be solved through the technical dividend of cloud native.

Application architecture modification

1. Microservice architecture

Complex applications are segmented from a business perspective into highly cohesive, low-coupling modules that are developed and distributed independently. The business domain is divided into four layers, namely, the core domain service layer, the business domain service layer, the application service layer and the interface service layer. The core domain service layer includes authorization, UOM, organization (Party), product, pricing, promotion and inventory model modules, mainly providing core domain knowledge and ability services. Business domain service layer is to provide business functions of good business, including procurement, inventory management and sales domain services; The application service layer invokes domain services based on specific application scenarios to solve specific business problems in the application. Each layer of service is a separate micro-service, which manages the whole life cycle of the service based on EDAS, and realizes the registration discovery and governance of the service through the introduction of Spring Cloud.

In addition, due to the seamless integration of the ACK, EDAS supports hosting applications in the form of containers to Aliyun Kubernetes clusters or mixed clusters (self-built clusters in other cloud domains or IDC), so it can get through with Changjietong low-level K8S clusters. The timing elasticity of K8S cluster and the automatic elasticity of K8S cluster based on CPU/RT/Load are realized.

2. Data consistency

Traditional single service architecture, all the functions of the domain are aggregated in a process, can ensure strong business consistency through database transactions. However, Changjietong is now designed according to the distributed micro-service architecture. Different domain modules will build micro-services that run independently, and the consistency of data between micro-services needs to be guaranteed according to the final consistency scheme. For scenes requiring high real-time performance, Changjietong adopts TCC model; For those with low real-time requirements and long process processing, Changjietong adopts message queue for service decoupling to achieve final consistency.

Transformation of technical architecture

1. Containerized management

The core application is deployed in the container, orchestrated and run through Kubernetes. For seckill scenarios or computation-intensive asynchronous tasks, build on demand using function calculations.

2. Service governance

After the introduction of micro-service architecture, service management is particularly complicated. For this reason, Changjietong introduces Spring Cloud one-stop solution, so that the development only needs to focus on the development of business, without paying attention to technical details. Through Spring Cloud, service discovery and registration, configuration management, stream-limiting degradation, service invocation, data monitoring, etc., are completed to achieve cost reduction and efficiency improvement, as well as reduce operation and maintenance costs.

3. Gitops pipeline

Based on Gitlab, Jenkins, Rundeck and K8S, a self-developed DevOps pipeline is built. Independent construction of micro-applications, free package grouping of micro-services, and release and deployment in accordance with containerized conditions are carried out to ensure that the operating environment of each stage of research and development, testing and online remains unchanged.

4. Database transformation

All tenants share the database, and divide the database into tables according to application, data volume and other factors; The OLAP database is introduced to separate the transaction library and the analysis library to avoid the large query dragging the user’s transaction.

Cloud native application technology framework

The system is divided into front-end display layer, business middle platform, technology platform, operation and maintenance middle platform and infrastructure layer.

Front-end presentation layer: H5 front-end development framework formed based on micro-front-end application integration idea is used, and Qiankun is used to realize the integration of homogeneous applications and heterogeneous applications, supporting multi-terminal development.

Business Middle Platform: It adopts the design idea of micro-service architecture and is built based on the EDAS platform. In the application service layer, it can realize elastic scaling, current limiting degradation, flow monitoring, etc. The business model is metadata driven and supports GraphQl queries; RocketMQ is used to decouple the business.

Technology middle platform: including container management, DevOps pipeline, micro-service governance, link tracking, etc., is the cornerstone of fast business delivery, stable system operation, rapid business innovation.

Infrastructure layer: including data storage layer and middleware. Data is stored based on relational databases such as MySQL, PolardB, etc., with multi-tenant database sharing at the database table level through a Tenant identity; The database also supports read-write separation, and the query operation only needs to access the read database. The rest of the middleware are secondary encapsulated by the technology platform, shielding the specific technical details from the business.

The front desk uses a micro front frame

六四事件

Front-end application due to the increase in the number of personnel to participate in, product application module as demand increases unceasingly, has evolved from a common monomer evolved into a boulder application, good business, intelligence +, micro mall products such as front development project together, mutual influence, lead to development of function and application of iterative on-line can’t according to separate deployment. In addition, the unclear stratification of business and technology in the early stage leads to insufficient abstraction of public components, strong coupling between business and technology, and extremely difficult upgrading of technology alone.

Changjietong needs to design a set of framework to ensure the smooth migration of its business code, and ensure that Changjietong can carry out technical iteration and update without affecting the business online.

Changjietong only needs to construct a lightweight enough base in the main system, plus common reusable components (including basic technical components, basic business components, common business services, etc.), and then let each sub-application to implement according to the common protocol. This protocol includes how child applications should be loaded by the master application, how children should be perceived and scheduled by the master application, and how applications communicate with each other. At the same time, the system of Changjietong should also provide a unified build project, so that all sub-applications no longer pay attention to configuration, dependency, build, release and other operations. In this way, each micro-application can be distributed and deployed independently, and the weak coupling between different applications reduces the complexity of development.

Technology Platform – Container Management

Quick Release:

From git source code management platform and based on the interaction of configuration management platform to realize the rapid automatic generation and management of container mirror, based on the unification of the environment variable configuration and differentiating center controls, can achieve a container mirror deployment environment pattern, and the code of each safety and consistency of scanning, security code link security and stability.

Closed-loop management:

After it is published online, the abnormal information is sent to the message center based on Prometheus monitoring of Ali Cloud, and the data aggregation and strategy arrangement in the message center form the workorder flow mode to realize the closed-loop management of effective data.

Business guarantee link:

In terms of container elasticity and expansion, Changjietong makes use of the HPA mechanism of K8S, based on the ability of AliCloud container service ACK to maximize the utilization of resources and the custom indicators of the business layer, to realize the rapid expansion and expansion capacity of micro-services in the face of sudden traffic such as live streaming, seckilled and online exam.

Technology Platform -DevOps pipeline

The pipeline mode is adopted to connect the tasks that originally run independently in a single or multiple nodes, so as to realize the complex process layout that is difficult for a single task to complete. Automating the build, test, and release process makes it easy to test every code change and catch bugs that are easy to fix.
By building the DevOps tool chain, the whole process of support from requirement issuance, code submission and compilation, test and verification, deployment and operation is realized, so as to open the complete path of software delivery and provide end-to-end support for software research and development.

Technology Platform – Microservice Governance

With the rapid development of business, Changjietong has carried out a lot of micro-service-oriented transformation on the original IT system to meet the needs of rapid iteration and frequent release of large Internet applications. Because SaaS enterprise management cloud service has the characteristics of large number of users, complex business, long call link and deep integration with third-party application system, it brings great challenges to the micro-service-oriented transformation work. Changjietong must improve the overall microservice governance and monitoring ability to ensure the stability and robustness of the system in frequent version iteration.

Finally, Changjietong’s micro-services are deployed to the enterprise-level distributed application service EDAS provided by Aliyun. Spring Cloud applications running on the EDAS can enjoy a series of enhanced capabilities in micro-service governance, such as application life cycle management, non-destructive offline, full-link flow control and so on. Especially in the process of application release, the smooth upper and lower lines and grayscale mechanism provided by EDAS greatly improve the stability of the system during version update and reduce the risks caused by application release.

Technical platform – full link monitoring

Hayne’s Law states that for every serious accident, there must be 29 minor accidents, 300 attempted mishaps and 1,000 potential accidents.

Especially after Changjietong adopted micro-service architecture, due to the extremely complex business links involved in SaaS products, when users feedback system bugs or performance problems, the technical team needs to spend a very long time to locate fault sources and analyze performance bottlenecks among the complex links. Chang access business group also USES some APM tools products, including application performance monitoring, monitoring the user experience, link tracking, problem diagnosis, etc., but this kind of tools can only be positioned level framework, for a custom function and process of the asynchronous processing cannot link tracking, in a distributed application architecture of this kind of APM’s role is less.

Because Changjietong’s application is hosted in Aliyun’s EDAS, EDAS integrates application real-time monitoring service ARMS, which can monitor the health status and key indicators of micro-services, set alarms for monitoring indicators, timely detect and deal with possible anomalies or failures, so as to ensure the health and availability of applications. Therefore, Changjietong only needs to connect the business operation with the system log, system log and ARMS. Starting from the business and running through the whole business life cycle, Changjietong can quickly locate the application performance bottlenecks and the location of abnormal faults.

To this end, Chang Jietong has implemented a set of mechanism, which abstractly models business operations according to Timeline, and forms a trinity full-link tracking mechanism by combining the system log and Aliyun ARMS system.

In principle, any interaction with a write permission point other than a read should be treated as a BI. So ChangJitong can simply think of the permission point of each write operation as a BI. This in turn requires that the REST APIs provided by the back end be scenario-oriented, with one permission point for each API.
There needs to be an association between the BI and the REST API’s request_id in order to trace the relationship between business operations and system logs.
WebFilter intercepts the Request and Response of all RESTful APIs, obtains the Request and Response information, and analyzes the specific characteristics, roles, relationships and other data from the intercepted data through the value formula in the interaction protocol. Create operation is added to Request Request to record the original value of the entity automatically. Complete operation is added when Response returns to write the new value and record the changes before and after. The two are paired at the beginning and end of a call to an interface.
In the background log, a comparison between user_req_id and the Uber-trace-id of ARMS in Aliyun is recorded.

Through the full-link tracking mechanism, Changjietong connects the business interactive operation with the monitoring of Aliyun Arms application. Especially, there are some decoupling operations through message queue in the business, which can be tracked by Changjietong through BI, which further provides the monitoring capability for Changjietong’s micro-service system. After the access to ARMS, the workload of locating system fault sources and performance bottlenecks was reduced to less than 50% of the previous level through full link information screening and application of real-time diagnosis and other tools, which greatly improved the work efficiency of the IT team.

Technical platform – Grayscale release

A grayscale release (also known as a canary release) is a release that has a smooth transition between black and white. A/B testing can be done on it, that is, some users continue to use product feature A and some users start to use product feature B.

Grayscale period: The period between the release of a new feature in the grayscale environment and the deployment of the new feature in the official online environment is called the grayscale period. For the application of 2C, the user is used as the basic unit of gray level for shunt. For the application of 2B, the tenant is the basic unit for distribution.

The grayscale environment includes database grayscale and application grayscale. If the grayscale is supported in the database level, a new grayscale DB is needed to import the customer data participating in grayscale into the grayscale DB. After the grayscale is over, the data cleaning will be incorporated into the formal production DB. This process requires a lot of operation and high cost, because of this, the database level does not consider grayscale. Based on this setting, the following constraints need to be followed:

The amount of grayscale customers is controlled in a small range to minimize the range of data repair;
The model is designed in strict compliance with the principle of compatibility, such as: only increase, not decrease, field reuse is avoided;
There needs to be an assessment of the system data that affects the business logic. For example, if a system-level enumeration value is added, which is only visible to grayscale customers, it can be considered to copy a copy of the system data for grayscale customers and delete it after grayscale. The back-end code logic of the system gives priority to the tenant data, and if not, the system-level data can be obtained.

Since the services provided by the back end are REST APIs, load balancing or reverse proxy supporting the invocation of external services can be used to obtain the list of grayscale users and then distribute the users. Backend interface compatibility needs to be guaranteed, and when compatibility cannot be guaranteed, the front end needs to be forced to upgrade with it.

Infrastructure layer – database read and write separation

When the cloud service was launched in the early stage, the user scale was small and the amount of data was small, so the online environment MySQL did not implement the read-write separation scheme. During a period of stable operation, PV and UV increased with the user scale, which brought great pressure to the database, mainly reflected in the following three points:

Complex business, multi-table joint inspection efficiency is low, resulting in business function response is not timely enough or even query timeout;
The traditional database can not be upgraded quickly during the peak of business, so it can only wait for the peak of business to pass or do flow control;
Business code is sensitive to master-slave delays and cannot take full advantage of slave libraries.

At the beginning, Sharding-JDBC was used to achieve read-write separation, but the read-write operations of existing products are interwinding, which makes it difficult to separate the read operations, and the synchronization delay between the read and write libraries is large, which does not meet the business needs. After investigating PolardB, we found that its cluster address directly supports read-write separation, and is 100% compatible with MySQL syntax, with no impact on business. With the minute-level bounce lift capability of POLARDB, it can be upgraded quickly. Its unique parallel query ability can also reduce the response time of complex queries and improve the system’s concurrency ability.

Therefore, Changjitong migrated the database from MySQL to Polardb, adopted the mode of “one write, many read”, and designated separate read nodes for time-consuming report query business, which reduced the impact of time-consuming operations on other businesses and ensured the normal transaction operations of users.

Operation and maintenance center – monitoring system

The traditional alarm mechanism takes the message as the carrier and is as simple as possible. It is more inclined to the mode of alarm and fault, which covers less information and often plays a state of reminding the late judgment.

With the improvement of operation and maintenance mode, a large number of efficient tool combination, self-healing, timing event and other related systems continue to emerge. At the same time, the emergence of cloud native mode in recent years will realize the age-old problems of intelligent early warning: It provides a high-quality solution to the problem of information standardization, and also accelerates the pace of upgrading the monitoring structure. The single mode of alarm no longer meets the demand, and the operation and maintenance personnel prefer to alarm with meaning, process and conclusion.

Changjietong has been committed to three-dimensional monitoring architecture based on customer experience for a long time. From the latitude of customer performance and experience loss, it classifies and decentralises monitoring information, and accurately monitors user experience through customer portraits and feature models in each link of the business chain. It can alert and enter the processing process when the user is not aware of the situation, and the event entering the process can be reversed in the internal message center, correlation calculation, root cause analysis and self-healing scheme. And according to the event processing model to respond to the event, send an alarm, the alarm covers the key information and conclusion of the event. It improves the effectiveness of early warning and avoids all kinds of efficiency loss in the traditional fault processing model.

Cloud native application service features

DevOps Tool – Troubleshooting Center

It is well known that distributed systems follow CAP theory, and Chang Jite Tong usually chooses to meet AP over C (Consistency). According to Murphy’s law, Chang Jie Tong ultimately needs to ensure the final consistency of data through human intervention. You need a system to help you check for inconsistencies, identify error scenarios, and fix them according to protocol.

Different from system operation and maintenance, business maintenance is oriented to business and service, and business problems can be found and solved by monitoring tenants and tenant data. In the operation of the system, in addition to data inconsistency, there are some other common failure, also need to support and solve the problem of some of the more common have common customers to provide convenient, and provide some analysis views help r&d or operations master understand the tenant status and trend, because the demand response will have production environment and safety requirements of physical isolation, and become inefficient, Only by solidifying into the system to form a toolset can it be better solved.

Therefore, Chang Jietong provides a set of business maintenance system, which is responsible for business data monitoring, problem diagnosis, fault repair, situation warning, etc. The overall design is as follows:

At present, the system carries out regular checks for six types of faults, such as failure of opening, failure of scheduled tasks, dead letter, violation of data audit, import timeout and TCC failure, and integrates with the MIDAS message system of operation and maintenance to find problems in time and send a pin warning to the specific person in charge. The person in charge can check the specific reason through the Kanban board and detailed list of the failure display; For specific faults, it also provides auxiliary means to view the context log, re-open, dead-letter replay, quantity account recalculation, resubmit and so on.

Security services – content security

Cheong access business group products will involve the synergy of the chat messages, electricity class product evaluation, the content may be used by outsiders are malicious, will some illegal advertising, Internet fraud, terrorist information content such as entry chang of products access business group and spread out, so the smooth access business group operational safety department need to increase the content of security monitoring, to ensure the company product purity, health, To avoid cyber crimes occurring in the products and users of Changjietong Company.

Cheong access business group to borrow system operation log and message queue, decouple the content security detection and business functions, with the help of a function to calculate the capacity, on-demand invocation of cloud services vendors security detection (text, pictures, etc.) to identify can share business, content is in line with safety, test results also added artificial audit and misjudgment forgive.

Number with separation

Statistics and analysis data. Changjietong will incremental synchronize the original data from the relational database to the data warehouse through DTS every day. After data cleaning and statistical analysis at a fixed time, it will transfer this data to ElasticSearch. At the same time, MaxCompute also uses the multitasking computing power of MaxCompute to perform tagging calculation of business data. The calculation results are also passed to ElasticSearch. Business systems access these data through ElasticSearch’s REST API and present them to the end users.

Full link flow control technology + terminal cloud co-modulation

Because of the dependencies between microservices, developers cannot do development and testing locally and must rely on a test development environment.

Research and development efficiency: when hundreds of developers share a set of environment, code changes frequently in the development state, the quality is low, and services interact with each other, the development environment is often interrupted, and debugging is difficult, which seriously affects the development efficiency. Research and development hope that each project can provide a set of environment, as shown in the figure:

Research and development cost: if a full environment is provided for each project, the cost of computing resources is very high, and the cost of operation and maintenance will also surge. (In the development state, there are nearly one hundred micro-services, and the computing resource cost and operation and maintenance cost of nearly 2000 POD/ECS are needed for parallel development and calculation based on 20 projects.) In the growth period of the product, the number of parallel projects and services is increasing.

Considering the factors of efficiency and cost, ChangJietong introduces gateway, full-link traffic precise control technology and end-cloud co-regulation technology, and overlaid the project development environment with the application of baseline environment + incremental modification: traffic marking is carried out at the entrance according to the enterprise ID (environment identification is injected in the request according to the enterprise ID), such as: https://cloud.chanjet.com/req… = Enterprise ID, environment ID flow with the request throughout the execution process (HTTP request, RPC request, timing task, message, etc.), through the environment ID, micro-service invocation/message precise control. As shown in figure:

Changjietong deploys the latest online code in the baseline environment, and only deploys the application involving modification in the project environment:

(1) Provide a stable independent project environment for R&D, enabling rapid collaboration in a “small, multi-batch” (DevOps best practice) manner; (2) Saving resources and reducing operation and maintenance costs. (Modify 5% of the application according to each project, and there are nearly 100 micro-services in the development state. If you develop 20 projects in parallel, it will save 2000*95%=1900 ECS/POD); (3) The use of end-cloud co-modulation technology facilitates the development of local PC to be added to the project environment debugging. As shown in the figure:

The technical value of cloud native

Highly elastic and expandable

The system can support horizontal scaling in application service layer and database layer.

Application: Through micro-service architecture and containerized deployment, the technology platform can sense server load, flexibly scale service nodes, realize cluster expansion/shrinkage, and quickly replenish computing capacity.

Databases: Databases support rapid expansion of read nodes to support more concurrent requests, while tenants implement data isolation and cross-library migration based on load.

Micro service

During the microservice architecture and design phase:

MDD domain-driven methodology was introduced to design application architecture and domain model.
According to the design principle of micro-service, service separation, responsibility and relationship definition are carried out, and service interface and domain model are determined.

As a complex TOB application, Changjietong split micro-services according to the following principles:

Available – split modules to meet the needs of the application;
Easy to use – split modules can be composed by a relatively simple, clear way of application, service application value;
Beauty – the simplest way to express complex issues; Easy for business people to understand.

Back-end For Front-end

After the micro-service architecture is adopted on the server side, Changjietong needs to add a BFF layer between the front and rear ends to simplify the collaborative development complexity of the staff at the front and rear ends. The BFF layer is realized based on Node. Chang Jitong chooses Egg. JS and makes layered encapsulation based on Egg. In order to improve efficiency, Changjietong also connects to the Node layer cache middleware Redis, and uses GraphQl to do aggregate queries.

End-cloud co-modulation scheme

After the adoption of micro-service architecture, each set of environment deployment needs more than 30 micro-service containers. If there are multiple feature parallel in the development stage, the environment is deployed separately for each feature branch, which will consume too much resources. For this reason, Changjietong introduced gateway, full-link traffic marking control technology, and terminal cloud co-modulation technology, and used baseline environment + incremental modification application to stack the project development environment, so as to quickly build a complete set of development and debugging environment at the minimum cost and reduce the development cost. At the same time, developers can register their local computers directly in the micro-service environment to complete the verification and testing of upstream and downstream businesses locally, which greatly improves the working efficiency of developers and the quality of the submitted code.

Offline data analysis

In the early days, the database of Changjietong only had OLTP, and the data was stored in MySQL. With the increase of data volume, the query of statistical analysis not only took a long time for itself, but also consumed the resources of the online transaction database, affecting the response time of online transactions. Reading and writing separation through PolarDB, the cloud native database, can only relieve some of the query pressure.

For some analysis with low real-time requirement in the system, Changjietong introduced data warehouse for offline data processing, ETL and unified scheduling through DataWorks tool, and big data index calculation and label calculation in MaxCompute.

Full link grayscale scheme

The system of Changjietong realizes the gray scale of full link from front end to back end service, message queue and timing task. Front-end static resources: distinguish gray static resources and formal environment static resources. After logging in, the tenant in the login information determines which environment should be routed to. Message queue: Distinguishes grayscale message queue from formal message queue. Consuming timed tasks based on environment variables: The task executor of timed tasks, using environment variables and tenant list, decides whether to execute or not. Backend REST interface: Through Lua script in Nginx, the URL path is parsed and routed according to the tenant information in the grayscale list.

Cloud native business value

Containerized operation and maintenance management:

Containerized deployment and management based on EDAS+ACK mode realizes the promotion of application release, and in elastic scaling and capacity management, can also speed up abnormal identification and fault self-healing. And it’s climbing up the scale from 2 (anomaly recognition) to 5 (quick location) to 10 (self-healing stop).

Observability:

Based on the combined analysis of the log platform and the data center platform, the display of user portraits and user tracks can be realized, and the continuous mining of data can achieve the quantification of user experience percentage and product quality percentage, so as to provide digital guarantee for the connection of service teams.

Cost:

The upgrade from the container mode of IDC room to the containerization of cloud platform does not need to invest a large amount of hardware equipment and replacement equipment, thus greatly saving a lot of equipment cost and labor cost. In addition, new demands and ideas can be invested according to the amount and flexibly scaled up.

DevOps: Connecting DevOps to the cloud native environment saves 50% of staff in R&D requirements support, and improves the efficiency of build and deployment by a factor of four. Among them, the annual updating times were 12,734 in 2020, with a success rate of 91.8%.

Multiple environmental resources:

Through the mode of multi-feature gray scale scheme, 7 development environments are merged into 1 set. Each set of environment has 90 micro services, which can realize flexible dynamic adjustment and expansion. Save 450 nodes.

Cloud native containerization: containerized EDAS+ K8S deployment mode can realize the elastic expansion of nodes, especially supporting Changjietong live broadcast activities and seckill scenes. It not only saves the investment of human deployment, but also saves the cost of second scale.

Cloudy platform:

Through the establishment of DevOps and containerized mode, it meets the needs of users in cloudy scenarios, saves 50% of manpower, and realizes the automatic operation and maintenance mode in cloudy environment.

Database Upgrade:

After upgrading from MySQL to PolardB, the processing performance improved by 20%-40% and the unsupported low-end handheld PDA adapters were successfully resurrected. Sampling return visits to customers, satisfaction also increased significantly.

Stability:

Based on the docking of cloud native infrastructure, cloud products can realize SLA services of 5 9, thus avoiding the stability and security risks of building open source components by themselves. And in the component function can also enjoy the benefits of various business requirements function. After connecting to the cloud, the stability guarantee target of 2 (alarm) -5 (positioning) -10 (stop loss) can be put forward, so as to continuously improve the user satisfaction.

Deep interpretation of changjietong cloud native architecture transformation process