Directory:
What is a distributed system?
Second, why choose distributed system architecture?
Three, how to split the system?
Four, the technical challenges brought by distributed?
What is a distributed system?
Before we talk about distributed system architecture, let’s first look at, what is distributed system?
So let’s say we have a system with 300,000 lines of code. Now it’s broken down into 20 small systems with over 10,000 lines of code each.
The original code is directly based on the Spring framework through THE JVM memory call, now disassembled, 20 small systems deployed on different machines, and then based on the distributed service framework (such as Dubbo) to make an RPC call, interface to interface through network communication to request and respond.
So an important feature of a distributed system is that services are called across the network. Let’s look at the following figure:
In addition, distributed systems can be roughly divided into two categories.
(1) The underlying distributed system
For example, HDFS (distributed storage system), Spark (distributed computing system), Storm (distributed streaming computing system), ElasticSearch (distributed search system), Kafka (distributed publish and subscribe messaging system), and so on.
(2) Distributed business system
Distributed business system, the original Java development of a large system, to split into a number of subsystems, multiple subsystems call each other, forming a large system as a whole.
For example, suppose you have made an OA system, which contains the permission module, employee module, leave module, financial module, and a project, which contains a bunch of modules, which will call each other and deploy one machine.
Now if you take his system apart, the access system, the employee system, the leave system, the financial system, four systems, four projects, deployed on four machines.
Then a request comes and completes the request. The employee system calls the permission system, the leave system and the financial system, and the four systems respectively complete part of the work.
The request is not considered complete until all four systems are finished. This is called a distributed business system.
Again, let’s take a picture to get a feel for the process:
Second, why choose distributed system architecture?
Some students may want to ask, I ran a server well, all systems a project all fixed, how good. Why do we have to build distributed systems, call each other remotely, and seem to add a lot of work?
I’m going to talk about this with the example of a company I once worked for.
Many years ago, before the distributed architecture, I worked for a company whose business lines were vertical “smokestacks”.
With the rapid development of Internet, the development of the company’s business has been registered users increase, site application of the function, scale expands unceasingly, especially the development of mobile Internet, APP, WeChat, self-service terminals access channels, a variety of new business, pouring into the new demand, system encountered all sorts of problems.
First of all, the project becomes bloated and huge unrestrained, the system complexity increases, hundreds of thousands of lines of code, dozens of developers, service layer, DAO layer code is used in a large number of copy, often a variety of code merge conflicts to deal with, very time-consuming.
Often I change my code, others call my interface, resulting in his code also has problems, need to re-test, troublesome death.
Then each release is hundreds of thousands of lines of code system released together, we have to be nervous to prepare for the online, hundreds of thousands of lines of code online, each time the online have to do a lot of checks, a lot of abnormal problem processing, everyone is highly nervous, was made almost collapse.
And if I have a new business and want to upgrade the dependencies, such as to the latest Spring version, I still can’t, because it may cause other people’s code error, dare not arbitrarily change the technology. And a Web project can take minutes to start each time, and debugging code in the native IDE is a pain.
Secondly, with the increase of user access traffic, the system load becomes heavy and becomes overwhelmed. By increasing the number of instances, the effect of increasing hardware capacity expansion is negligible, resulting in frequent failures and low efficiency. System quality is becoming increasingly difficult to guarantee, and the test cycle is becoming longer and longer, which cannot meet the needs of the company’s business development.
The above is the company before some “unbearable” past, in general, the problem is mainly reflected in the following aspects:
(1) Serious application code coupling and difficult function extension
(II) Long interaction cycle of new requirement development and heavy test workload
(iii) It takes a long time for new development colleagues to get familiar with the system
(four), upgrade maintenance is also very difficult (change any place to upgrade the whole system)
(V) It is difficult to improve system performance, low availability and instability.
Now that we’ve seen the pain of system coupling, let’s look at some of the benefits of decoupling:
First of all, when the system is broken down, it feels like the whole world is clean.
A system with hundreds of thousands of lines of code is split into 20 services, with an average of 10,000 to 30,000 lines of code per service, and each service is deployed on a separate machine. 20 projects, 20 Git repository code, 20 developers, each maintaining their own service.
- Because it is their own independent code, has nothing to do with others. No more code conflicts, cool!
- Every time I test my own code, cool!
- Every time I release my own a small service is ok, cool!
- Technically upgrade however you want, keep the interface definition unchanged, input and output content unchanged, cool!
To sum up, the split of distributed systems can greatly improve the development efficiency of large teams of complex systems.
Three, how to split the system?
Generally speaking, to split a system, you need to be familiar with the system as a whole. You can take the idea of multiple rounds of splitting, the first splitting is the previous each large module coarse-grained splitting.
For example, an e-commerce system can be divided into order system, commodity system, store system, membership system, promotion system, payment system and so on.
Later, each system may become more and more complex, such as the order system can be further divided into shopping cart system, inventory system, price system and so on.
Generally speaking, it is based on the idea of field-driven design and practical experience summary, while referring to some conventional practices in the industry, we discuss to carry out separation, step by step optimization, multi-round separation, small steps and fast running, and finally achieve a better state.
Four, the technical challenges brought by distributed?
The first is the selection of distributed service framework. At present, dubbo and Spring Cloud are the mainstream in China.
Let’s think about this for a moment. What are the main problems that a service framework is used to solve? Is it possible to do distributed architecture without Dubbo or Spring Cloud?
It is certainly possible to do without a service framework such as Dubbo or Spring Cloud, but that requires a lot of work on your own.
For example, each subsystem is called through a restful interface, so it is HTTP call. In this case, for example, an object is sent, we need to make a JSON, and then after a call failed to retry what to do?
In addition, generally cluster deployment, the target system has multiple instances, so I have to write a load balancing algorithm, how to randomly select one of the multiple target machines to call?
Also, if a new instance is deployed on the target system for capacity expansion, or an instance is taken offline by a server failure, how can callers be notified dynamically? If you don’t use a service framework, you can run into all kinds of problems.
Here’s a picture of the process:
If you choose a distributed service framework, you need to have an in-depth understanding of the use and underlying principles of this framework. For example, Dubbo needs to understand the following questions:
(I) How does Dubbo work?
(2) Which serialization protocol is supported by Dubbo?
(c) Dubbo’s load balancing and high availability strategy? Dynamic proxy policy?
(4) Dubbo’s SPI idea?
(v) How to conduct service governance, service degradation, retry after failure and retry after timeout based on Dubbo?
(VI) How to design the idempotency of dubbo service interface (such as no repeated deduction, no repeated order generation, no repeated creation of card number)?
(vii) How to ensure the sequence of dubbo service interface requests?
(8) How to design a RPC framework similar to Dubbo?
The same is true with Spring Cloud, how does Eureka work? How does feign declarative invocation work? And so on all kinds of underlying principles to understand.
There are other common technical issues that need to be addressed with distributed architecture:
(1) Distributed session
(2) distributed lock
(3) Distributed transactions
(4) distributed search
(5) Distributed cache
(6) Distributed message queue
(VII) Unified configuration center
(8) distributed storage, database sub-library sub-table
(IX) current limiting, fusing, degradation, etc.
Each of these issues will require perhaps N articles to elaborate on, but we will continue to talk about the various technical issues of distributed architecture in several articles.