01 preface

The term “Microservices” comes from a post by Martin Fowler called Microservices, which can be found on his official blog HTTP :/ / Martin Fowler. Com/articles/microservices. HTML, in a nutshell, the service is a kind of design style, on the system architecture is the main idea of it will be an independent system is divided into several small service, these small services are in the process of independent operation, between services based on HTTP RESTfuL AP implements communication collaboration. Common micro-service frameworks :Spring Cloud of Spring, Ali Dubbo, Huawei ServiceComb, Tencent Tars, Facebook Thrift, Sina Weibo Motan. This chapter begins with an understanding of the components that make up a complete system. The next chapter will use these components to build a complete distributed system.

1.1 Spring Cloud

This goes without saying, the official website has a detailed introduction.

1.2 Spring Cloud Alibaba

Spring Cloud Alibaba is committed to providing a one-stop solution for microservice development. This project contains the necessary components for developing distributed application microservices that developers can easily use to develop distributed application services through the Spring Cloud programming model

Sentinel: Takes traffic as the entry point to protect the stability of services from multiple dimensions, such as flow control, fuse downgrading and system load protection. Nacos: A dynamic service discovery, configuration management, and service management platform that makes it easier to build cloud-native applications. RocketMQ: An open source distributed messaging system that provides low-latency, highly reliable message publishing and subscription services based on highly available distributed clustering technology. Dubbo: Apache Dubbo™ is a high-performance Java RPC framework. Seata: Alibaba open source product, an easy-to-use high-performance microservices distributed transaction solution.

02 Service registration and discovery

Eureka: 2.x is officially no longer open source (closed source), previous versions have been discontinued; That said, Eureka has no more technology to offer in the future. So, if you want registries to be more powerful, you need to take a different approach. Zookeeper: In enterprise Zookeeper registries, Dubbo is used in kafka. With Eureka’s stop, we can use the spring-cloud-starter-Zookeeper-Discovery initiator. Make Zookeeper the springCloud registry. Consul: Go language development, is also an excellent service registration framework, the use of more. Nacos: From SpringCloudɵɹibaba, which has passed millions of registration tests in enterprises, can not only replace Eureka perfectly, but also replace other components, so Naocs also strongly recommends using it.

2.1 Introduction to Nacos as a registry

(1) Introduction to Nacos

Nacos (Dynamic Naming and Configur Ation Service) is an open source project of Alibaba in July 2018, which is dedicated to discovering, configuring and managing micro-services.

(2) Nacos installation

A single node

Docker pull nacOS/nacOS-server :1.3.1 -- Start the container docker run --name nacos --env MODE=standalone --privileged=true -p 8848:8848 --restart=always -d dc833dc45d8fCopy the code

Access:

http://127.0.0.1:8848/nacos account password is nacos

The cluster

Install the prerequisite

64-bit OS Linux, Unix, or Mac. Linux is recommended. The cluster needs to rely on mysql, and a single machine does not need 3 or more Nacos nodes to form a clusterCopy the code

(3) Steps for building a Nacos HA cluster:

1. Clone Nacos cluster project nacOS-Docker 2. Nacos-docker is composed with docker Compose, so you need to install Docker Compose first. https:/ /nacos.io/zh-cn/docs/quick-start-docker.html

Docker Compose Compose project is the official Docker open source project, responsible for the implementation of Docker container cluster rapid choreography.

# under Linux download download (to/usr/local/bin) curl - L > https://github.com/docker/compose/releases/download/1.25.0/run.sh /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-composeCopy the code

2) Clone nacOS-Docker project

Clone git clone https://github.com/nacos-group/nacos-docker.gitCopy the code

3) Run the nacos-Docker script

Docker-compose -f /usr/local/nacos/nacos-docker/example/cluster-hostname.yaml upCopy the code

Pulling from nacOS/nacOS-mysql Version is 5.7 (execute initialization script) Pulling nacos3 (NACOS/NACOS-Server :latest) latest release

4) Stop and start

Docker-compose -f /usr/local/nacos/nacos-docker/example/cluster-hostname. Yaml start docker-compose -f /usr/local/nacos/nacos-docker/example/cluster-hostname /usr/local/nacos/nacos-docker/example/cluster-hostname.yaml stopCopy the code

5) Visit Nacos

http://192.168.1.1:8848/nacos
http://192.168.1.1:8849/nacos
http://192.168.1.1:8850/nacos
Copy the code

(4) Nacos Quick start

Configuring the service Provider

Service providers can register their services with Nacos Server through Nacos’s service registry discovery feature.

Add nacOS dependencies

<dependency>
    <groupId>com.alibaba.cloud</groupId>
    <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
    <version>${latest.version}</version>
</dependency>
Copy the code

Add the configuration

Server port = 8070 spring. Application. Name = nacos - demo spring. Cloud. Nacos. Discovery. The server - addr = 127.0.0.1:8848Copy the code

Start the class

@SpringBootApplication @EnableDiscoveryClient public class NacosProviderApplication { public static void main(String[] args) { SpringApplication.run(NacosProviderApplication.class, args); } @RestController class EchoController { @RequestMapping(value = "/echo/{string}", method = RequestMethod.GET) public String echo(@PathVariable String string) { return "Hello Nacos Discovery " + string; }}}Copy the code

After startup, console:

Note If the registration is successful, you can view the service in the background:

支那

  1. I sorted out some information, and friends in need can click to get it directly
  2. Microservice architecture: RPC+Dubbo+SpirngBoot+Alibaba+Docker+K8s
  3. Java Core Knowledge Set +25 topic interview set

支那

Let’s use Spring Cloud to integrate NAOCS for service invocation

Configuring service consumers

Add the configuration

Server port = 8080 spring. Application. Name = service - consumer spring. Cloud. Nacos. Discovery. The server - addr = 127.0.0.1:8848Copy the code

Add a startup class: The service consumer implements the service invocation using @LoadBalanced RestTemplate

@SpringBootApplication @EnableDiscoveryClient public class NacosConsumerApplication { @LoadBalanced @Bean public RestTemplate restTemplate() { return new RestTemplate(); } public static void main(String[] args) { SpringApplication.run(NacosConsumerApplication.class, args); } @RestController public class TestController { private final RestTemplate restTemplate; @Autowired public TestController(RestTemplate restTemplate) {this.restTemplate = restTemplate; } @RequestMapping(value = "/echo/{str}", method = RequestMethod.GET) public String echo(@PathVariable String str) { return restTemplate.getForObject("http://service-provider/echo/" + str, String.class); }}}Copy the code

(5) Test

Start ProviderApplication and ConsumerApplication, called http://localhost:8080/echo/2018, return to the content for the Hello Nacos Discovery in 2018.

03 Distributed Configuration Center Solution and application

At present, there are more configuration centers in use in the market (chronological order) Disconf: In July 2014, Baidu open source configuration management center also has configuration management ability, but it is no longer maintained, the last submission is 4-5 years ago. Spring Cloud Config: Open source in September 2014, Spring Cloud ecosystem component, can be seamlessly integrated with the Spring Cloud system. Apollo: In May 2016, Ctrip open source configuration management center, with standard permissions, process governance and other features. Nacos: June 2018, Ali open source configuration center, also can do DNS and RPC service discovery

3.1 Introduction to Nacos as a distributed configuration center

Once Nacos Server is started, you can follow the example code below to start the Nacos configuration management service for your Spring Cloud application. See nacos-spring-cloud-config-example for a complete example code

  1. Add dependencies:

    com.alibaba.cloud spring-cloud-starter-alibaba-nacos-config ${latest.version}

Note: Version 2.1.X. lease corresponds to Spring Boot version 2.1.x. Release corresponds to Spring Boot 2.2. x, and 1.5.x. release corresponds to Spring Boot 1.5.x.

For more version mappings, see version Description Wiki

  1. Configure the address and application name of the Nacos Server in bootstrap.properties

    Spring. Cloud. Nacos. Config. Server – addr = 127.0.0.1:8848

    spring.application.name=example

Note: You need to configure Spring.application.name because it forms part of the Nacos configuration management dataId field.

In Nacos Spring Cloud, the full format of dataId is as follows:

${prefix}-${spring.profiles.active}.${file-extension}
Copy the code

  • Prefix for spring. The default application. The name of the value, but can be by spring configuration items. The cloud. Nacos. Config. The prefix to configuration.

  • Spring.profiles. active is the profile corresponding to the current environment. For details, refer to the Spring Boot documentation. Note: When spring.profiles. Active is null, the corresponding connector – will also not exist and the dataId concatenation format becomes prefix.{prefix}.prefix.{file-extension}

  • File – exetension as the configuration of content data format, can be configured a spring. Cloud. Nacos. Config. The file – the extension to the configuration. Currently, only properties and YAML types are supported.

  1. Automatic configuration updates via the Spring Cloud native annotation @refreshScope:

    @RestController @RequestMapping(“/config”) @RefreshScope public class ConfigController {

    @Value("${useLocalCache:false}")
private boolean useLocalCache;

@RequestMapping("/get")
public boolean get() {
    return useLocalCache;
}
Copy the code

    }

  2. The configuration is first published to the Nacos Server by calling the Nacos Open API: dataId is example.properties and useLocalCache=true

    The curl -x POST “http://127.0.0.1:8848/nacos/v1/cs/configs? dataId=example.properties&group=DEFAULT_GROUP&content=useLocalCache=true”

  3. Run NacosConfigApplication, calls curl http://localhost:8080/config/get, returns the content is true.

  4. Call the Nacos Open API again to publish the configuration to the Nacos Server: dataId is example.properties and useLocalCache=false

    The curl -x POST “http://127.0.0.1:8848/nacos/v1/cs/configs? dataId=example.properties&group=DEFAULT_GROUP&content=useLocalCache=false”

  5. Visit http://localhost:8080/config/get again, this time to return to false content, show the useLocalCache in application value has been updated dynamically.

04 Distributed Service Invocation

4.1 summary of the RPC

The primary functional goal of RPC is to make it easier to build distributed computing (applications) without losing the semantic simplicity of local invocation while providing powerful remote invocation capabilities. To achieve this goal, the RPC framework needs to provide a transparent call mechanism so that consumers do not have to explicitly distinguish between local and remote calls.

Advantages of RPC: distributed design, flexible deployment, decoupled services, and strong scalability.

4.2 RPC framework

Dubbo: The earliest open source RPC framework in China, developed by Alibaba and opened source at the end of 2011, only supporting Java language. Motan: RPC framework for internal use of Weibo, opened source in 2016, only supports Java language. Tars: RPC framework for internal use in Tencent, opened source in 2017, only supports C++ language. Spring Cloud: An open source RPC framework from Pivotal in 2014, which provides rich ecological components. GRPC: Google open source cross-language RPC framework in 2015, supporting multiple languages. Thrift: Originally developed by Facebook as a cross-language RPC framework for internal systems, Thrift was contributed to the Apache Foundation in 2007 as one of the Apache: Open Source projects, supporting multiple languages.

4.3 Advantages of RPC framework

RPC frameworks generally use long links to reduce network overhead by eliminating the need for three handshakes for each communication. RPC framework usually have registry, is rich in monitoring management issue, offline interface, such as dynamic extensions, for the caller is no perception, unification of private operation agreement, safe the RPC protocol more simple content smaller, more efficient, service architecture, service management, the RPC framework is a strong support.

4.4 RPC framework application: Use Spring Cloud Alibaba to integrate Dubbo implementation

Since Dubbo Spring Cloud is built on top of the native Spring Cloud, its service governance capabilities can be considered as Spring Cloud Plus, which not only fully covers Spring Cloud native features, but also provides a more stable and mature implementation. The feature comparison is shown in the following table:

4.5 Dubbo is invoked as a Spring Cloud service

By default, Spring Cloud Open Feign and @loadBalanced ‘RestTemplate are the two service invocation methods of Spring Cloud. Dubbo Spring Cloud provides a third option where the Dubbo service appears as an equivalent citizen of the Spring Cloud service invocation, Applications can use Apache Dubbo annotations @service and @Reference to expose and Reference Dubbo services and realize communication between services over multiple protocols. At the same time, a service gateway can be easily implemented using the Dubbo generalization interface.

4.6 Getting Started

In the traditional Dubbo development pattern, the first step before building the service provider is to define the Dubbo service interface for the service provider and the service consumer. To ensure contract consistency, it is recommended to package the Dubbo service interface into a second or third party artifact (JAR) without even adding any dependencies. For service providers, Dubbo service interfaces not only need to be introduced as dependencies, but also need to be implemented. The corresponding service consumer, again, needs to rely on the artifact and execute remote methods in the form of interface calls. The next step is to create the artifact.

4.7 Creating a Service API

Create an API module for each interface:

/** * @author original * @date 2020/12/8 * @since 1.0 **/ public interface TestService {String getMsg(); }Copy the code

4.8 Creating a Service Provider

Import dependence

<dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId>  </dependency> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-openfeign</artifactId> < version > 2.2.2. RELEASE < / version > < / dependency > < the dependency > < groupId > com. Alibaba. Cloud < / groupId > <artifactId>spring-cloud-starter-dubbo</artifactId> </dependency> <! Demo </groupId> <artifactId> dubo-demo API </artifactId> < version > 0.0.1 - the SNAPSHOT < / version > < / dependency > < / dependencies >Copy the code

Write the configuration

Dubbo: scan: # dubbo scan service benchmark package base - packages: org. Springframework. Cloud. Alibaba. Dubbo. The bootstrap protocol: Port: -1 Spring: Cloud: nacos: # nacos service discovery and registration configuration: server-addr: 127.0.0.1:8848 implementation /** * @author Original * @date 2021/1/28 * @since 1.0 **/ @service //dubbo Service annotation public class TestServiceImpl implements TestService { @Override public String getMsg() { return "123123"; }}Copy the code

Start the class

/** * @author original * @date 2021/1/28 * @since 1.0 **/ @springBootApplication @enableDiscoveryClient public class DubboProviderApplication { public static void main(String[] args) { SpringApplication.run(DubboProviderApplication.class,args); }}Copy the code

4.9 Creating service consumers

Reuse the provider’s code in addition to the API’s implementation classes

Writing test classes

/** * @author Original * @date 2021/1/28 * @since 1.0 **/ @restController Public class TestController {@reference TestService testService; @GetMapping("/dubbo/test") public String getMsg(){ return testService.getMsg(); }}Copy the code

Access:

Back to 111

05 Service Traffic Management

5.1 Why is flow control degraded?

Traffic is very random and unpredictable. One second may be calm, the next second may appear flood peak (such as double eleven zero scene). However, the capacity of our system is always limited. If the sudden flow exceeds the system’s capacity, it may lead to unprocessed requests, slow processing of accumulated requests, high CPU/Load, and finally lead to system crash. Therefore, we need to limit such sudden traffic to handle requests as much as possible while ensuring that the service is not overwhelmed, which is flow control.

A service often calls another module, perhaps another remote service, a database, a third-party API, and so on. For example, when making a payment, you may need to remotely call the API provided by UnionPay. Querying the price of an item may require a database query. However, the stability of the dependent service is not guaranteed. If the dependent service is unstable and the response time of the request is longer, the response time of the method that invokes the service is also longer, threads pile up, and eventually the business’s own thread pool may be exhausted and the service itself becomes unavailable.

Modern microservice architectures are distributed and consist of a very large number of services. Different services call each other and form a complex call link. The above problems can have a magnified effect in link calls. If a link in a complex link is unstable, it may cascade to make the whole link unavailable. Therefore, we need to fuse downgrade unstable weak dependent services to temporarily cut off unstable calls to avoid local unstable factors leading to an avalanche of the whole.

Stop/upgrade/replace of fault tolerant components

Service degradation: Hystrix: It is not recommended by the official website, but it is still widely used in Chinese enterprises. Although the current version of stream limiting and fuse downgrading is officially supported (the version is stable), now it is officially recommended to use Resilience4j: The official website recommends using, but this is rarely used in China. Sentienl: from Spring Cloud Alibaba, replacing Hystrix components by Chinese companies, strongly recommended in China

So this is Sentinel.

5.2 Sentinel introduces

Sentinel is an open source project of Alibaba, providing multiple dimensions such as flow control, circuit breaker degradation and system load protection to ensure stability between services. Sentinel flow control is very simple to operate and can be configured on the console to see results. What you see is what you get

Sentinel has the following characteristics:

  • Rich application scenarios: Sentinel has undertaken the core scenarios of Alibaba’s double Eleven traffic drive in the past 10 years, such as SEC killing (i.e. burst traffic control within the range of system capacity), message peaking and valley filling, cluster flow control, real-time fusing of unavailable downstream applications, etc.

  • Complete real-time monitoring: Sentinel also provides real-time monitoring capabilities. From the console, you can see a summary of the performance of a single machine-by-second data, or even a cluster of less than 500 machines, for accessing the application.

  • Extensive Open source ecosystem: Sentinel provides out-of-the-box integration modules with other open source frameworks/libraries, such as Spring Cloud, Dubbo, and gRPC. You can quickly access Sentinel by introducing the appropriate dependencies and simple configuration.

  • Sophisticated SPI extension points: Sentinel provides an easy-to-use, sophisticated SPI extension interface. You can quickly customize the logic by implementing an extension interface. For example, customize rule management and adapt dynamic data sources.

website

https://github.com/alibaba/Sentinel https://github.com/alibaba/Sentinel/wiki/%E4%BB%8B%E7%BB%8D in Chinese https://sentinelguard.io/zh-cn/docs/introduction.htmlCopy the code

The use of Sentinel can be divided into two parts: Core library (Java client) : a runtime environment that does not rely on any framework/library and can run on Java 7 and above, and also has good support for frameworks such as Dubbo/Spring Cloud. Console (Dashboard) : The console is mainly responsible for managing push rules, monitoring, cluster traffic limiting allocation management, and machine discovery

Usage scenarios

In a Service Provider scenario, we need to protect the Provider itself from being overwhelmed by a rush of traffic. In this case, traffic control is usually based on the service provider’s service capability or restricted to a particular service caller. We can evaluate the bearing capacity of the core port in combination with the preliminary pressure test, and configure the flow limiting in QPS mode. When the number of requests per second exceeds the set threshold, the redundant requests will be automatically rejected.

To avoid getting bogged down by an unstable Service calling another Service, we either isolate and fuse unstable Service dependencies at the Service caller. The means include semaphore isolation, abnormal proportion degradation, RT degradation and so on.

At a certain point, when a system is stuck at a low water level for a long period of time and traffic suddenly increases, jacking the system up to a high water level can overwhelm the system instantly. At this time, the WarmUp flow control mode of Sentinel can be used to control the slow increase of the flow through and gradually increase to the upper limit of the threshold in a certain period of time, instead of releasing all the traffic in a moment. This gives the cold system time to warm up and prevents the cold system from being overwhelmed.

At the same time, the system processes as many requests as possible while keeping the load within the request processing water level.

The Gateway flow control features of Sentinel are used to protect traffic at the gateway entrance, or to limit the frequency of API calls.

5.2 Sentinel installation

1. Download the JAR package github.com/alibaba/Sen…

2, start,

Java - Dserver. Port = 8787 - Dcsp. Sentinel. Dashboard. Server = 127.0.0.1:8787 - Dproject. Name = sentinel - dashboard - jar / home/sentinel/sentinel - dashboard - 1.8.0 comes with. The jarCopy the code

3, access,

http://127.0.0.1:8787/#/login
Copy the code

Default Password Sentinel /sentinel

You can see that Sentinel is its own monitoring

5.3 Sentinel Quick Start

1. Import dependencies

< the dependency > < groupId > com. Alibaba. CSP < / groupId > < artifactId > sentinel - core < / artifactId > < version > 1.8.0 comes with < / version > </dependency>Copy the code

2. Test classes

public class TestService { public static void main(String[] args) { initFlowRules(); while (true) { Entry entry = null; try { entry = SphU.entry("HelloWorld"); /* Your business logic - start */ system.out.println ("hello world"); /* Your business logic - End */} catch (BlockException e1) {/* Flow control logic - start */ system.out.println ("block!" ); /* Finally {if (entry! = null) { entry.exit(); }}}} // Set the flow control rule Set the flow control rule when the QPS reaches 20. Private static void initFlowRules(){List<FlowRule> rules = new ArrayList<>(); FlowRule rule = new FlowRule(); rule.setResource("HelloWorld"); rule.setGrade(RuleConstant.FLOW_GRADE_QPS); // Set limit QPS to 20. rule.setCount(20); rules.add(rule); FlowRuleManager.loadRules(rules); }}Copy the code

Execution Result:

As you can see, the program consistently prints “Hello world” 20 times per second, the same threshold set in the rule. Block represents a blocked request.

Official usage document: github.com/alibaba/Sen…

5.4 Sentinel integrates SpringCloud to achieve service flow limiting/fusing

Import dependence

<dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId>  </dependency> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-sentinel</artifactId> </dependency> </dependencies>Copy the code

configuration

Server port = 8082 spring. Application. Name = sentinel - demo spring. Cloud. Nacos. Discovery. The server - addr = 127.0.0.1:8848 Spring. Cloud. Sentinel. Transport. Dashboard = 127.0.0.1:8787 / / in need of flow control method combined with @ SentinelResource @ RestController public class TestController { @GetMapping("/sentinel") @SentinelResource public String getMsg(){ return "11"; }}Copy the code

Start the application, visit http://127.0.0.1:8082/sentinel to sentinel after the background

Now let’s match the simplest flow control rule. Configure traffic limiting rules for the sentinel_spring_web_context /sentinel service call (which requires traffic to see). We have a flow control rule with a QPS of 1, which means that calls to the service method should not exceed 1 call per second, beyond which calls are rejected outright.

Now fast access: http://localhost:8082/sentinel

View the real-time monitoring page:

The use of other functions, we can refer to the official documents to explore.

5.5 How do I Select a Flow Control Degrade Component?

Here’s how Sent Inel compares to other fault-tolerance components:

06 Distributed Transactions

6.1 Why are Distributed transactions needed?

A distributed transaction means that the participants of the transaction, the server supporting the transaction, the resource server, and the transaction manager are located on different nodes of different distributed systems. Simply put, the units that make up the transaction are on different database servers. How to ensure consistency between services? When a microservice node in the middle of a long microservice invocation chain fails, how can data consistency be guaranteed for the entire service? The introduction of distributed consistency inevitably brings performance problems. How to solve the distributed consistency problem more efficiently has always been the key starting point of our efforts to solve this problem.

6.2 Classification of distributed transaction solutions

Rigid transaction

Rigid transactions refer to strongly consistent transactions that follow the four properties of local transactions (ACID). It is characterized by strong consistency, requiring each unit of a transaction to be committed or rolled back immediately. It has no time elasticity and requires synchronous execution. It is usually used in single architecture projects, usually enterprise applications (or LAN applications). For example, log when contract is generated, generate credentials after successful payment, and so on. However, in the current popular Internet projects, this rigid transaction to solve the distributed transaction will have many disadvantages. The most obvious or fatal of these is performance.

Because a participant cannot commit a transaction on its own and must wait until all participants are OK to commit the transaction together, the transaction is locked for a very long time, resulting in poor performance. Based on this, we came to the conclusion that the more phases, the worse performance.

Flexible transaction

Flexible transactions are for rigid transactions, we just analyzed rigid transactions, it has two characteristics, the first is strong consistency, the second is near real-time (NRT). Flexible transactions, on the other hand, do not require immediate execution (synchronization) and do not require strong consistency. It just needs to be basically available and ultimately consistent. To really understand, we need to start from the BASE theory and CAP theory.

6.3 CAP theory and BASE Theory

CAP theory, also known as CAP principle, has a very clear and consistent description of its concept on the Internet. In other words, the descriptions of CAP theory that we find on the Internet are basically the same. CAP refers to Consistency, Availability, and Partitiontolerance in a distributed system. C,A, and P are described as follows: Consistency (C) : Whether all data backups in A distributed system have the same value at the same time. Availability (A) : Whether the cluster as A whole can respond to clients’ read/write requests after some nodes fail. (High availability for data updates) Partitioning fault tolerance (P) : In practical terms, partitioning is equivalent to time-bound requirements for communication. If the system cannot achieve data consistency within the time limit, it means that A partitioning situation has occurred and that it must choose between C and A for the current operation.

The CAP principle is that, at best, these three elements can achieve two, not all at once. Therefore, when designing a distributed architecture, trade-offs must be made. For distributed data systems, fault tolerance of partitions is a basic requirement, otherwise it loses value. So designing distributed data systems is about striking a balance between consistency and availability. For most Web applications, strong consistency is not really required, so sacrificing consistency for high availability is the current direction of most distributed database products.

The term “BASE theory” is usually Available, soft-state, and Eventual Consistency. BA (Basically Available) : refers to the distributed system in the case of failure, allowing the loss of part of the availability, to ensure the core Available. But that’s not the same as not being available. For example, the search engine returns the query result in 0.5 seconds, but responds to the query result in 2 seconds due to a fault. When the web page is too large, some users provide degraded services. Basically, it’s available. 2. Soft State S: (Soft State) : The Soft State indicates that the system can have an intermediate State and the intermediate State does not affect the overall system availability. That is to allow the system to have a delay in the synchronization of copies between different nodes. Simply put, states can be out of sync for a period of time. E: (Eventually Consistent) :

After a certain period of time, all data copies in the system can reach the consistent state, which does not need to ensure the strong consistency of system data in real time. Final consistency is a special case of weak consistency. BASE theory is oriented towards large, highly available and scalable distributed systems, which achieve availability by sacrificing strong consistency. ACID is a conceptual design commonly used in traditional databases, which pursues a strong consistency model. In simple terms, the final data can be agreed within a certain time window.

BASE theory is based on CAP principles and is the result of a trade-off between consistency and availability in CAP.

Core idea: Even if strong consistency cannot be achieved, each business adopts appropriate ways to achieve final consistency of the system according to its own characteristics.

6.4 Distributed transaction solution

(1) Phase 2 Submission 2PC(3PC)

For more on 2PC and 3PC, see the blog post: 2PC and 3PC Principles, which are not the focus of our discussion today.

(2) XA

The XA standard has not been continuously evolving for more than 20 years since it was put forward. There are relevant researches on protocol optimization and log co-processing in the academic world, and there are relatively few XA implementation schemes in the industry, mainly focusing on application server scenarios. XA scheme requires relevant vendors to provide the implementation of the specific protocol. Currently, most relational databases support XA, but the degree of support is not the same. For example, MySQL did not fully support xA_prepare semantics until 5.7. XA scheme is criticized for its performance, but more seriously for the occupation of connection resources, resulting in insufficient connection resources to respond to requests in high concurrency, becoming the bottleneck of the system. With the expansion of microservice links, XA transaction scheme becomes an anti-scaling mode, which further aggravates the occupation of resources. In addition, XA transaction scheme requires all resources in transaction link to implement XA protocol. If one resource does not meet the requirement, data consistency of the whole link cannot be guaranteed.

(3) TCC compensation scheme

TCC stands for Try, Confirm, Cancel, respectively. It is a compensatory distributed transaction solution. What is compensation? In fact, it is easy to understand what the three parts of TCC do. First, let’s take a look at their main functions:

In the Try phase, service systems are detected and resources are reserved. The Confirm phase is used to Confirm the submission of the service system. If the Try phase succeeds and the Confirm phase starts, the Confirm phase does not fail by default. If the Try succeeds, Confirm succeeds. In the Cancel phase, services that need to be rolled back due to errors are cancelled and reserved resources are released.

From this, we can conclude that the Try commit transaction is performed in the Try phase, and when the Try execution is OK, Confirm execution is assumed to be successful by default. But when the Try commit fails, Cancel handles the rollback and resource release.

The TCC framework can be considered as a universal framework from the concept, but its difficulty lies in the relatively high development cost of the implementation of the three interfaces, which makes it difficult for many businesses to do the logical processing related to resource reservation, and whether it is necessary to ensure the isolation from the business level while reservating resources. Therefore, this model is more suitable for the deduction model which is easy to make resource reservation in the financial scene.

(4) the Saga

Saga is actually a concept mentioned in a database paper 30 years ago. In the paper a Saga affairs is a long running transaction, the transaction is composed of multiple local firm, each local transactions have corresponding compensation module, execution module and when an error in the Saga of any local affairs, by calling the corresponding compensation methods, related affairs to eventual consistency of the transaction.

Why do you need a Saga transaction solution when you have a TCC solution? As mentioned above, the cost of business transformation in TCC scheme is high. For the internal system, it is possible to reform the system from top to bottom. However, it is difficult to push the third party to reform TCC by invoking the interface of the third party, so that the third party can reform TCC scheme for you, but other users do not need. It is also clearly demand-irrational. It is reasonable to require third-party service interfaces to provide positive and negative interfaces, such as deduction and refund, for necessary data flushing in abnormal scenarios. In addition, Saga scheme is more suitable for long transaction scheme of working stream and can be asynchronous.

(5) Final consistency scheme

Local message table

This implementation method should be the most widely used in the industry, and its core idea is to split distributed transactions into cost-specific transactions for processing, which is derived from ebay. It is implemented in the same way as MQ transaction messages, using MQ to inform different services to implement transactions. The difference is that there are two different implementations of trust for message queues. The local message table is distrust of the stability of the message queue, thinking that the message may be lost, or the network of the message queue will be blocked, so an independent table is established in the database for storing the state of the transaction execution, and the control of the transaction is realized with the message queue.

MQ transaction messages

There are some third-party MQ services that support transactional messages, such as RocketMQ and ActiveMQ, and they support transactional messages in a manner similar to the two-phase commit adopted. However, some common MQ systems do not support transactional messages, such as RabbitMQ and Kafka. Take Alibaba’s RocketMQ middleware as an example. The idea is roughly as follows: The first stage Prepared message will get the address of the message. The second phase performs the local transaction. Phase 3 accesses the message using the address obtained in phase 1 and modifies the state. That is, two requests, one send message and one acknowledgement message are submitted to the message queue within the business method. RocketMQ will periodically scan the message cluster for transaction messages if it finds Prepared. It will acknowledge the message to the sender, so the producer needs to implement a Check interface. RocketMQ will decide whether to roll back or continue sending the confirmation message based on the policy set by the sender. This ensures that message delivery and the local transaction both succeed or fail at the same time.

Use an order flow chart to represent:

Both comparisons

6.5 Distributed transaction framework Seata

FESCAR is an open source distributed transaction middleware of Alibaba, which solves the distributed transaction problems in microservice scenarios in an efficient and zero-intrusion way. Seata is an upgraded version of Fescar, renamed Seata as of April 2019.

(1) Typical case analysis of SEATA

Seata’s official website provides a detailed set of business processes, which we use as an example to illustrate.

First, the transaction scenario of single architecture is introduced: three business modules (inventory, order and account) in e-commerce shopping. They operate on the local database, and since the same repository is the same, local transactions guarantee consistency, as shown in the following figure:

However, in the microservice architecture, as each business module becomes an independent service and each service connects to its own database, the original homology of the same database becomes different data sources of different databases. Although the database operation of each service can still use local transaction control, But to keep transactions consistent between services, distributed transactions are needed.

At this point, SEATA was introduced, which provided a “perfect” solution. As shown below:

The three seATa-related components in the figure, which were introduced earlier in fEScar and are the basic components of SEATA, are: 1) Transaction Coordinator (TC) : Maintains the state of global and branch transactions and drives global commit or rollback. 2) Transaction manager (TM) : Define scope of global transaction: start global transaction, commit or roll back global transaction. 3) Resource Manager (RM) : Manages the resources of branch transactions, communicates with TC to register branch transactions and report the status of branch transactions, and drives the commit or rollback of branch transactions.

Seata manages the life cycle of distributed transactions

  1. TM asks TC to start a new global transaction. TC generates xids that represent global transactions.

  2. Xids propagate through the invocation chain of microservices.

  3. RM registers the local transaction as a branch of the corresponding global transaction from XID to TC.

  4. TM requires the TC to commit or roll back the corresponding global transaction for XID.

  5. The TC drives all branch transactions under the corresponding global transaction of the XID to complete the branch commit or rollback.

The inventory service

public interface StorageService {

    /**
     * deduct storage count
     */
    void deduct(String commodityCode, int count);
}
Copy the code

Order service

public interface OrderService {

    /**
     * create order
     */
    Order create(String userId, String commodityCode, int orderCount);
}
Copy the code

Account service

public interface AccountService {

    /**
     * debit balance of user's account
     */
    void debit(String userId, int money);
}
Copy the code

Master business logic

public class BusinessServiceImpl implements BusinessService { private StorageService storageService; private OrderService orderService; /** * purchase */ public void purchase(String userId, String commodityCode, int orderCount) { storageService.deduct(commodityCode, orderCount); orderService.create(userId, commodityCode, orderCount); } } public class OrderServiceImpl implements OrderService { private OrderDAO orderDAO; private AccountService accountService; public Order create(String userId, String commodityCode, int orderCount) { int orderMoney = calculate(commodityCode, orderCount); accountService.debit(userId, orderMoney); Order order = new Order(); order.userId = userId; order.commodityCode = commodityCode; order.count = orderCount; order.money = orderMoney; // INSERT INTO orders ... return orderDAO.insert(order); }}Copy the code

All we need is a @GlobalTransactional annotation about the business method:

@GlobalTransactional
    public void purchase(String userId, String commodityCode, int orderCount) {
        ......
    }
Copy the code

6.6 Examples supported by Dubbo + SEATA

(1) Step 1: Establish database

  • Requirements: MySQL with InnoDB engine.

Note: In fact, in the example use case, these three services should have three databases. However, for the sake of simplicity, we can only create one database and configure three data sources.

Modify The Spring XML with the database URL/username/password you just created.

dubbo-account-service.xml dubbo-order-service.xml dubbo-storage-service.xml

<property name="url" value="jdbc:mysql://x.x.x.x:3306/xxx" />
        <property name="username" value="xxx" />
        <property name="password" value="xxx" />
Copy the code

Step 2: Create the UNDO_LOG table

UNDO_LOG SEATA AT mode requires this table.

Ux_undo_log CREATE TABLE undo_log (' id 'bigint(20) NOT NULL AUTO_INCREMENT, `branch_id` bigint(20) NOT NULL, `xid` varchar(100) NOT NULL, `context` varchar(128) NOT NULL, `rollback_info` longblob NOT NULL, `log_status` int(11) NOT NULL, `log_created` datetime NOT NULL, `log_modified` datetime NOT NULL, `ext` varchar(100) DEFAULT NULL, PRIMARY KEY (`id`), UNIQUE KEY `ux_undo_log` (`xid`,`branch_id`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;Copy the code

(3) Step 3: Create a table, such as business

DROP TABLE IF EXISTS `storage_tbl`;
CREATE TABLE `storage_tbl` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `commodity_code` varchar(255) DEFAULT NULL,
  `count` int(11) DEFAULT 0,
  PRIMARY KEY (`id`),
  UNIQUE KEY (`commodity_code`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;


DROP TABLE IF EXISTS `order_tbl`;
CREATE TABLE `order_tbl` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `user_id` varchar(255) DEFAULT NULL,
  `commodity_code` varchar(255) DEFAULT NULL,
  `count` int(11) DEFAULT 0,
  `money` int(11) DEFAULT 0,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;


DROP TABLE IF EXISTS `account_tbl`;
CREATE TABLE `account_tbl` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `user_id` varchar(255) DEFAULT NULL,
  `money` int(11) DEFAULT 0,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Copy the code

(4) Step 4: Start the server

  • From github.com/seata/seata…

    Usage: sh seata-server.sh(for linux and mac) or cmd seata-server.bat(for windows) [options] Options: –host, -h The host to bind. Default: 0.0.0.0 –port, -p The port to listen. 8091 –storeMode, -m log store mode: file, db Default: file –help

    e.g.

    Sh -p 8091 -h 127.0.0.1 -m file

(5) Step 5: Run the example

Head to the sample warehouse: Seata-Samples

  • Start the DubboAccountServiceStarter

  • Start the DubboStorageServiceStarter

  • Start the DubboOrderServiceStarter

  • Run DubboBusinessTester for the demo test

TBD: A script to run the demo application