preface

With the expansion of business, more and more functions. Putting all the functions in the same service, the code is mixed, which makes maintenance difficult, and it is easy to cause a small bug that makes the whole service unavailable. Therefore, according to the business function, we will be divided into many different services (the formation of microservices). The system composed of multiple services has a resounding name: distributed system; And how should we manage the service state in the system? What are the relevant theories?

  • Distributed and clustered
  • Database transaction
  • Distributed transaction
  • Distributed data consistency
  • Theory of CAP
  • The BASE theory of

Pay attention to the public account, communicate together, wechat search: sneak forward

Distributed and clustered

  • Distributed refers to the system formed through the exchange of information by multiple services or components connected through the network
  • A cluster is an ensemble of two instances of the same service component
  • The two concepts are not completely in conflict, and a distributed system can also be a cluster. A ZooKeeper cluster is also a distributed system whose services communicate and collaborate with each other
  • A cluster is a situation where a cluster is not a distributed system, such as multiple HTTP servers that are load-balanced and do not communicate with each other, and cannot be called a distributed system without a load-balanced part

Database transaction

  • Transactions operate based on data and need to ensure that the data of transactions is usually stored in the database. Therefore, when introducing transactions, we have to introduce ACID characteristics of database transactions
  • Atomicity, all the operations in the whole transaction, either complete or not complete, can not be stopped in some intermediate step
  • Consistency. The Consistency constraints on database data are not broken before and after a transaction
  • Isolation. Isolation prevents data inconsistency caused by cross execution when multiple transactions are executed concurrently
  • Durability, modifications to data are permanent after transactions end and they are not lost even if system failures

Distributed transaction

  • Distributed system is generally composed of a number of independent subsystems, which cooperate with each other to complete each function through network communication. This collaboration process requires data consistency across systems, and we call such cross-system transactions distributed transactions

  • There are multiple scenarios in the above scenario; Inventory service and order service are all successful. Or the inventory service and order service are partially successful, and the traditional stand-alone transaction theory is no longer applicable

Difficulties in distributed transactions

  • Atomicity: transactions operate across different nodes. When a node operation fails on multiple nodes, do the multi-node operations either do nothing or do both
  • Consistency: When a network transmission failure or node failure occurs, the data replication channel between nodes is interrupted. Therefore, data consistency must be ensured during transaction operations
  • Isolation: In distributed transaction control, it is possible to commit asynchronously, with “partially committed” transactions

Distributed data consistency

  • ACID is not suitable for distributed transactions, and the difficulties of distributed transactions involve problems, which ultimately lead to data inconsistency. Therefore, the distributed system will focus on ensuring system consistency.

Theory of CAP

  • The problems involved in the difficulties of distributed transactions mentioned above will ultimately lead to data inconsistency. The following is a theoretical analysis of the consistency of distributed systems, and the following is an introduction of distributed schemes based on these theories (conflict between availability and consistency :CAP theory).
  • Consistence: All nodes access the latest copy of the same data
  • Availability: Non-failed nodes return reasonable responses within a reasonable time (not error or timeout responses)
  • Partition tolerance: Distributed systems can still provide services against network partitions

When a network partition occurs, if we want to continue service, it is a choice between strong consistency and availability. That is to say, when P is the premise after network partition, C and A can be selected only after P is determined. That is to say, we must realize Partition tolerance

Why not also guarantee CA?

If Partition is displayed, a node in the system is writing data. In order to ensure consistency of C, read and write operations of other nodes must be prohibited, which conflicts with A. If the read and write operations of other nodes are normal to ensure A, data consistency cannot be guaranteed, which conflicts with C

CAP practical application cases

  • ZooKeeper guarantees CP. All read requests to ZooKeeper can be consistent at any time, but ZooKeeper does not guarantee the availability of each request. For example, the service is unavailable during the Leader election process or when more than half of the machines are unavailable
  • Eureka guarantees AP. Eureka is designed to ensure A (availability) first. There are no Leader nodes in Eureka; each node is equal and equal. So Eureka won’t be unavailable during elections or when more than half of the machines are unavailable, as ZooKeeper is. Eureka guarantees that the failure of most nodes will not affect normal service delivery, as long as only one node is available. It’s just that the data on this node may not be up to date

The BASE theory of

  • BASE is Basically Available, soft-state, and Eventually Consistent. BASE theory is A trade-off between consistency (C) and availability (A) in CAP
  • Final consistency is a special case of weak consistency. The system ensures that data consistency can be achieved within a certain period of time

Basic available

Basic availability means that a distributed system is allowed to lose part of its availability in the event of unpredictable failures. What is allowed to lose some usability?

  • Loss in response time: Normally, it takes 0.5 seconds to return a result to process a user request. However, due to a system failure, the time to process a user request becomes 3s
  • Loss of system functions: Under normal circumstances, users can use all functions of the system, but some non-core functions of the system cannot be used due to the sudden increase of system visits

Soft state

  • Soft state refers to allowing the existence of intermediate states (data inconsistency in CAP theory) of data in the system, and considering that the existence of such intermediate states will not affect the overall availability of the system, that is, allowing the system to delay the process of data synchronization between data copies of different nodes

Final consistency

  • Final consistency emphasizes that all copies of data in the system can eventually reach a consistent state after a period of synchronization. Therefore, the essence of final consistency is that the system needs to ensure the consistency of the final data, rather than ensuring the strong consistency of the system data in real time

Corrections are welcome

Refer to the article

  • CAP and BASE theory? Can you talk about the actual case?
  • What is the difference between distributed and clustered?
  • Data consistency issues