Research | 5 kinds of distributed transaction solution for the advantages and disadvantages compared

Distributed transaction is a technical difficulty in enterprise integration, and it is also involved in every distributed system architecture, especially in microservices architecture, which is almost unavoidable.

Refers to the four basic elements of the proper execution of a database transaction:

• Atomicity

• Consistency

• Isolation

• “Durability”

CAP principle, also known as CAP theorem, refers to Consistency, Availability and Partition tolerance in a distributed system. The CAP principle is that, at best, these three elements can achieve two, not all at once.

• Consistency: Whether all data backups in a distributed system have the same value at the same time.

• Availability: Whether the cluster can respond to read/write requests from clients after some nodes fail.

• Partitioning tolerance: In practical terms, partitioning is equivalent to time-bound requirements for communication. If the system cannot achieve data consistency within the time limit, it means that A partitioning situation has occurred and that it must choose between C and A for the current operation.

BASE theory is the result of balancing consistency and availability in CAP. The core idea of the theory is: we cannot achieve strong consistency, but each application can adopt appropriate ways to achieve the final consistency of the system according to its own business characteristics.

• Basically Available

• Soft state

• Eventually consistent

Two-phase commit 2PC is one of the most powerful transaction types in distributed transactions. Two-phase commit is a two-phase commit where the first phase asks each transaction data source if it is ready and the second phase actually commits the data to the transaction data source.

To ensure that the transaction will meet ACID, a Cooradinator will be introduced. The other nodes are called participants. The coordinator is responsible for scheduling the behavior of the participants and ultimately deciding whether they want to commit the transaction. The processing process is as follows:

• Phase one

A) The coordinator sends transaction content to all participants, asks if the transaction can be committed, and waits for a response.

B) Each participant performs a transaction and writes undo and redo information to the transaction log (but does not commit a transaction).

C) If the participant performs successfully, feedback yes to the coordinator; otherwise, feedback no.

• Phase two

If the coordinator receives a failure message or timeout message from each participant, send a rollback message directly to each participant. Otherwise, send a commit message. The two cases are handled as follows:

Case 1: When all participants report yes, the transaction is committed

A) The coordinator issues a formal commit transaction request (i.e., commit request) to all participants.

B) The participant performs the COMMIT request and frees the resources occupied during the entire transaction.

C) Each participant feedback the ack completion message to the coordinator.

D) The coordinator will complete the transaction submission after receiving ack messages from all participants.

Case 2: When a participant reports no, the transaction is rolled back

A) The coordinator issues rollback requests (i.e., rollback requests) to all participants.

B) The participant uses the undo information in phase 1 to perform a rollback and release the resources occupied during the entire transaction.

C) Each participant feedback the ack completion message to the coordinator.

D) The coordinator will complete the transaction after receiving ack messages from all participants.

• The problem

1) Performance problem: all participants are synchronously blocked in the transaction submission stage, occupying system resources and easily leading to performance bottleneck.

2) Reliability issues: If the coordinator has a single point of failure, or a failure occurs, the provider will remain locked.

3) Data consistency issues: In phase 2, if both the coordinator and the participant are dead, there may be data inconsistency.

The three-phase commit is an improved version of the two-phase commit. The most critical problem of 3PC is that both the coordinator and the participant fail at the same time. Therefore, 3PC divides the preparation stage of 2PC into two parts again, so that the three-phase commit. The processing process is as follows:

• Phase one

A) The coordinator issues a canCommit request containing transaction content to all participants, asks if the transaction can be committed, and waits for all participants to respond.

B) After receiving the canCommit request, if the participant thinks that the transaction operation can be performed, he/she will feedback yes and enter the preparatory state; otherwise, he/she will feedback no.

• Phase two

The coordinator has two possibilities, depending on how the participants respond.

Case 1: All participants report yes, and the coordinator pre-executes the transaction

A) The coordinator issues a preCommit request to all participants to enter the preparation phase.

B) After receiving a preCommit request, the participant performs a transaction, recording undo and redo information to the transaction log (but not committing the transaction).

C) Each participant feeds back ACK response or NO response to the coordinator and waits for the final instruction.

Case 2: The transaction is interrupted as soon as one participant reports no or the coordinator does not receive feedback from all providers after a timeout

A) The coordinator issues abort requests to all participants.

B) The participant interrupts the transaction whether it receives an ABORT request from the coordinator or if a timeout occurs while waiting for the coordinator’s request.

• Stage three

The actual transaction commit at this stage can also be divided into the following two scenarios.

Case 1: All participants respond with an ACK response and perform a true transaction commit

A) If the coordinator is at work, issue a DO Commit request to all participants.

B) After receiving the DO Commit request, the participant formally performs the transaction Commit and releases the resources occupied during the entire transaction.

C) Each participant feedback the ack completion message to the coordinator.

D) The coordinator will complete the transaction submission after receiving ack messages from all participants.

Case 2: The transaction is rolled back as soon as one participant reports no or the coordination group does not receive feedback from all providers after a timeout.

A) If the coordinator is working, issue a ROLLBACK request to all participants.

B) The participant uses the undo information in phase 1 to perform a rollback and release the resources occupied during the entire transaction.

C) Each participant feedback the ack completion message to the coordination group.

D) The coordination group will complete the transaction rollback after receiving ack messages from all participants.

TCC is a two-stage programming model of servitization. The compensation mechanism is as follows:

Condition:

• The validation and compensation logic needs to be implemented

• Need to support idempotent

Processing process:

A) In the Try stage, it mainly detects and reserves resources for the service system.

This stage is mainly completed:

• Complete all business checks (consistency).

• Reserve required service resources (quasi-isolation).

• Try Attempts to execute the service.

B) The Confirm stage is mainly to Confirm and submit the business system.

If the Try phase succeeds and the Confirm phase starts, the Confirm phase does not fail by default. If the Try succeeds, Confirm succeeds.

C) In the Cancel phase, services that need to be rolled back due to errors in service execution are cancelled and reserved resources are released.

The core idea is to split distributed transactions into local transactions for processing.

The scheme creates additional transaction message table in the consumer, the consumer processes the business and records the transaction message in the local transaction, polls the data of the transaction message table and sends the transaction message, and the provider consumes the transaction in the transaction message table based on the message middleware.

Condition:

• The service consumer needs to create a message table to record message status.

• Service consumers and providers need to support idempotent.

• Compensation logic is required.

• A timed thread starts on each node to check for unfinished or failed messages and reissue messages, i.e. retry mechanism and idempotent mechanism.

Processing process:

1. The service consumer commits the business data along with the message to initiate the transaction.

2. The message is sent through MQ to the service provider and the service consumer waits for the processing result.

3. The service provider receives the message, completes the business logic and notifies the consumer of the processed message.

Fault tolerance processing is as follows:

• When the step 1 processing fails, the transaction is rolled back and nothing happens.

• When steps 2 and 3 fail, since the message is stored in the consumer table, it can be re-sent to MQ for retry.

• If an error occurs in Step 3 and the service fails, the service provider sends a message to inform the consumer of the transaction failure. In this case, the consumer initiates a rollback transaction for rollback.

MQ transaction messages (Final Consistency)

MQ supports transaction messages in a manner similar to two-phase commit.

The distributed transaction scheme based on MQ is actually the encapsulation of the local message table. The local message table is based on THE internal MQ, and the other aspects of the protocol is basically consistent with the local message table.

Condition:

A) Need to compensate logic

B) Business processing logic needs to be idempotent

Processing process:

C) Consumers send half messages to MQ.

D) After the message is persisted, MQ Server confirms to the sender ACK that the message has been successfully sent.

E) The consumer starts executing the transaction logic.

F) The consumer submits a secondary acknowledgement or rollback to MQ Server based on the execution result of the local transaction.

G) MQ Server receives the commit state and marks the half message as deliverable.

H) The service provider receives the message and executes the local business logic. Returns the processing result.

Saga mode is a distributed asynchronous transaction, a final consistent transaction, and a flexible transaction. There are two different ways to implement Saga transactions. The two most popular ways are:

In the absence of a central coordinator (no single point of risk), each service generates and listens to events from other services and decides if it should take action.

The implementation first executes a transaction and then publishes an event. Intercepting the event by one or more services, these services to implement local transaction and release (or not) the new event, when the last service execution local transactions and not to release any event, mean the end of a distributed transaction, or any event has not been released it Saga participants heard all means the end of the transaction.

Processing process:

1. The order service saves the new order, sets the state to pengding pending, and publishes an event named ORDER_CREATED_EVENT.

2. The payment service listens on ORDER_CREATED_EVENT and publishes the event BILLED_ORDER_EVENT.

3. The inventory service listens to BILLED_ORDER_EVENT, updates the inventory, and publishes ORDER_PREPARED_EVENT.

4. The shipping service listens on ORDER_PREPARED_EVENT and delivers the product. Finally, it publishes ORDER_DELIVERED_EVENT.

5. Finally, the order service listens for ORDER_DELIVERED_EVENT and sets the status of the order to completed.

Suppose the inventory service fails during a transaction. Rollback:

1. The inventory service produces PRODUCT_OUT_OF_STOCK_EVENT

2. The ordering service and the payment service listen for this event from the inventory service above:

   ① Payment service will be refunded to the customer.

   ② Order service sets the order status to failed.

Command/coordination orchestrator: A central orchestrator is responsible for centralized decision making and business logic ordering of events.

The Saga orchestrator communicates with each service in the form of command/reply, telling them what to do.

1. The order service stores the pending state and asks the Order Saga Coordinator (OSO for short) to start the order transaction.

2. OSO sends the execute collection order to the collection service, which replies with the Payment Executed message.

3. OSO sends the order preparation command to the inventory service, which responds to the OrderPrepared message.

4. OSO sends the Order delivery command to the shipping service, which replies with the Order Delivered message.

The OSO Order Saga coordinator must know in advance the process required to perform the “Create Order” transaction (obtained by reading the BPM Business process XML configuration). It is also responsible for coordinating distributed rollbacks by sending commands to each participant to undo previous actions in the event of any failures. Rollback is much easier when you have a central coordinator to coordinate everything, because by default the coordinator executes the forward process and just executes the reverse process when rolling back.