What is MQ?

MQ, which stands for Message Queue, is the means by which applications communicate with each other.

What can MQ be used for?

What can it be used for, which is the case for MQ.

In the current microservices are popular, MQ is also used more and more, commonly used for “business asynchronous decoupling”, “decoupling microservices”, “traffic peak load filling”, “message distribution”, “distributed transaction data consistency”, let’s take a look at respectively.

1. Asynchronous decoupling

In a normal business process, an operation that is time-consuming and does not require immediate results. These operations can be treated as “asynchronous processing”, which greatly speeds up the response time of the request.

The most common scenario is that after registration, users need to send SMS or email notifications to inform relevant information.

Normal practice, is to go through three steps: user information processing, sending emails, sending SMS, and so on these three steps are all completed, before returning to the front end, to tell you a successful registration.

Using MQ, you only need to send two messages to MQ after processing user information. The mail service and SMS service listen for task messages of MQ and send them based on the messages.

2. Decouple microservices

Again, as an example of user registration, it makes sense to think of user registration and email/SMS as two separate microservices.

3. Flow peak cutting and valley filling

Flow control, MQ is also a more common scenario, generally used in the second kill, activities. At this time, the average user requests will surge, which may exceed the maximum capacity of the current system. If no processing is done, the system may break down.

Using MQ, you can put all the messages that need to be processed into it, and the system can consume the messages according to its maximum processing capacity. In this way, the requests that come in a moment can be scattered over a period of time to process, avoiding system crash.

4. Message distribution

This one is used a lot. Multiple systems interested in the same data need only listen for the same type of message.

Payment systems, for example, after payment successfully, the normal practice is to inform the peripheral system order payment is successful, or peripheral system time to pull the payment results, after using the MQ, payment system can be in payment after success, put messages to MQ, and want to know the result of system message can subscribe to this topic, is very convenient, There’s no need to pull data on a regular basis.

5. Data consistency of distributed transactions

As a first note, there is an open source framework called “Seata” that specializes in distributed transactions. Before Seata, the distributed transactions involved were typically handled by message-oriented middleware.

Seata is an open source distributed transaction solution dedicated to providing high performance and easy to use distributed transaction services. Seata will provide users with AT, TCC, SAGA and XA transaction modes to create a one-stop distributed solution for users.

The mainstream MQ framework supports distributed transaction messages. Many people may be confused at first contact with it. How to ensure transaction consistency? The figure below comes from Ali Cloud, which is very clear, a bit complicated flow chart, calm down to see or skip to see

Distributed transactions (photo from Aliyun)

Transaction message sending steps are as follows:

  1. The sender sends a semi-transactional message to the message queue RocketMQ.
  2. Message queue RocketMQ, after successfully persisting the message, returns an Ack to the sender confirming that the message has been successfully sent, which is a semi-transactional message.
  3. The sender starts executing the local transaction logic.
  4. According to the local transaction execution result, the sender submits a secondary confirmation (Commit or Rollback) to the server. When the server receives the Commit status, the semi-transaction message is marked as deliverable, and the subscriber finally receives the message. The semi-transaction message is deleted when the server receives the Rollback status and will not be accepted by the subscriber.

The steps for checking back transaction messages are as follows:

  1. In the case of network disconnection or application restart, the secondary confirmation submitted in Step 4 does not reach the server. After a fixed period of time, the server checks the message back.
  2. After receiving the message, the sender needs to check the final result of the local transaction execution of the corresponding message.
  3. The sender submits a second acknowledgement based on the final status of the local transaction, and the server continues to perform operations on the half-transaction message according to Step 4.

Three, what shortcomings

1. System availability decreases

System availability is reduced to some extent. Before joining MQ, you don’t have to worry about message loss or MQ failure, etc., but after you introduce MQ, you do!

2. Improved system complexity

The first thing you need to do when joining MQ is to have knowledge of message queues, to ensure that messages are not re-consumed, to handle message loss, to ensure that messages are delivered sequentially, and so on!

3. Consistency issues

From the usage scenario above, the asynchrony of message queues can actually improve system response times. But what if the true consumer of the message doesn’t consume the message correctly? This will lead to inconsistent data!

Four,

MQ has many advantages, but also has disadvantages, so it is up to you to analyze the need for message-oriented middleware in your specific business scenario. Sometimes simple techniques may work better.

Kafka, RocketMQ, RabbitMQ, etc. What are their advantages and disadvantages? Follow the public account “Hugh’s Whiteboard” for the answer in our next post.