Make writing a habit together! This is the fifth day of my participation in the “Gold Digging Day New Plan · April More text Challenge”. Click here for more details.

Message sending model

Since RocketMQ is a message queue, if we were to think about it as a designer, I would definitely design a mechanism for storing messages:

  • To ensure that incoming messages are consumed first, it is natural to use data structures such as queues;
  • Referring to Redis’s send subscription model, consumers can realize targeted delivery by subscribing to messages under topics.
  • In addition, to improve the reliability of MQ, we will design some redundant policies to ensure that messages are not lost.

From a storage model perspective, there are three important concepts in MQ:

  • Broker: a Server that accepts connections from clients and implements services
  • Topic: A business identifier used for categorization, which can be understood as a Topic in the Redis publish-subscribe model
  • Queue: Also known as Message Queue, a Message Queue that stores messages and forwards them to consumers

In addition, you can see from the picture above:

  1. Messages sent by users are recorded in a Queue in the Broker
  2. Data within a Topic may be stored in multiple brokers
  3. A Broker holds multiple queues
  4. Queues within a Topic may be redundantly backed up across multiple brokers


Message Sending Process

Ok, now that we have solved the problem of how to store the messages, it is time to design the specific flow of sending them. If you are a designer, I suggest that you don’t get bogged down in the implementation details at the outset, but rather that you move from the abstract to the concrete, drawing out the overall process first and improving it gradually.

With this in mind, and with reference to Redis’ send subscription model, we can try to sketch the RocketMQ message delivery process:

Isn’t that easy? In the process of sending a message to receiving, the sender first sends the message to a Topic, and then the consumer goes to the Topic to get the message.


Now that we have the simplest process for sending a message, we can draw a more detailed send/receive model in combination with the Broker that stores the message we drew earlier:

In fact, RocketMQ also allows message senders to send messages for multiple topics, and a Topic can be subscribed to by multiple consumers:

Message routing (load balancing) : In the previous storage model, we designed the message to be stored in the Queue of the Broker, the Queue to which the message should be sent, and the Queue from which the consumer gets the message.

To solve this problem, we can introduce routing logic for selection, which can also be understood as load balancing:





Consumption message model

After the analysis and design of message sending model, it is not difficult to find that consumers should be able to support two modes: broadcast message and one-to-one message when consuming messages.


Broadcast messages

The so-calledBroadcast messagesA message can be consumed by multiple consumers, as shown below:


One on one message

One-to-one messaging does not really refer to a machine-to-machine relationship, but now more of a topic-to-cluster relationship.

For example, if you have A Topic, Topic A, that has four queues, and A Consumer group that has four consumers, the four queues in that Topic can be consumed by A Consumer in that Consumer group by A certain rule.


There are four types of allocation rules:

1. Divide it evenly Notice that the average here is not strictly average


2. Consistent hashing According to theConsumerIn the order ofQueueConsists of the distribution in the ring diagram.


3. The equipment room is assigned nearby

For example, we have several rooms in Hangzhou, Beijing and Guangzhou. In order to improve performance, user requests will be routed to the nearest server by CDN. MQ is the same: