This article has participated in the third “topic writing” track of the Denver Creators Training Camp. For details, check out: Digg Project | Creators Training Camp third is ongoing, “write” to make a personal impact.

From the perspective of producers, MQbrokers and consumers, RabbitMQ is introduced in detail to ensure that messages are delivered reliably and not lost.

Message loss occurs when a producer loses a message during transmission, when RabbitMQ fails to persist a message, or when a Consumer fails to consume a message and commits a message. In fact, all message queues, such as Kafka and RocketMQ, rely on these three aspects of message reliability.

1 Producer Reliability

Normally, messages can be persisted if they are queued through the exchange, but if an accident occurs before the message reaches the broker, the message is lost, which is producer data loss.

RabbitMQ provides transaction messages and confirm confirmation modes to ensure that messages are not lost.

1.1 Transaction Mechanism

The implementation of the transaction is mainly the setting of channels, which are called before and after the message is sent:

Channel.txselect try {// send a message} catch (Exception e) {channel.txrollback } // The transaction is completed, and the transaction channel.txcommit is committedCopy the code

The transaction mechanism ensures that producer messages reach the RabbitMQ server successfully, but uses synchronous blocking mechanism, which has poor performance and severely affects throughput.

1.2 confirm mechanism

Transactions reduce RabbitMQ’s message throughput, so is there a more efficient way to do this? B: Yes, of course. That is to use Confirm mode. Note that message acknowledgements and transactions cannot be used together.

By using channel.confirmSelect() before sending the message; Enable the Confirm mode.

After the message is sent, Confirm can wait for the message confirmation. Confirm can be confirmed in three ways:

channel.waitForConfirms(); // Normal sender confirmation mode; channel.waitForConfirmsOrDie(); / Batch confirmation mode; channel.addConfirmListener(); Async listens for sender confirmation mode, which can be added before messages are sent.Copy the code

The greatest benefit of the asynchronous Confirm mode is that it is asynchronous. Once a message is published, the producer application can continue to send the next message while waiting for the channel to return an acknowledgement. When the message is finally acknowledged, the producer application can process the acknowledgement via a callback method.

When asynchronous confirm mode is enabled, each written message is assigned a unique sequence number (starting from 1), which can be obtained using the channel#getNextPublishSeqNo method before the message is sent. Once the message has been posted to all the matching queues, The broker sends an ACK to the producer (including the sequence number of the message), which lets the producer know that the message has reached the destination queue correctly. If the message and queue are persistent, the acknowledgement message will be sent after the message is written to disk.

If RabbitMQ fails to process the message, a nACK message will be sent. The producer application can also process the NACK message in the callback method, which we can retry.

Channel.confirmselect (); // Enable confirm mode channel.confirmSelect(); // async confirm, Register the listener channel. AddConfirmListener ((sequenceNumber, multiple) - > {} / / confirmation message code, (sequenceNumber, Multiple) -> {// code for unacknowledged messages});Copy the code

In general, asynchronous confirm is used the most.

2 RabbitMQ reliability

To prevent RabbitMQ data loss, RabbitMQ persistence is usually enabled.

So how to persist, there are two steps:

  1. The durable property of a queue is set to true when it is durable. This ensures that RabbitMQ will persist the metadata of the queue, but it will not persist the data in the queue.
  2. When a producer sends a message, the deliveryMode of the message is set to 2, which means the message is persistent, and RabbitMQ will persist the message to disk.

After this two-step setup, RabbitMQ can be suspended and data can be recovered upon restart.

However, before the message is persisted to the hard disk, the service may have died, while the message is still in memory, or it may be lost. This allows persistence to be used in conjunction with confirm, and if enabled RabbitMQ will send an Ack to the producer after the message persists to the disk. This way, if rabbitMQ dies before the message persists to disk, the producer will not receive the Ack signal and can resend it.

It is also a good idea to set Exchange persistence. The durable property of Exchange is set to true. If Exchange is not persisted, the reliability of messages will not be affected. However, if Exchange is not persisted, when the broker is restarted, Exchange will not exist and the producer rabbitMQ producer will not be able to send messages.

3 Consumer Reliability

Enabling manual confirmation solves this problem:

  1. Automatic confirmation mode: After a message is sent to the consumer, it is confirmed immediately. However, the message may be lost. For example, the server hangs up just after the message is sent to the consumer, and the data is lost.
  2. Manual confirmation mode, if the consumer has no time to deal with the hang, no response ACK will repeatedly send a message to other consumers; If the listener handles an exception and does not catch the exception, it keeps receiving the message repeatedly, and then keeps throwing the exception. If an exception is caught but not ack in finally, the message will always be sent repeatedly (retry mechanism).
  3. Unacknowledged mode, acknowledge=” None “No acknowledgement mechanism is used. Once a message is sent, it is immediately removed from the queue. Regardless of whether the client is abnormal or disconnected, the message is removed after being sent without resending.

Configure the attribute acknowledge= “manual” in

for manual acknowledgement. NONE indicates no acknowledgement, and AUTO indicates automatic acknowledgement.