Moment For Technology

You are out of your depth about Rabbitmq's advanced features

Posted on Nov. 27, 2023, 10:47 p.m. by Navya Sastry
Category: The back-end Tag: The back-end rabbitmq

The author xiaosan is a full stack engineer who just graduated from the university. The technical article written is basically made up of notes during the learning process. If you like it after reading it, you can give a little praise to my little brother. Exception little brother also has a programmer exchange group, welcome everyone to touch fish. Click add group

RabbitMQ advanced features - Message reliability delivery + consumption

So what is the reliable delivery of messages? Have you ever considered that when producers send data to Rabbitmq, messages can get lost in the process due to network problems and so on?

Let's start with a simple consumption flow chart:

In the figure above you can think of:

1. Messages from the Producker to the broker are lost on their way in.

2. After the message is sent to MQ, it is stored in memory but has not had time to process the message because the data in memory is lost due to the outage.

After consuming the data, the consuming process hangs up before it can be processed, causing MQ to assume that the message has been processed when it has not.

For the above reasons, we need to ensure that messages can be successfully received to the MQ node. How can we ensure message reliability?

The rabbitMQ message delivery path is producer - switch - queue - consumer, with two points of control to ensure reliable delivery of messages.

ConfirmCallback the producer to the switch via confirmCallback is disabled by default. When this mode is enabled, the producer sends a message to the switch, which sends an acknowledgement message to the consumer when it receives the message

ConfirmCallback is called regardless of whether the message successfully reaches the switch, whereas returnCallback is only triggered when the switch fails to reach the queue. When the callback function is called, it indicates that the switch message did not reach the queue smoothly.

A tip: With confirmation enabled to ensure accurate delivery of messages, rabbitMQ's overall efficiency and throughput will be severely reduced due to frequent confirmation interactions. So it's really not advisable to use message confirmation for messages that are not very important

Rabbitmq message reliability confirmCallback actual combat

The producer sends a message to the switch, and if the Broker receives the message, it sends an ACK to the producer. ACK is like a feedback message. The producer can use this ACK to confirm whether the message has been properly sent to the Broker.

Turn on the confirmCallback method

In older versions, the ConfirmCallBack interface could be implemented to send a message to the Exchange and trigger a callback after confirming that the message was sent successfully

Copy the code

In the new version, NONE is the default for disabling publish confirmation mode, and a value for CORRELATED is a callback method that is triggered when a message is successfully published to the exchange

Copy the code

We practice

@Autowired private RabbitTemplate template; @Test void testConfirmCallback() { template.setConfirmCallback(new RabbitTemplate.ConfirmCallback() { /** * * @param CorrelationData Configure * @param ack whether the switch receives a message. True indicates success. False Yes No * @param cause Cause of the failure */ @override public void confirm(CorrelationData CorrelationData, Boolean ack, String cause) { System.out.println("confirm====="); System.out.println("confirm====ack="+ack); System.out.println("confirm====cause="+cause); TODO}}); Template. convertAndSend(rabbitmqconfig. EXCHAN GE_NAME+,""," new order 1"); }Copy the code

If you need to simulate an exception, you need to change the delivery switch every time! So much for simple practice

RabbitMQ dead letter queue +TTL

What is TTL?

TTL means time to live The duration of a message. If a message remains in the queue unconsumed and has outlived its lifetime, the message is labeled "dead letter" (more on that later) and cannot be consumed later. There are two ways to set THE TTL: the first way is to configure the TTL for a single message, the second way is to configure the TTL for the entire queue, and the second way is mostly used. With TTL out of the way, let's introduce dead-letter queues

What is a dead letter queue?

Conceptually, a "dead letter" is a message that cannot be consumed. Generally speaking, the producer delivers the message to the broker or queue, and the consumer retrieves the message from the queue and consumes it. This is fine, but at some point some messages in the queue will not be consumed for some special reason, and if they are not processed, they will become dead letters, and a dead letter queue is a dead letter queue.

Rabbitmq can produce dead letters for several reasons:

  • The message was rejected and requeue is false
  • The keepalive time of the message expired
  • The maximum length of the queue reached (the queue is full and there is no way to add data to MQ)

Rabbitmq dead letter switch

When a message becomes dead letter, in order to prevent the loss of the message, the unprocessed message is sent to a dead letter switch, and then sent to the corresponding dead letter queue through the routing key bound to the dead letter switch. When a message becomes dead letter in a queue, if a dead letter queue is configured, it is re-published to a dead letter switch. The private switch sends the dead letter to the queue, forming a dead letter queue.

Test the message TTL on the RabbitMQ console

Set expiration x-message-TTL for the entire queue, and expiration parameters for the message expiration time (if the header message of the queue does not expire, the messages in the queue will expire and remain in the queue)

Web console testing for RabbitMQ

Create a dead letter switch, this and ordinary no difference

Create a dead letter queue

Bind dead letter switches to queues

Create a normal queue and set the expiration time and the specified dead-letter switch

【 答 案 】RabbitMQ core skills interview

1. Interviewer: How do you understand hosting in RabbitMQ?

A vhost can be thought of as a virtual mini-rabbitMQ queue with its own queue, exchange, binding and so on. It also has a separate permission system to control the range of users in the vhost and to separate more users from each other. For logical isolation of different service modules, a VirtualHost can have one or more exchanges and queues, but a VirtualHost cannot have exchanges or queues of the same name

2. Interviewer: How do you choose which queue products in the project?

The main message queues are ActiveMQ, Kafka, RabbitMQ and RocketMQ

ActiveMQ: Apache produced, has a long history, supports a variety of language client and protocol, supports a variety of languages Java,.net, C++, etc., based on JMS Provider implementation

Disadvantages: low throughput, performance degradation when there are many queues, there is the situation of message loss, less large-scale use

Kafka is an open source stream processing platform developed by the Apache Software Foundation, written in Scala and Java. Kafka is a high-throughput component

Distributed publishing and subscription message system, which can process all action flow data (web browsing, searching and other user actions) in large-scale websites, copy set mechanism, realize data redundancy, protect data as far as possible not to lose; Supports multiple producers and consumers similar to MQ and provides simple MQ functions

Disadvantages: Does not support batch and broadcast messages, difficult operation and maintenance, few documents, need to master Scala

RocketMQ: Alibaba open source message middleware, pure Java development, high throughput, high availability, suitable for large-scale distributed system applications

Features, strong performance (zero copy technology), support mass accumulation, support the specified number and interval of failure message retransmission, support consumer end tag filtering, delayed messages, etc., in ali for large-scale use, suitable for e-commerce, Internet finance and other fields, based on JMS Provider implementation

Cons: Relatively quiet community, fast updates, pure Java support

RabbitMQ is an open source AMQP implementation written in Erlang. It supports Python, Ruby,.NET, Java, AND C for storing and forwarding messages on distributed systems. It is easy to use, scalable, and highly available

Disadvantages: Using Erlang development, difficult to read and modify the source code

3. Interviewer: Explain how to avoid double spending

Any message queue product does not guarantee message non-repetition. If your business needs to guarantee strict message non-repetition, you need to de-duplicate the message on the business side. Kafka, RocketMQ, RabbitMQ, etc., are the same.

Using redis


TODO Boolean flag = jedis.setnx (key); If (flag){// consume}else{// ignore, repeat consume}Copy the code

Incr atomic operation: the key increases automatically. If the returned value is greater than 0, it indicates that the consumption is too high

int num = jedis.incr(key); If (num == 1){// consume}else{// ignore, consume again}Copy the code

The above two methods can be used, but the atomic problem can not be considered for rearrangement. The expiration time needs to be set for the large amount of data, and the atomic problem can be considered

About (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.