In RabbitMQ you will ask me a little story called “interview with Wang” because of his technical anxiety. You will ask me a little story via RabbitMQ. It is expected that the reader will have a clear understanding of the core concepts of RabbitMQ, the common message model, how to ensure that messages are reliable, how to ensure that messages are not consumed repeatedly, and the sequential consumption of messages. This will provide a framework for RabbitMQ learning and will be the focus of RabbitMQ interviews. I hope I can help those of you who are learning RabbitMQ, and those of you who will be interviewing soon!
1. Technology anxiety
Xiao Wang has been working in the company for nearly two years. When he first joined the company, he was very excited. He was able to apply the Java knowledge he had learned to practical software development problems. When it comes to the associative query between the businesses of 3 tables, I have no idea. Every time at this point, Xiao Wang sighs: Ah, the business of the company is too complicated. He thought to himself: The company’s project uses Spring+Hibernate, the previous architect has built the whole framework, the configuration file is using the XML file configuration method learned during school, now there are all SpringBoot projects outside, feel the technology here is backward, I am very anxious. To think, small wang decided to find big list to solve problems.
Xiao Wang, I haven’t seen you for a long time, and I haven’t sent you any news in the circle of friends. What are you busy with recently?
Wang: I feel I have developed a sense of anxiety about technology.
Hahaha, “anxiety of technology” is a good word, I’m a little curious, what do you mean by technology?
Xiao Wang: Well, THESE days, I have summarized my two years’ work in the company. Although I have learned something, the software development technology is growing very slowly. And if you want to stay with this company for the long term, you have to understand the business. I sometimes observe colleagues who have been at the company for 5 + years. They don’t have great skills, but they know the business, they build on the framework that the architects put together, they work well, they leave work early and go home to take care of the kids. Our colleagues who have just been here for a year or two need to work overtime to complete complicated tasks because they are not familiar with the business.
Big list: listen to you so say, if can boil these old employee walk, estimate you be the employee that the company understands business most, the time leadership must esteem you boy.
Xiao Wang: Ha ha ha, yeah. If I get to that time, I will also be 35 years old, although the technology has not grown much, but the business of this industry should be thorough, and I can make a living in this business at that time.
Yes, we programmers, if the technical strength is not so good, but if we do in-depth understanding of a certain area of business, we can also make a world.
Wang: You have no problem. But sometimes I wonder if I’m too young to spend more than 10 years in development in this business. I want to learn more new technologies and expand my technology stack. Last month, we discussed RabbitMQ messaging middleware.SpringBoot is integrated with RabbitMQ to send and receive messages (code to run)I’ve been working on RabbitMQ technology after work.
Good boy, use business time to recharge, have the good habit of good programmers.
Wang: Good for you, Bang brother. How about this, you test my understanding of RabbitMQ and test the results of a month or so of learning. You as the interviewer, I as the candidate, let’s start talking.
2, RabbitMQ you ask me answer
Candidate: Hello, I am the senior software development siege lion of XXX company, welcome to attend the interview. Would you like to introduce yourself first?
Wang: Bang brother has talent as an actor. He can change roles quickly! Younger brother admire! But we’ve been friends for over ten years, so why don’t we just talk about RabbitMQ and test my understanding of RabbitMQ?
Big list: ha ha ha, want to temporarily act as the interviewer, enjoy the addiction. I see on your resume that you are familiar with RabbitMQ messaging middleware and have experience in messaging middleware development. What is the concept of RabbitMQ?
Wang: RabbitMQ is an open source middleware based on the advanced messaging protocol AMQP and written in Erlang.
2.1 Differences between RabbitMQ and other messaging middleware
Why does your company use RabbitMQ as its messaging middleware? Is it possible to use other message-oriented middleware?
Xiao Wang: Interviewer, I chose RabbitMQ as the company’s messaging-oriented middleware after comparing the characteristics of ActiveMQ, RabbitMQ, RocketMQ and Kafka after a period of technology selection. Companies use RabbitMQ to decouple and communicate asynchronously between service modules to improve system response time.
Is it feasible to use other messaging middleware? This is the connection and difference between RabbitMQ and other message queues. Currently, ActiveMQ, RabbitMQ, RocketMQ and Kafka are mainly used in the industry. First, Kafka’s business scenario focuses on log collection in the big data direction. Kafka is not used because this part of the company’s business is not log collection.
For ActiveMQ, it was early projects that used ActiveMQ, but the throughput was low relative to RabbitMQ and the community was not active.
For RocketMQ, it is alibaba open source. Currently, Alibaba does not maintain it. Generally, large software companies (such as Bytedance) do customized development on the basis of open source, requiring the company to be strong in infrastructure research and development.
For RabbitMQ, throughput is good, usability is good, the client interface is simple and useful, the community is active, and problems can be solved by the open source community. RabbitMQ was chosen as the consumer middleware because of the disparity between the company’s r&d capabilities and those of large software companies. So it is possible to use other message-oriented middleware, but RabbbitMQ is easier to maintain than other message-oriented middleware.
Good answer compared to other messaging middleware, it seems that your company used RabbitMQ very carefully. What are the advantages and disadvantages of RabbitMQ as a messaging middleware? Can you give me a scenario?
Wang: The main advantages of RabbitMQ are peak clipping, decoupling and asynchronous communication.
When all users click to buy something at the same time, a huge amount of user traffic is generated. When we introduce RabbitMQ, users will queue up their purchase requests to RabbitMQ, which indirectly helps the purchase interface to limit/peak load.
For example, the user login is the main process and the User login Trace is the secondary process. These two services can be decoupled and communicated asynchronously using RabbitMQ. When user login is successful, a message is sent to the queue through RabbitMQ. The Record User Login Track module acts as a consumer and consumes this message to achieve decoupling and asynchronous communication between services. The advantages are basically these three.
2.2 Supplementary reminder from the interviewer
Yes, but RabbitMQ has another important advantage: latency. Can you tell me the specific scenario of service delay processing?
Xiao Wang: Thank you for the supplementary reminder. For businesses that need to be processed after a certain period of time, for example, the system will cancel the order if the payment expires after the purchase of goods in e-commerce. RabbitMQ can handle this delay by setting the TTL.
The main disadvantages of RabbitMQ are: reduced system availability; System complexity increases; Consistency issues.
Reduced system availability: One more middleware is introduced. If the middleware is abnormal, the system cannot run normally.
Increased complexity: With the introduction of RabbitMQ, we have to deal with message loss, repeated consumption of messages, and message ordering.
Consistency problem: for example, system A returns success after processing, thinking that the request is successful; However, the actual situation may be: in the three systems B, C and D, the write library of SYSTEM B and D succeeds, but the write library of system C fails, which causes data inconsistency.
[Interviewer’s point of view: This candidate can be very specific about the advantages of RabbitMQ and the disadvantages of introducing RabbitMQ. He has a good understanding of RabbitMQ, bonus.]
2.3 RabbitMQ core concepts and common message models
You understand RabbitMQ quite well and have spent a lot of time in private. What are the core concepts of RabbitMQ?
Wang: Core concepts, do you mean producers, messages, message models, consumers? Wang holds out a picture:
Big bang: Uh-huh. Message models, what kinds of message models are there? Where and how are they used?
Wang: The message model is made up of switches, routes and queues. We commonly use three main message models, based on FanoutExchange, DirectExchange, TopicExchange these three message models.
For the FanoutExchange message model, the application scenario is a scenario where business data needs to be broadcast, for example, “user operations are written to multiple logging systems”. Using this messaging model, we create a FanoutExchange switch and bind the queue to the switch.
For the DirectExchange message model, the application scenario is a scenario where business data needs to be transferred and consumed directly.
TopicExchange is also one of the RabbitMQ switches for the TopicExchange messaging model, which is a publish-subject-subscribe switch. On the switch, route names support wildcards (*) and (#) for fuzzy matching. * corresponds to one word, while # corresponds to zero or more words. Wang again holds out a diagram about topic switches:
2.4 how to ensure reliable transmission of RabbitMQ messages?
Big bang: Uh-huh. Now that your company has introduced RabbitMQ as its messaging middleware, how do you ensure the reliable transmission of RabbitMQ messages?
Xiao Wang: The main reason why the information is unreliable is that the information is lost. Message loss can be divided into producer loss, message model loss, and consumer loss.
1. From the point of view of producer loss, RabbitMQ provides transaction and confirm modes to ensure that messages are not lost.
For the transaction mechanism: before sending a message, start a transaction and then send a message. If an exception occurs during the sending process, the transaction is rolled back. If the transaction is sent successfully, the transaction is committed.
For the confirm mechanism, producers are required to “send confirmation” after sending the message. If the confirmation succeeds, the message has been sent successfully. If sending fails, the producer tries again.
2. From the point of view of message loss model, setting the switch, queue and message to persistent mode ensures that even if the RabbitMQ server goes down suddenly, the switch and queue will still exist and messages will not be lost after the server restarts.
3. From the point of view of the message loss, the confirmation consumption mode of the consumer is set to “no confirmation consumption”, namely NONE consumption mode. In this mode, the consumer does not need to send feedback to the RabbitMQ server after listening to the message, so the message is likely to be lost.
In AUTO mode, the RabbitMQ component sends an Ack message to the RabbitMQ server to remove the message from the RabbitMQ queue when the consumer hears it. Automatic mode: After receiving a message, the consumer will automatically reply that the RabbitMQ server has received the message before processing it. If there is a problem with the logic that processes the message, the message is lost.
Manual: Unlike automatic, the consumer processes the logic of the message and then manually writes code to notify the RabbitMQ server to confirm the purchase before the message is removed from the queue, ensuring that the consumer does not lose the message.
Interviewer’s point of view: The young man’s solution to message loss is very clear.
2.5. How to solve the problem of repeated consumption?
Yes, good understanding of the reliability of the message. Is there any repeat consumption of RabbitMQ? If so, why? How can RabbitMQ messages be guaranteed not to be re-consumed?
Wang: Repeated consumption of RabbitMQ, mainly due to network unreachable. Specifically, if a consumer sends an Ack to the RabbitMQ server after consuming a message and RabbitMQ does not receive the Ack due to a network disconnect or some other reason, RabbitMQ will not delete the Ack. After the connection is reestablished, the consumer will still consume the message, causing repeated consumption.
The solution to repeated consumption is as follows: The data written into the message queue is uniquely identified. When consuming a message, it can determine whether the message has been consumed by the unique identifier. We can also solve the problem from the business level, that is, the message is transmitted multiple times. We ensure that the multiple consumption of the message will not affect the business, that is, to ensure the idempotency of the message.
2.6 how to ensure the ordering of RabbitMQ messages?
How to ensure sequential consumption of RabbitMQ messages? For example, a piece of data that is normally added, modified, and removed from RabbitMQ is removed out of order and changed to deleted, added, and modified, causing data synchronization problems.
Wang: Let’s look at the out-of-order scenario: one producer, multiple consumers in RabbitMQ. For example, if a producer sends three packets of data1, data2, and data3 to the RabbitMQ queue, and three consumers consume one of these packets from MQ, it is possible that consumer 2 will finish first, and data2 will consume first, which is obviously confusing.
There are two ways to do this: the first is to ensure that producer-queue-consumer relationships are one-to-one, that is, a single thread is used to ensure that messages are consumed sequentially, since a single consumer corresponds to a single thread that takes data in turn from the RabbitMQ queues.
The second method: the message is numbered, and the consumer processes the message according to the number.
Wang, your answer is quite good. Sure enough, he recognized the three days and should be impressed. You have a good understanding of the core concepts of RabbitMQ, ensuring reliable transmission of messages, ensuring that messages are not consumed twice, and ensuring that messages are consumed in sequence.
Wang: I’m flattered! I’ve been learning RabbitMQ for a month or so while trying to export the results, which is a quick way to absorb the knowledge. I will also investigate the operation and maintenance of RabbitMQ and how to make RabbitMQ highly available through RabbitMQ clusters.
Good, I am also watching these days, then discuss RabbitMQ operation and maintenance, exchange experience.
Xiao Wang: It’s so nice to study with Bang Brother!
The ~