preface

We often use asynchronous messages in our work, using two main message patterns:

  • The message queue
  • Publish/subscribe

Message queue: Multiple producers can send messages to the same message queue, but a message can only be consumed by one consumer.

Publish/subscribe: A message can be concurrently retrieved and processed by multiple subscribers.

Both Kafka and RabbitMQ satisfy these features, so how do we choose which one to use? What is the difference between the two MQS? In what circumstances is Kafka appropriate and in what circumstances RabbitMQ appropriate? Do you have such doubts? I hope this article will help you.

How to choose?

Development of language

Kafka: Scala, supports custom protocols.

RabbitMQ: Erlang, supporting protocols such as AMQP, MQTT, and STOMP.

Delays in the queue

If you have the following requirements scenario:

  • 60 seconds after the order is generated, text message is sent to the user.
  • Recall push for users who have not logged in for 7 days.
  • 15 minutes after placing the order, the order is closed without payment.

Choose RabbitMQ, x-delayed-message is available right out of the box.

Message ordering

If your requirement scenario is to ensure that the messages are in order, for example, if the messages passed are MySQL binlog, the messages should not be corrupted.

Choose Kafka, which ensures that all messages sent to the same topic partition are processed in sequence.

Priority queue

If you have a requirement scenario where you need to prioritize message execution, for example, you need to deal with VIP customers first, and then regular customers.

Select RabbitMQ and set X-max-priority when creating queues.

A message left

If your requirement scenario is not to delete messages immediately after consumption but to keep them for a while longer.

Select Kafka, it can set a timeout for each topic, as long as the message does not reach the timeout will remain, rest assured that Kafka performance does not depend on the size of the storage, it stores messages in theory has little impact on performance.

The message filter

If your requirement scenario is to filter the received messages with certain filtering rules.

Choose RabbitMQ because it supports message routing. In the case of Kafka, however, there are other ways to do this.

Scalable line

If your requirements scenario is one that has huge scaling and throughput requirements.

Select Kafka.

summary

This article is purely to introduce jade, if you have any questions, welcome criticism.

Hope to be able to bring you some ideas on the choice of both.

Recommended reading

  • The ultimate consistency implementation scheme for distributed transactions
  • Understanding distributed transactions
  • Answer two frequently asked code writing questions
  • How do I write Git Commit message?