preface
“Middleware” is a combination of “middle” and “ware”, meaning “something in the middle”.
In computers, middleware is software that sits between two pieces of software, generally between application software and system software.
Taobao, agent visa such business, but no matter the Internet or face to face sign, all visa processing agencies should be open to you, and can save direct handling and agent price difference, but such business is still very popular.
Real estate transactions. The state’s real estate departments are entirely open to individuals, but some agencies are still heavily acting as hubs for strangers’ real estate transactions and earning commissions.
In e-commerce, Alipay and Qantong act as the intermediary between people and merchants, making transactions more secure and fast.
What commonalities can be abstracted from these three things? I will try to describe a few characteristics: professional, safe, fast, low cost, they are the “middleware” in our life.
If we think of the visa center, the real estate exchange and the merchant as the operating system, each of us is equivalent to the application layer software. Among them taobao platform, real estate transaction intermediary, as well as Alipay, wealth management is the middle of the software.
The message queue
Before we get to RabbitMQ, let’s talk about a Message queue (MQ), which is literally a container for holding messages. So why do we need such a container? It’s really about decoupling the systems.
Using efficient and reliable message transmission mechanism for asynchronous data transmission, and based on data communication for distributed system integration. Interprocess communication can be extended in a distributed environment by providing a message queue model and messaging mechanism.
Let’s take an example:
Here is A simple scenario where system A is responsible for generating the userID and calling systems B and C. If system BC frequently changes whether the userID parameter is required, then system A’s code will have to constantly change, and if system DEF comes again one day… If this parameter is also required, A lot of business logic will be added to system A. In this way, it is easy for other systems to interact with each other. In addition, A large number of systems interact with A, which is also easy to cause problems.
After adding the message queue, A is only responsible for generating the userID. Who uses this parameter and how? System A doesn’t care. The system that is interested in this data can take it by itself and realize decoupling between each system. Moreover, after decoupling, the whole service industry becomes an asynchronous mode. After system A generates data, it does not need to call BCD successively to accumulate time, but each system can simultaneously access the data of message queue for processing and increase throughput.
Message queue characteristics
1. First-in, first-out: The order of the message queue has been basically determined at the time of queue entry, generally without manual intervention.
Publish and subscribe: Publish and subscribe is a very efficient way of processing, if not blocking, can be treated as a synchronous operation.
3. Persistence: Persistence ensures that message queues are not used as an auxiliary tool for part of the scenario, but as a way to store core data like a database.
4, distributed: in the use of large traffic, big data scenarios, support distributed deployment, can be widely used. Message queue localization is a high-performance middleware.
Four patterns of messaging middleware
Respectively is:
- Inhibits PTP model
- The Pub/Sub model
- Partition model
- Transfer model
1 inhibits PTP model
Point-to-point communication model. PTP is queue-based, and a Queue can have multiple producers and consumers. The message server queues messages in the order in which they are received. Each message in the queue can only be consumed by one consumer, after which it is removed from the queue.
Features:
- Only one consumer per message;
- The sender and receiver are not time dependent;
- The receiver confirms that the message was received and processed successfully.
P represents producer, C represents consumer, and red represents queue.
2 Work queue mode
Working mode: one message producer, one exchange, one message queue, multiple consumers. Also known as point-to-point mode
If we have two consumers, by default RabbitMQ will send each message in order to the next consumer, and on average each consumer will receive the same number of messages. This method of message distribution is called polling.
If you have some very time-consuming task that one consumer is working on slowly while another is idle, it is obviously very resource-intensive.
Let me give you an example
A 1-year old programmer is given the same amount of tasks as a 3-year old programmer. Obviously, the 3-year old programmer is much easier to handle and will soon be idle, but the 3-year old programmer gets a very high salary! Obviously the three-year programmer should take more responsibility, so what?
Fair distribution
The reason for the problem in this case is that RabbitMQ receives the message and distributes it immediately without confirming the number of unacknowledged messages returned by each worker
So we can use the basicQos method and set prefetchCount to 1 to tell RabbitMQ that I am processing one message at a time and that you should wait until I am finished before sending the next one to me. RabbitMQ will not distribute in turn, but will look for free workers to distribute.
3 the Pub/Sub model
Publish-subscribe model. In the Pub/Sub model, a producer publishes a message to a Topic, and all downstream consumers that subscribe to that Topic can receive the message.
Features:
- Each message can have multiple subscribers;
- The client must subscribe to receive the message;
- Persistent and non-persistent subscriptions.
Note:
- Publishers and subscribers are time-dependent: recipients and publishers can only receive messages if they have established a subscription relationship;
- Persistent subscriptions: Once a subscription is established, messages do not disappear, whether the subscriber is online or not;
- Non-persistent subscriptions: Subscribers must always be online in order to receive messages. Approximately equal to point-to-point mode when there is only one subscriber
P represents the producer, X represents the switch, C1C2 represents the consumer, and red represents the queue.
Normally, a message needs to be consumed only once, so when do all consumers need to consume this message? Typically, data needs to be cached in memory and updated in real time.
For example, the author has made a prohibited word system to detect prohibited words in user input comments. The contraband word system is deployed on N servers. To improve detection performance, each machine loads the contraband word library into memory, and updates the dictionary by sending MQ messages. Since the Pub/Sub model is adopted, the consumer on each machine can receive this message and directly update the sensitive lexicon in memory.
3 Routing Mode
The routing mode is similar to the publish-subscribe mode, but the type is added to the subscription mode. The subscription mode is distributed to all queues bound to the switch, and the routing mode is only distributed to queues bound to the specified routing key on the switch
P represents the producer, X represents the switch, C1C2 represents the consumer, and red represents the queue
For example, if we send a message to Exchange with routingKey=”error”, the message will be routed to Queue1 (amqp.gen -s9b… This is the Queue name automatically generated by RabbitMQ) and Queue2 (amqp.gen-Agl…). . If we send a message with routingKey=”info” or routingKey=”warning”, the message will only be routed to Queue2. If we send messages with other Routingkeys, the messages will not be routed to either Queue.
In contrast to the publish-subscribe model, we can see that instead of receiving all messages as broadcast, we can see selective consumption.
4 Theme Mode
Topics mode is similar to Routing mode, except that routing mode specifies a fixed routingKey, while topic mode fuzzy matches the routingKey, similar to SQL = and like.
P represents the producer, X represents the switch, C1C2 represents the consumer, and red represents the queue
Common message queue comparison
1.Kafka
Kafka is LinkedIn’s open source distributed publish-subscribe messaging system, currently part of the Apache Top-level project. Kafka is designed primarily for high-throughput subscription publishing systems with a focus on speed and persistence. Kafka messages are composed of keys, values, and timestamps. Kafka does not record who is using each message. Kafka only records which messages are unread by offsets.
2.RabbitMQ
RabbitMQ is an open source message queue system based on the AMQP protocol developed using the Erlang language. The main characteristics of AMQP are message orientation, queue, routing (including point-to-point and publish/subscribe), reliability, and security. AMQP protocol is more widely used in enterprise systems, which requires high data consistency, stability and reliability, and requires less performance and throughput.
3.RocketMQ
RocketMQ is alibaba’s open source messaging middleware. It is developed in pure Java and features high throughput, high availability and suitable for large-scale distributed system applications. RocketMQ originated from Kafka, but is not a Copy of Kafka. It optimizates the reliable transmission and transactional of messages. Currently, It is widely used in Alibaba Group in transactions, recharge, stream computing, message push, log stream processing, binglog distribution and other scenarios. There are not many client languages supported, currently Java and C++, among which C++ is not mature.
