reference

Rabbitmq.mr-ping.com/

The RabbitMQ documentation in Chinese www.kancloud.cn/yunxifd/rab…

The overall architecture

The main application

1. Message communication

The main function of message queue is to send and receive messages. It has an efficient communication mechanism inside, so it is very suitable for message communication.

2. Asynchronous processing

Using message queues for asynchronous processing can return results faster, speed up the response of the server, and improve the performance of the server.

3. Service decoupling

A message queue is introduced between application A and application B to decouple services. If application B fails, application A’s services are not affected.

4, flow peak cutting

By introducing message queue, the burst traffic can be effectively buffered and the effect of “peak cutting and valley filling” can be achieved.

The core concept

Producer: Applications that send messages.

Consumer: An application that receives messages.

Queue: A cache that stores messages.

Message: a Message sent by a producer to a consumer via RabbitMQ.

Connection: TCP Connection between RabbitMQ and the application server.

Channel: A virtual Channel within a connection. When you send or receive a message on a message queue, the operation is done over a channel.

Exchange: The switch is responsible for receiving messages from producers and distributing them to the corresponding message queue based on the Exchange type. To receive messages, a queue must be bound to a switch.

Binding: A Binding is an association between a queue and a switch.

Five message sending modes for RabbitMQ

RabbitMQ includes five queue modes: simple queue, work queue, publish/subscribe, route, topic, RPC, etc.

A simple model

  • There is only one producer, one consumer, and one queue.
  • To send and receive messages, producers and consumers only need to specify the queue name, not the Exchange to which the messages are sent. The RabbitMQ server automatically uses the default Exchange of the Virtual host, which is of direct type.

Working mode

  • There can be multiple consumers, but a message can only be retrieved by one consumer.
  • Messages sent to queues are distributed equally by the server to different consumers for consumption.

Publish/subscribe

  • In publish/subscribe mode, the type of Echange is fanout.
  • When a producer sends a message, it does not need to specify a queue name. Exchange forwards the received message to the bound queue.
  • Messages are forwarded to multiple queues by Exchange, and a message can be retrieved by multiple consumers.

Routing patterns

  • In routing mode, the Exchange type is direct.
  • The destination queue of messages can be specified by the producer according to the routingKey rule.
  • Consumers bind their interested queues with BindingKey.
  • A message team can be retrieved by multiple messages.
  • Only queues with RoutingKey matching BidingKey will receive messages.

The theme mode

  • Words can be used to match multiple queues
  • Topic Mode The value of Exchange type is topic.
  • A message can be retrieved by multiple consumers.

Switch Type

Default switch

The default Exchange is actually a direct exchange with no name (the name is an empty string) pre-declared by the message broker. It has a special property that makes it especially useful for simple applications: every new queue is automatically bound to the default switch with the same routing key name as the queue name.

For example, when you declare a queue named “search-indexing-online”, the AMQP proxy will automatically bind it to the default switch with a routing key named “search-indexing-online”. Therefore, when a message is sent to the default switch with a routing key named “search-indexing-online”, the message is routed by the default switch to a queue named “search-indexing-online”. In other words, the default switch looks as if it can deliver messages directly to the queue, even though it doesn’t technically do that.

Direct switch

A direct exchange sends a message to a queue based on the routing key carried by the message. A direct switch is used to handle unicast routing for messages (although it can also handle multicast routes). Here’s how it works:

  • Bind a queue to a switch and give the binding a routing key.
  • When a message with a routing key of R is sent to a directly connected switch, the switch routes it to a queue with the same binding value of R.

Direct switches are often used to circulate tasks to multiple workers. When doing so, we need to understand that in AMQP 0-9-1, the load balancing of messages occurs between consumers, not between queues.

Fan switch

A funout exchange routes messages to all queues bound to it, regardless of the bound routing key. If N queues are bound to a fan switch, when a message is sent to the fan switch, the switch will send a copy of the message to all N queues. The fan type is used by the switch to process the broadcast routing of messages.

Because a fan switch posts copies of messages to all queues bound to it, its application cases are very similar:

  • Massively multiplayer online (MMO) games can use it to handle global events such as leaderboard updates
  • Sports news sites can use it to distribute score updates to mobile clients in near real time
  • Distribution systems use it to broadcast various status and configuration updates
  • In group chats, it is used to distribute messages to users who participate in group chats. (AMQP doesn’t have a presence concept built in, so XMPP might be a better choice)

Topic switch

Topic switch

Topic exchanges route messages to one or more queues by matching messages’ routing keys and queue-to-switch binding patterns. Topic switches are often used to implement various distribution/subscription patterns and their variants. Topic switches are usually used to implement multicast routing of messages.

Topic switches have very extensive user cases. Topic switches can be considered whenever a problem involves multiple consumers/applications that want to be targeted to receive messages.

Use cases:

  • Distribute data about specific geographic locations, such as points of sale
  • A background task performed by multiple workers, each of whom is responsible for some specific task
  • Stock price updates (and other types of financial data updates)
  • News updates involving categories or tags (for example, specific sports or teams)
  • Coordination of different kinds of services in the cloud
  • Distributed architecture/system-based software encapsulation in which each builder can work on only one particular architecture or system.

The first switch

Sometimes the routing operation of a message involves more than one attribute, which is easier to express using headers than routing keys. This is why a Headers exchange was born. A header switch uses multiple message attributes instead of routing keys to establish routing rules. Routing rules are established by determining whether the value of the message header matches the specified binding.

We can bind a queue to the switch and give the bindings between them multiple matching headers. In this case, the message broker needs to get more information from the application developer, in other words, it needs to consider whether a message needs to be partially matched or fully matched. The “more messages” is the “X-match” parameter. When “X-match” is set to “any”, any value of the header is matched. When “X-match” is set to “all”, all values of the header are matched successfully.

A head switch can be considered another representation of a direct switch. A header switch can work just like a direct switch, except that the routing rules for a header switch are based on the value of the header attribute rather than the routing key. The route key must be a string, while the header attribute values are free of this constraint, and can even be integers or hashes (dictionaries), etc.

Queue persistence

Durable queues are stored on disk and survive when the message broker restarts. Queues that are not persisted are called Transient queues. Not all scenarios and cases require persistence of queues.

A persistent queue does not make messages routed to it persistent. If the message broker dies and is restarted, the persistence queue is redeclared during the restart, and only persistent messages can be restored anyway.

The binding

A Binding is a rule that exchanges follow to route messages to queues. If switch “E” is instructed to route messages to queue “Q”, “Q” needs to be bound to “E”. The binding operation requires defining an optional routing key attribute for certain types of switches. The meaning of the routing key is to select some messages from the many messages sent to the switch and route them to the bound queue.

Message to confirm

Consumer applications — apps that receive and process messages — occasionally fail or crash while processing messages. And network reasons may also cause all kinds of problems. This poses a difficult question: when is it correct for an AMQP agent to delete a message? The AMQP 0-9-1 specification gives us two recommendations:

  • Delete messages as soon as the message broker sends them to the application. (Use the AMQP method: basic.deliver or basic.get-ok)
  • Delete the message after the application sends an acknowledgement. (Using the AMQP method: basic.ack)

The former is known as the automatic Acknowledgement model and the latter as the explicit Acknowledgement model. In the explicit mode, it is up to the consumer application to select when to send the acknowledgement. An application can send a message as soon as it receives it, store an unprocessed message and send it, or wait until the message is processed before sending a confirmation receipt (for example, after a web page content has been successfully retrieved and stored).

If a consumer hangs without sending a confirmation receipt, the AMQP broker redelivers the message to another consumer. If there are no consumers available at that time, the message broker will wait for the next consumer to register to the queue and try again.

Message queue transaction

The implementation of transactions is mainly the setting of channels, the main methods are as follows:

The sender:

  • Channel.txselect () declares the transaction mode to be started
  • Channel.txcommit () commits the transaction
  • Channel.txrollback () rolls back the transaction

If the transaction is confirmed and the transaction is switched to the consumption queue for consumption, the rollback() method is used to rollback the transaction

Consumer:

The consumer must use manual validation. Automatic validation transactions do not take effect

supplement

Problem of repeated consumption

  • Use a unique key to determine if it has been consumed
  • Use message ID + Redis to solve the double consumption problem
  • Use hash()+redis to resolve double consumption issues