[TOC] When it comes to messaging-oriented middleware, we can tell a lot about it, such as ActiveMQ, RabbitMQ, RocketMQ, Kafka, JMS, AMQP, etc. Today, Songo is here to clean up with the boys.

1. Several protocols

Let’s start with a few common protocols in messaging middleware.

JMS 1.1

1.1.1 JMS is introduced

Let’s start with JMS.

JMS is a Java Message Service, similar to JDBC. Different from JDBC, JMS is the Message Service interface of JavaEE. There are two versions of JMS:

  • 1.1
  • 2.0.

Compared with the two, the latter mainly simplifies the code for sending and receiving messages.

Given that message-oriented middleware is a very common tool, JavaEE has a special specification JMS for this purpose.

But like JDBC, JMS as a specification, it is only a set of interfaces, does not contain a specific implementation, if we want to use JMS, then generally need the corresponding implementation, just as JDBC needs the corresponding driver.

1.1.2 JMS model

The JMS message service supports two message models:

  • Point-to-point or queue models
  • Publish/subscribe model

In a point-to-point or queue model, a producer publishes a message to a particular queue, and a consumer reads the message from that queue. Here, the producer knows the consumer’s queue and sends the message directly to the corresponding queue. This is a point-to-point messaging model, which is summarized as:

  1. Only one consumer will get the message.
  2. The producer does not need to be in a running state while the consumer consumes the message, nor does the consumer need to be in a running state when the message is sent, that is, the producer and consumer of the message are completely decoupled.
  3. Each successfully processed message is signed for by the message consumer.

The publisher/subscriber model supports publishing messages to a specific message topic, and consumers can define topics of interest. This is a point-to-point messaging model, which can be summarized as:

  • Multiple consumers can consume messages.
  • There is a time dependency between the publisher and the subscriber, and the publisher needs to create a subscription so that the customer can subscribe; Subscribers must remain online to receive messages; Of course, if the subscriber creates a persistent subscription, messages published by the message producer when the subscriber is disconnected will be republished when the subscriber reconnects.

1.1.3 JMS implementation

Open source message-oriented middleware that supports JMS are:

  • Kafka
  • Apache ActiveMQ
  • HornetQ for JBoss community
  • Joram
  • The MantaRay Coridan
  • OpenJMS

Some commercially available message-oriented middleware that supports JMS are:

  • WebLogic Server JMS
  • EMS
  • GigaSpaces
  • iBus
  • IONA JMS
  • IQManager (Acquired by Sun Microsystems in August 2005)
  • JMS+
  • Nirvana
  • SonicMQ
  • WebSphere MQ

Many of them are excavated from Songgo archaeology. In fact, Kafka and ActiveMQ may be the ones we come into contact with more in daily development.

1.2 it

1.2.1 closer profile

Another protocol associated with messaging middleware is AMQP.

Message Queue has been in demand for a long time. In the 1980s, Teknekron was first used in financial transactions by Goldman Sachs and other companies. At that time, Message Queue software was called the Information Bus (TIB). TIB was adopted by telecom and communications companies, and Reuters acquired Teknekron. IBM developed MQSeries and Microsoft developed Microsoft Message Queue (MSMQ). The problem with these commercial MQ vendors is vendor lock-in and high prices. In 2001, The Java Message Service tried to solve the problem of locking and interactivity, but it became even more troublesome for applications.

In 2004, jP Morgan chase and iMatrix began to develop an open standard for Advanced Message Queuing Protocol (AMQP). In 2006, the AMQP specification was released. RabbitMQ 1.0, developed by Rabbit Technologies based on the AMQP standard, was released in 2007.

The latest version of RabbitMQ is 3.5.7, based on AMQP 0-9-1.

In THE AMQP protocol, message sending and receiving involves the following concepts:

  • Broker: An application that receives and distributes messages. Our daily RabbitMQ is a Message Broker.
  • Virtual Host: Designed for multi-tenancy and security reasons, the basic components of the AMQP are divided into Virtual groups, similar to the namespace concept in the network. When multiple users use the same RabbitMQ service, multiple vhosts can be created for each userExchange/queueWait, this Songko has written about portals before:VirtualHost in RabbitMQ.
  • Connection: TCP Connection between publisher/consumer and broker. Disconnected operations are performed only on the client side. The broker is not disconnected unless there is a network failure or a problem with the broker service.
  • Channel: If a Connection is established for each RabbitMQ access, it is expensive and inefficient to establish a TCP Connection when messages are heavy. A Channel is a logical Connection established within a Connection. If the application supports multiple threads, each Thread usually creates a separate Channel for communication. The AMQP Method contains a Channel ID to help the client and Message Broker identify channels, so they are completely isolated from each other. As a lightweight Connection, a Channel greatly reduces the overhead of establishing a TCP Connection for the operating system. Songo explains how to use channels in his RabbitMQ management page.
  • Exchange: The first destination of the Message to the Broker, which matches the routing key in the query table to the queue according to the distribution rules. Common types are: Direct (peer-to-peer), Topic (publish and subscribe), and FANout (broadcast).
  • Queue: This is where messages are eventually sent to be picked up by the Consumer. A Message can be copied to multiple queues at the same time.
  • Binding: Virtual connection between exchanges and queues. Binding can contain routing keys. Binding information is stored in the Exchange query table as a basis for Message distribution.

1.2.2 closer to realize

Let’s take a look at some of the specific messaging middleware products that implement the AMQP protocol.

  • Apache Qpid
  • Apache ActiveMQ
  • RabbitMQ

Some people may wonder why ActiveMQ is still available. In fact, ActiveMQ not only supports JMS, but also supports AMQP.

There is also Alibaba’s RocketMQ, which is a custom protocol and the community also provides JMS, but it is not mature enough, songo explains.

1.3 the MQTT

MQTT (Message Queuing Telemetry Transport) is an instant messaging protocol developed by IBM. MQTT is one of the most important protocols in iot development. This protocol supports all platforms and can almost connect all networked objects with the outside world. It can be used as a communication protocol for sensors and actuators (such as connecting houses to the Internet through Twitter). Its advantages are simple format, small bandwidth consumption, mobile communication, PUSH support, and suitable for embedded systems.

1.4 XMPP

XMPP (Extensible Messaging and Presence Protocol) is an XML-based Protocol that is mainly used for INSTANT Messaging (IM) and online field detection. It applies to quasi-real-time operations between servers. The core is xmL-based streaming, a protocol that could eventually allow Internet users to send instant messages to anyone else on the Internet, even if their operating systems and browsers are different. It has the advantages of general openness, strong compatibility, extensibility and high security, but its disadvantage is that XML encoding format takes up a lot of bandwidth.

JMS 1.5 Vs closer

JMS and AMQP are both protocols, so what is the difference between them? Take a look at the following image:

This picture makes it pretty clear, so I won’t bore you.

2. Important products

2.1 ActiveMQ

ActiveMQ is a subproject under Apache. It uses JMS Provider implementation that fully supports JMS1.1 and J2EE1.4 specifications. Advanced application scenarios can be efficiently implemented with a little code and pluggable transport protocols are supported, such as: In-vm, TCP, SSL, NIO, UDP, Multicast, JGroups and JXTA Transports.

ActiveMQ supports a variety of commonly used language clients such as C++, Java,.net, Python, Php, Ruby, etc.

Now ActiveMQ is divided into two versions:

  • ActiveMQ Classic
  • ActiveMQ Artemis

ActiveMQ Classic here is the original ActiveMQ, while ActiveMQ Artemis is developed on the basis of HornetQ server code donated by RedHat. The two codes are completely different, and the latter supports JMS2.0. Using Netty based asynchronous IO improves performance and, amazingly, supports NOT only JMS, but also AMQP, STOMP, and MQTT, which offers a lot of gameplay.

Therefore, it is suggested to choose ActiveMQ Artemis directly when using it.

2.2 the RabbitMQ

RabbitMQ is one of the most important products in the AMQP system. It is based on the Erlang language. Many people will be tortured by RabbitMQ installation. Save worry and effort (public number background reply docker tutorial).

RabbitMQ supports AMQP, XMPP, SMTP, STOMP and other protocols.

Take a look at the RabbitMQ structure:

Songko has recently posted about a dozen tutorials on RabbitMQ, so I won’t go into the details here.

2.3 RocketMQ

RocketMQ is an open-source distributed messaging middleware developed by Alibaba. Its original name was Metaq, and its name was changed to RocketMQ since version 3.0. It is a set of MQ implemented by Alibaba based on the Design concept of Kafka using the Java language. RocketMQ integrates alibaba’s internal MQ products (Notify, Metaq) to maintain only the core functions, remove all other runtime dependencies, and ensure the most simplified core functions. Based on this, RocketMQ architecture is implemented in different scenarios with alibaba’s other open source products. Currently, it is mainly used for order trading system.

RocketMQ has the following features:

  • Ensure strict message order.
  • Provides message filtering function.
  • Provides rich message pull patterns.
  • Effective subscriber level expansion capabilities.
  • Real-time message subscription mechanism.
  • Hundred million message accumulation capability

For Java engineers, this is also a frequently used VERSION of MQ.

2.4 Kafka

Kafka is an open source stream processing platform under Apache, written in Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that processes the flow of data for all consumer actions (web browsing, search, and other user actions) within a website. Kafka is designed to unify online and offline message processing through Hadoop’s parallel loading mechanism, and to provide real-time messaging across clusters.

Kafka has the following features:

  • Fast persistence: With sequential disk reads and writes and zero copy, message persistence can be performed with O(1) overhead.
  • High throughput: 10W/s throughput can be achieved on a normal server.
  • High accumulation: Consumers under the support topic are offline for a long time, resulting in a large amount of message accumulation.
  • Fully distributed systems: Brokers, producers, and consumers all automatically support distribution, and Zookeeper automatically implements more complex load balancing.
  • Supports parallel loading of Hadoop data.

Kafka is a lot of big data development, and Java development is a lot of Kafka, but it’s relatively rare.

2.5 ZeroMQ

ZeroMQ claims to be the fastest message queue system. It is developed for high throughput/low latency scenarios and is often used in financial applications, with an emphasis on real-time data communication scenarios. ZeroMQ is not a separate service, but an embedded library, which encapsulates network communication, message queue, thread scheduling and other functions. It provides simple API to the upper layer. Applications load library files and call API functions to achieve high-performance network communication.

ZeroMQ features:

  • Lock-free queue model: For PIPE, a data exchange channel between client and session interactions, a lock-free queue algorithm CAS is adopted. Asynchronous events are registered on both sides of the pipe. Read and write events are automatically triggered when a message is read or written to the pipe.
  • Batch processing algorithm: For batch messages, adaptive optimization, can receive and send messages in batches.
  • Multi-core thread binding without CPU switching: Unlike traditional multi-threaded concurrent mode, semaphores or critical sections, ZeroMQ takes full advantage of multi-core, running one worker thread per core binding, avoiding CPU switching overhead between multiple threads.

2.6 other

In addition, there are also such as Redis can also do message queues, Songko previously also posted an article and we introduce the use of Redis to do ordinary message queues and delayed message queues, here is not too verbose.

3. Compare

Finally, let’s compare the message middleware with a graph.

If you reply MQPKMQ in the background of the official account, you can get this Excel spreadsheet link.

All right, that’s it.