Chapter 1 Overview of RocketMQ
Overview of MQ
1. Introduction to MQ
MQ, Message Queue, is a middleware that provides Message Queue services. Also known as Message middleware, it is a software system that provides apis for the whole process of Message production, storage and consumption. Messages are data. The volume of messages is not very large.
2. MQ usage
There are many descriptions of MQ’s uses available on the Web, but they boil down to the following three points.
Current limiting peak clipping
MQ can be used to temporarily store excessive requests on a system so that the system can process them later, without losing requests or overwhelming the system.
Asynchronous decoupling
If the upstream system calls the downstream system synchronously, the throughput and concurrency of the system will be greatly reduced, and the coupling degree of the system is too high. Asynchronous invocation solves these problems. So a common practice for synchronous to asynchronous transition between the two layers is to add an MQ layer between them.
The data collection
A distributed system generates massive data flows, such as service logs, monitoring data, and user behaviors. It is a necessary technology for current Internet platforms to collect and summarize these data streams in real time or in batches, and then conduct big data analysis on these data streams. Such data collection is best done through MQ.
3. Common MQ products
ActiveMQ
ActiveMQ is an MQ product developed using Java language. It was used by many companies and projects in the early days. But community activity is already low. It’s rarely used in current projects.
RabbitMQ
RabbitMQ is an MQ product developed in the ErLang language. Throughput is lower than Kafka and RocketMQ, and because it is not developed in The Java language, it is difficult to customize internally.
Kafka
Kafka is an MQ product developed using the Scala/Java language. Its biggest characteristic is high throughput, and it is often used in real-time computing, log collection and other scenarios in the field of big data. It does not follow any of the common MQ protocols, but uses a proprietary protocol. For Spring Cloud Netcix, only RabbitMQ and Kafka are supported.
RocketMQ
RocketMQ is an MQ product developed in the Java language. After several years of testing of Ali Double 11, the performance and stability are very high. It does not follow any of the common MQ protocols, but uses a proprietary protocol. For Spring Cloud Alibaba, RabbitMQ and Kafka are supported, but RocketMQ is advocated.
contrast
| keywords | ACTIVEMQ | RABBITMQ | KAFKA | ROCKETMQ |
|---|---|---|---|---|
| Development of language | Java | ErLang | Java | Java |
| Single machine throughput | All level | All level | Hundreds of level | Hundreds of level |
| Topic | – | – | A hundred level Topic can affect system throughput | Kilo-level topics affect system throughput |
| Community activity | low | high | high | high |
4. MQ common protocols
MQ implementations generally follow some common protocol. Common protocols are as follows:
JMS
JMS, Java Messaging Service. MOM (Message Oriented Middleware PO/OO/AO) is a technical specification for MOM (Message Oriented Middleware PO/OO/AO) on the Java platform. It facilitates the elimination exchange of Java applications in messaging systems and provides a standard interface for generating, sending, and receiving messages. Simplify enterprise application development. ActiveMQ is a typical implementation of this protocol.
STOMP
STOMP, for Streaming Text Orientated Message Protocol, is a simple Text Protocol designed by MOM. STOMP provides an interoperable connection format that allows clients to interact with any STOMP message Broker. ActiveMQ is a typical implementation of this protocol, which RabbitMQ supports through plug-ins.
AMQP
AMQP (Advanced Message Queuing Protocol) is an application layer standard that provides unified messaging services. It is an open standard of application layer protocols and a MOM design. The client and message-oriented middleware based on this protocol can transmit messages, regardless of different products of client/middleware and different development languages. RabbitMQ is a typical implementation of this protocol.
MQTT
MQTT, Message Queuing Telemetry Transport, is an instant messaging protocol developed by IBM. It is a binary protocol used for communication between servers and low-power IoT devices. The protocol supports all platforms and can be used as a communication protocol for sensors and actuators by connecting almost any networked object to the outside world. RabbitMQ supports this protocol through plug-ins.
Overview of RocketMQ
1. Introduction to RocketMQ
RocketMQ is a unified messaging engine, lightweight data processing platform.
RocketMQ is an open source messaging middleware from Alibaba. On November 28, 2016, Alibaba donated RocketMQ to the Apache Software Foundation as an Apache Incubation project. On September 25, 2017, Apache announced that RocketMQ has hatched into the Apache Top-level Program (TLP), becoming the first top-level program for Internet middleware on Apache in the country.
Rocketmq.apache.org
2. Development history of RocketMQ
In 2007, Ali started the Multicolored Stone project. As the core message flow system in the project, Notify emerged at the historic moment. The Notify system was a prototype of RocketMQ.
In 2010, B2B used ActiveMQ as ali’s message kernel on a large scale. Ali is in urgent need of a messaging system with mass accumulation capability.
Kafka became open source in early 2011. Taobao middleware team developed a new MQ, MetaQ after in-depth research on Kafka.
In 2012 MetaQ moved to V3.0, abstracted it further, and made RocketMQ, which was then open sourced.
In 2015, Alibaba launched Aliware MQ, a messaging system specifically targeted at users on Alibaba Cloud, in addition to RocketMQ.
RocketMQ passed a new milestone on November 11 2016, hosting the flow of trillions of messages. On November 28, Alibaba donated RocketMQ to the Apache Software Foundation as an Apache Incubation project.
On September 25, 2017, Apache announced that RocketMQ has hatched into the Apache Top-level Program (TLP), becoming the first top-level program for Internet middleware on Apache in the country.