Before the light into the light out of the message queue, we understand the role of the message queue, the advantages and disadvantages, and usage scenarios, believe you of message queue have a general concept, said at the end of the article to the pit of buried themselves in the future will be to write a practical tutorial, just now the end of the internship, also for a long time did not write the actual combat tutorial, so it’s to fill in the pit.
Front knowledge
Before reading this article, it is recommended to have some prior knowledge, including but not limited to:
- Common Linux commands
- Knowledge of message queues
- Basic use of Docker
- Docker-compose basics
- Basic use of SpringBoot
So without further ado, let’s get started.
The code and configuration files involved in this article can be obtained by replying to “RocketMQ” in the background of wechat official account “01 binary”.
Why use RocketMQ as an example?
The main purpose of this article is to intuitively understand message queues through examples. There are so many message queues (ActiveMQ, RabbitMQ, Kafka), why should I choose RocketMQ? Here we do not talk about the principle, only about the experience, only personal choice, do not like to spray.
- Back to Ali, do not look at the evaluation, purely see he has experienced many double 11 test already know its performance is in the first batch.
- As a Java programmer, if you choose a piece of software written in pure Java, it will be much easier to read the source code later. (RabbitMQ is Erlang, kafka is Scala)
- During my internship at Alibaba, I used the internal version of RocketMQ, which was more familiar to me.
I met RocketMQ
Before using message queues, we need to know what a message queue is. This section is referred to the previous article “Shallow in shallow out message queue” and will not be covered here.
In this section to explain the concepts involved in RocketMQ, let’s start with a brief look at the official RocketMQ architecture diagram
- NameServer: serves as a registry to manage Topic information and routing information
- Broker: Responsible for storage, message tag filtering, and forwarding. You need to report your own information to NameServer
- -Serena: Well, I’m not being a Producer.
- -Penny: You’re a Consumer.
From the point of view of sending a letter
The above explanation may be difficult to understand, but let’s look at the following four parts of responsibility from the example of sending a letter.
- Needless to say, the producers and consumers of messages, the producers delivering messages and the consumers receiving messages, are the applications we write. It can be understood as sender and receiver.
- The Broker is responsible for message storage and stores messages in queues with a Topic dimension. It can be understood as a mailbox, which stores letters, and the Consumer can get the letters from it.
- NameServer is responsible for managing source data, including Topic and Broker management. The post office is responsible for managing the distribution of mail and maintaining the status of brokers.
According to the functions of the roles above, we need to install and start NameServer before starting the Broker to build RocketMQ
Install RocketMQ
If you already have RocketMQ configured on your computer, skip this section.
As we can see from the above introduction, we need to install Broker and NameServer before we can produce and consume messages.
The preparatory work
For easy deployment, I recommend using Docker to build the service. In addition, since RocketMQ requires that broker and Nameserver be deployed separately, I will use Docker-compose for this purpose in consideration of the inconvenience of deploying them separately. Therefore, you need to install Docker and Docker-compose in your host.
In addition, we need to set up a Web visual console to monitor mq service status and message consumption, using RocketMQ-Console, which will also be installed using Docker.
If you are not familiar with docker, you can read the docker tutorial 👉 docker tutorial first
The installation
Install the Docker
Linux:
Execute the following command
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun
Copy the codeMac:
Execute the following command
brew cask install docker
Copy the codeWin:
Download the corresponding installation file, and then double-click to run the installation. Download address in: hub.docker.com/editions/co…
Considering that downloading the file requires scientific Internet access, you can reply “Docker” in the background of wechat public number “01 binary” to obtain the download link of the Docker installation package.
If your Windows 10 system can use Winget, execute the following command. (Win finally has its own package management tool 🙏)
winget install Docker.DockerDesktop
Copy the codeSometimes it is difficult to pull the image from DockerHub in China, so you can configure the image accelerator. For the configuration tutorial, see 👉Docker Image Acceleration
Install the RocketMQ image
Rocketmq docker image we can make our own, the official documentation is detailed in 👉apache/ RocketMQ-Docker
For convenience, we’ll use an image that someone else has already created, at 👉 Foxiswho/Rocketmq
Create a directory for storing related scripts and run 👇 on the terminal
git clone https://github.com/foxiswho/docker-rocketmq.git
cd docker-rocketmq
cd rmq
chmod +x start.sh
./start.sh
Copy the codeAfter a period of waiting, visit localhost:8180 through the browser and check the following page. It indicates that the installation is successful.
Installation Script Parsing
A one-click installation script is convenient, but if you just install it and you’re done, it’s not. Let’s take a look at what the installation script contains:
start.sh
Lines 4-7 create directories, and lines 10-13 set permissions for the directory you just created, for reasons we’ll talk about later.
We saw that line 16 started the container with the docker-compose command and set it to automatically start in the background, so let’s take a look at the docker-compose. Yml file.
docker-compose.yml
version: "3.5"
services:
rmqnamesrv:
image: Foxiswho/rocketmq: 4.7.0
container_name: rmqnamesrv
ports:
- 9876: 9876
volumes:
- ./rmqs/logs:/opt/logs
- ./rmqs/store:/opt/store
environment:
JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
command: ["sh"."mqnamesrv"]
networks:
rmq:
aliases:
- rmqnamesrv
rmqbroker:
image: Foxiswho/rocketmq: 4.7.0
container_name: rmqbroker
ports:
- 10909: 10909
- 10911: 10911
volumes:
- ./rmq/logs:/opt/logs
- ./rmq/store:/opt/store
- ./rmq/brokerconf/broker.conf:/etc/rocketmq/broker.conf
environment:
JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
command:
[
"sh"."mqbroker"."-c"."/etc/rocketmq/broker.conf"."-n"."rmqnamesrv:9876"."autoCreateTopicEnable=true",]depends_on:
- rmqnamesrv
networks:
rmq:
aliases:
- rmqbroker
rmqconsole:
image: styletang/rocketmq-console-ng
container_name: rmqconsole
ports:
- 8180: 8080
environment:
JAVA_OPTS: "-Drocketmq.namesrv.addr=rmqnamesrv:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false"
depends_on:
- rmqnamesrv
networks:
rmq:
aliases:
- rmqconsole
networks:
rmq:
name: rmq
driver: bridge
Copy the codeWe created three services named RMQNamesrv, RMQBroker, and RMQConsole, which correspond to nameserver, Broker, and visual console, respectively. In addition, different port mappings are made for different services. At the same time, locally specified file directories are mounted to docker containers and network connections are made in the form of Bridges.
In the case of RMQNamesrv, the base image is FoxISWHO/RocketMQ :4.7.0. The container named RMQNamesrv is created and its internal port 9876 is mapped to the host port 9876. And mount the local./ RMQS /logs file to the /opt/logs directory of the Docker container.
rmqnamesrv:
image: Foxiswho/rocketmq: 4.7.0
container_name: rmqnamesrv
ports:
- 9876: 9876
volumes:
- ./rmqs/logs:/opt/logs
- ./rmqs/store:/opt/store
Copy the codeFor those unfamiliar with docker-compose, please refer to 👉 docker compose
SpringBoot integrates RocketMQ small instances
After completing the relatively complex installation and configuration, we were finally able to implement a small demo to get through the process.
Create message topics and subscription groups
When using RocketMQ to send messages, you must specify topic. For topic Settings, there is a switch autoCreateTopicEnable. The default setting autoCreateTopicEnable = true is used in development test environments. However, as a result, topic Settings are not easily managed, there is no uniform auditing, etc., so in a formal environment the parameter autoCreateTopicEnable = false is set at Broker startup. When topics need to be added, they need to be added on the Web management interface.
You can add a topic to the Web interface as follows:
Similarly, in accepting the message, we also need to subscribe for news group, for news subscription Settings, there is a switch autoCreateSubscriptionGroup normally, in a production environment, We need to set to autoCreateSubscriptionGroup = false, this requires managers must go to the web admin UI group create a subscription to receive the message.
The way to add a subscription group on the Web interface is similar, as shown in the following figure:
If we are just testing the environment, we can turn both switches on in a configuration file in the RMQ/RMQ/BrokerConf directory
Write the code
Apache has provided rocketMQ’s springBoot starter, which greatly simplifies the configuration process, so what we need to do is to create a New SpringBoot project and implement it as follows.
Import dependence
Start by importing the apache starter in pom. XML
<! -- https://mvnrepository.com/artifact/org.apache.rocketmq/rocketmq-spring-boot-starter --> <dependency> < the groupId > org. Apache. Rocketmq < / groupId > < artifactId > rocketmq - spring - the boot - starter < / artifactId > < version > 2.1.0 < / version > </dependency>Copy the codeThe configuration application. Yml
After the dependency import, we need to configure a name-server address in application.yml, depending on your machine.
rocketmq:
name-server: localhost:9876
producer:
group: myGroup
Copy the codeCreate a producer class
The producer sends a message:
@RestController
public class RocketController {
@Autowired
private RocketMQTemplate rocketMQTemplate;
// Sent to the Broker, topic is automatically created by default. Topic and tag are separated by colons
@GetMapping("/rocket/send")
public String rocketSend(a) {
LocalDateTime currentTime = LocalDateTime.now();
rocketMQTemplate.convertAndSend("rocket-topic-2", currentTime.toString());
return currentTime.toString();
}
// Delayed messages. RocketMQ supports these levels of delayed messages and cannot customize the duration
// 1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h
@GetMapping("/rocket/delayMsg/send")
public String rocketDelayMsgSend(a) {
LocalDateTime currentTime = LocalDateTime.now();
rocketMQTemplate.syncSend("rocket-topic-2:tag-2", MessageBuilder.withPayload(currentTime.toString()).build(), 2000.3);
returncurrentTime.toString(); }}Copy the codeCreate a consumer
Consumers listen for messages:
@Component
@Slf4j
public class RokcetServiceListener {
@Service
@RocketMQMessageListener(consumerGroup = "consumer-group-1", topic = "rocket-topic-2")
public class Consumer1 implements RocketMQListener<String> {
@Override
public void onMessage(String s) {
log.info("Consumer1 Rocket received a message: {}", s); }}RocketMQ supports two types of consumption, aggregator consumption and broadcast consumption
@Service
@RocketMQMessageListener(consumerGroup = "consumer-group-2", topic = "rocket-topic-2", selectorExpression = "tag2", messageModel = MessageModel.BROADCASTING)
public class Consumer2 implements RocketMQListener<String> {
@Override
public void onMessage(String s) {
log.info("Consumer2 Rocket received a message: {}", s); }}}Copy the codetest
We in the browser to access localhost: 8080 / rocket/send, you can see the returned timestamp.
Similarly, messages can be viewed in the visual console
We can also see how consumers and producers produce and consume messages on the visual console, which is left to the reader to explore. At this point, a complete example of RocketMQ installed with Docker and used with SpringBoot is complete.
The problem
Question 1: No route info of this topic: XXXXXX
This error is caused by the fact that no corresponding topic is generated in the message queue, so we should go to the console and create a new topic
Fault 2: The connection is abnormal
If an error similar to the following connection exception occurs
Com. Alibaba. Rocketmq. Remoting. Exception. RemotingConnectException: connect to < 10909 > 172.0.0.120: failedCopy the codeThe possible reason is that you are not putting your project in a Docker container, so your project code is not directly accessible to the RocketMQ container, so we need to remove the #brokerIP1= XXXXX from our broker.conf. And change the following IP address to your RocketMQ container host IP. The configuration file is in the RMQ/RMQ/BrokerConf directory.
The last
In order to fill the hole, I chose RocketMQ as the object of example explanation. In the first section, I explained why I used RocketMQ, and then explained several important concepts in RocketMQ. Then I used Docker to quickly deploy and install a single instance of RocketMQ. The installation script is also analyzed. Finally, we implement a producer and consumer example through Springboot, the current mainstream Web framework, and explain the possible problems and solutions.
The above is all the content of this article, if you feel helpful to you, do not put the focus on the support of a wave, your support is the biggest power of my update.