This article is based on a talk by Grigory Starinkin, engineer at Erlang Solutions, at the RabbitMQ Summit 2019. MQTT message server EMQ X and RabbitMQ are introduced and compared in the stress test. We supplement this with an in-depth analysis of the core architecture focus of EMQ X and RabbitMQ and how they affect the performance of MQTT clustering patterns.
MQTT protocol – Subscribe and publish
MQTT is a very lightweight subscription and publishing protocol that is now the most dominant and widely used transport protocol in the Internet of Things. MQTT protocol can communicate with only 2 bytes per message (the header only needs 1 byte and the rest bytes can be used as the message payload), which is designed for the hardware with limited resources and space and power consumption sensitive. The main mode is Pub/Sub (publish/subscribe), where the client can play two roles. One is the publisher, who sends messages to the server for a specific topic after connecting to the server. The other role is the subscriber, who can subscribe to topics of interest to receive messages from them. Subscribers can also subscribe to topics using wildcards, which allows them to subscribe to multiple different topics at once, and can use shared subscriptions for load-balancing distribution.
The following image shows how the MQTT protocol works:
There are many MQTT client SDKS on the market today, as well as many MQTT brokers. EMQ X and RabbitMQ are two of the most representative open source messaging servers in the Erlang family and will be compared in depth for the MQTT scenario.
With the RabbitMQ EMQ X
EMQ X is an open source MQTT messaging server based on the highly concurrent Erlang/OTP language platform, supporting millions of connections, distributed cluster architecture, publish and subscribe model. So far, EMQ X has been widely used in the global Internet of Things market. On the basis of the open source version, we have also developed the commercial version and provided cloud hosting (www.emqx.com/zh/cloud). EMQ X supports many plug-ins and has powerful expansion capability. Users can rely on plug-ins to achieve more functions.
RabbitMQ is open source message broker software (also known as message-oriented middleware) that implements the Advanced Message Queuing Protocol (AMQP). RabbitMQ servers are also developed in the Erlang language and can now be configured to support MQTT through plug-ins.
Not surprisingly, they all use Erlang as their development language and they all use Mnesia, the distributed database management system that Erlang carries. Mnesia is suitable for exchanging routing topologies and exchanging information between nodes within a cluster.
Pressure test
The performance of MQTT servers in real life is often used by users to judge the quality of a server, so this review will also focus on the performance of both servers under stress testing.
Testing tools
MZBench is used for this pressure test. MZBench (github.com/mzbench/mzb…). Erlang is a test tool written in Erlang. It has the following three types of nodes:
- MZSever: Can be used to create scenarios, such as creating a publisher and multiple subscribers. This information is passed to MZBench as a server;
- MZController: Where information generated from the server is further transmitted;
- MZNodes: These will connect to your cluster as MQTT clients, as shown below.
This review used an example of a cloud host, M5 Large, with each MQTT message server cluster consisting of three nodes, each configured with dual core and 8GB of memory. It is important to note that our EMQ X and RabbitMQ tests used exactly the same hardware resources to eliminate variables. All of these were equipped with Prometheus node exporters for pushing metrics to Prometheus and final data collection by Grafana.
Test scenarios
The stress tests will have two scenarios, “many to one” and “one to many.”
For one more
Many devices, such as temperature sensors or pressure sensors, send data to a server as publishers. The server then sends this data to a controller (the subscriber) to process the data.
More than a pair of
One controller acts as publisher and sends messages to the server, which in turn sends those messages to multiple devices as subscribers.
In each scenario, the number of “many” sides will gradually rise from 2,000 to 10,000. In each scenario, a 256-byte payload message is sent every second. Such a release does not result in excessive throughput. The purpose of using only 256 byte payloads is to show how the two servers work and how their clustering pattern responds to these scenarios.
The test results
The left Y-axis is CPU usage, and the bottom X-axis is the change in the number of clients on the “many” side.
For one more
As you can see from the many-to-one results, EMQ X is not much different from RabbitMQ.
More than a pair of
However, RabbitMQ does have a significant difference compared to EMQ X in terms of one-to-many results.
Copyright: EMQ
Original link: www.emqx.com/zh/blog/emq…
Technical support: If you have any questions about this article or EMQ related products, please visit askemq.com to ask questions. We will respond to you in time.
More technical dry goods, welcome to our official account [EMQ Chinese community].