MQTT is introduced

MQTT (MQ Telemetry Transport) protocol is a lightweight network protocol developed by IBM in 1999. It adopts publish-subscribe communication mode. It has three main characteristics:

  1. The binary message content encoding format is adopted, so payload content such as binary data, JSON and images can be easily transferred.
  2. The compact protocol header and simple protocol interaction ensure that the network transmission traffic is small.
  3. Supports three Quality of Service (QoS) levels, facilitating flexible application selection based on different scenarios.

MQTT protocol is very suitable for remote devices with limited computing power, low network bandwidth and unstable signal, so it has become the de facto network protocol standard for Internet of Things systems.

MQTT’s own genes are strong

MQTT protocol as the “first language” of iot devices, not only because of the ecological perfection of MQTT, the excellent design of MQTT protocol itself is also an important factor.

Suitable for most application scenarios of the Internet of ThingsPublish and subscribe.

Using a publish-subscribe model, MQTT protocol has many advantages, such as the ability to trigger a series of actions from a single sensor data; Temporary offline caused by network instability does not affect work; It is convenient to adjust the system scale dynamically according to requirements. This makes it suitable for most iot scenarios.

To meet the needs of resource-constrained devices in the Internet of ThingslightweightFeatures.

MQTT is a lightweight networking protocol, which is why it is so popular in Iot systems. After all, a large number of devices in the Internet of Things have limited computing resources and low network bandwidth. This lightness manifests itself in two ways.

On the one hand, MQTT messages are encoded in a binary format rather than the textual representation of the HTTP protocol. Each character in the HTTP text takes up 1 byte. With the MQTT protocol, a single byte can represent a lot. The following image shows the format of MQTT’s fixed header, which is only 2 bytes long:

The first byte is divided into the upper four bits (4-7) and the lower four bits (0-3); The lower four bits are the packet identifier bits, each of which can represent a different meaning. The last four bits are the identifying bits for the different packet types. The second byte represents the total number of bytes in the packet header and message body, where the highest bit indicates whether there is a third byte to represent the total number of bytes together with the second byte. If there is a third byte, its highest bit indicates whether there is a fourth byte to represent the total number of bytes along with the second byte and the third byte. There could be a fourth byte, a fifth byte, and so on, but the number of bytes in the variable header and body is only up to the fifth byte, so the maximum packet length that can be represented is 256MB.

For example, a connect-type packet requesting a connection takes 14 bytes in its header. The header of a PUBLISH packet contains only 2 to 4 bytes.

On the other hand, the design of the specific interaction flow embodied in the messages is very simple, so there are very few types of interaction messages in MQTT.

As can be seen from the table, MQTT 3.1.1 defines a total of 14 types of packets, with values from 1 to 14 in the top four bits of the first byte.

Keep an eye on iot devicesLow power consumptionOptimized design of requirements.

In addition to making the protocol light enough, MQTT protocol also attaches great importance to the optimization design of low power consumption, which is mainly reflected in the optimization of energy consumption and communication times.

For example, the MQTT protocol has a Keepalive mechanism. Its purpose is to detect and re-establish an MQTT connection between Client and Broker when the connection is interrupted, ensuring the reliable transmission of topic messages. The mechanism works as follows: Both the Client and the Broker determine whether a message has been transmitted between them within a period of time, based on the duration specified by Keepalive. The Keepalive duration is set when the Client establishes a connection. If no new packet is received after the Keepalive duration expires, the Client considers that the connection is disconnected.

Although Keepalive requires a packet to be transmitted at some time, the Client and Broker cannot transmit a topic message all the time. What does this do?

The solution to the MQTT protocol is to define PINGREQ and PINGRESP message types. None of them have variable headers and message bodies, which means they are only 2 bytes in size. The Client and Broker meet the requirements of the Keepalive mechanism by sending PINGREQ and PINGRESP messages respectively. As I’m sure you’ve already figured out, it would be a waste of battery life and network resources to keep sending messages on a regular basis. Therefore, if there is any data transfer between the Client and the Broker during the Keepalive duration, the Keepalive mechanism takes it into account, eliminating the need to send PINGREQ and PINGRESP messages.

In addition to the Keepalive mechanism, the repeated themes feature in MQTT 5.0 also helped us save network resources. When a Client repeatedly sends a message on a topic, it can set the length of the topic name to 0, starting with the second time. Then the Broker will automatically process the message according to the last topic. This is very common with sensor devices, so this feature is very practical in the workplace.

Aiming at the changeable network environment in the Internet of ThingsA variety of service quality levels are provided.

MQTT protocol designs three different QoS levels. You have the flexibility to choose between scenarios to ensure reliable communication in different environments.

** What is QoS? ** It refers to the negotiation between communication parties about the reliability of message transmission.

QoS 0: The message is sent only once and may be lost.

QoS level 1: The sender receives feedback to ensure that the message is delivered, but the message may be repeated.

QoS level 2: Ensures that messages are sent only once through multiple interactions between the sender and receiver.

It can be seen that the flow of QoS 0 and QoS 1 is relatively simple; In QoS 2, the process is more complicated to ensure the reliable transmission with only one time. Normally, QoS 2 has four interactions: PUBLISH, PUBREC, PUBREL, and PUBCOMP. For “abnormal cases,” the sender needs to send the message repeatedly. For example, if you have not received a PUBREC message for a while, you need to send a PUBLISH message again. Note, however, that the “duplicate” flag in the message is set to 1 so that the recipient can process it correctly. Similarly, if a PUBCOMP message is not received, the sender needs to send a PUBREL message again.

Support is becoming more and more important in iot applicationsData security.

Speaking of secure transport, first we need to verify that the Client has access to the MQTT Broker. To control Client access, MQTT provides a user name/password mechanism. During connection establishment, it can filter valid connection requests by determining the correctness of user names and passwords. However, this mechanism alone cannot guarantee the data security in network communication. Because in plaintext transmission, not only device data, but even user names and passwords can be intercepted from the network and leaked, so that others can send data disguised as legitimate devices. Therefore, we also need the support of communication encryption technology. The MQTT protocol supports SSL and TLS encrypted communication. With SSL/TLS encryption, MQTT is converted to MQTTS. This is similar to the relationship between HTTP and HTTPS.

Experience the MQTT

Simple MQTT experience using HBMQTT

The first step is to install HBMQTT, which is an open source Python-based MQTT Broker software that includes just the tools we need to use. Compared to the other options, this software is very easy to install because it is available in Python’s PYPI repository, so you can install it with the PIP command. This is the main reason to use it. Note, however, that HBMQTT is implemented based on Python3, so the pip3 tool is used here.

pip3 install hbmqtt
Copy the code

Once installed, you can use the two command-line tools hBMqtt_sub and hbmqtt_pub provided with HBMQTT. By name, you can also tell that hBMqTT_sub can act as a subscriber; Hbmqtt_pub can act as the publisher of messages.

For brokers between subscribers and publishers, known as MQTT Brokers, we use Eclipse’s free open online Broker service. Open the link and you’ll see information about ports, encrypted and unencrypted support, and a Websocket-based implementation, which is great for front-end web-based applications.

We start with port 1883 unencrypted, and then determine a Topic for message transport. A topic identifies the categories of messages that are used for message filtering. For example, the message we will test later could be set to “/ MQTT /test”.

Next, we can subscribe to the topic message by typing the following command on the terminal interface of the computer:

hbmqtt_sub --url mqtt://test.mosquitto.org:1883 -t /mqtt/test
Copy the code

Now let’s launch another terminal interface and publish a message with the subject “/ MQTT /test” via hBMqtt_pub:

hbmqtt_pub --url mqtt://test.mosquitto.org:1883 -t /mqtt/test -m Hello,World!
Copy the code
Set up your own MQTT Broker for an in-depth experience

useVerneMQAs the MQTT Broker

First, install VerneMQ using Docker.

docker run -p 1883:1883 -e "DOCKER_VERNEMQ_ACCEPT_EULA=yes" -e "DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on" --name vernemqtt -d vernemq/vernemq
Copy the code

Second, connect to VerneMQ using VSCode and install the VsMqtt plug-in.

The Client code for MQTT can be implemented using the Eclipse Paho project.