EMQ X, an MQTT messaging server that connects massive IoT devices, is developed based on the highly concurrency Erlang/OTP language platform and supports millions of connections and distributed cluster architecture. EMQ X is already widely used in the global IoT market and is free to use for product prototyping, IoT startups, or large-scale commercial deployments.

Product introduction: https://www.emqx.cn/products/broker

Download address: https://www.emqx.cn/downloads#broker

An overview of

EMQ X V4.3.0 fixes some known issues and adds a number of features, including major improvements in wildcard subscription performance, routing table footprint, rules engine performance, and a large number of client-side offline processing performance. At the same time, the underlying communication mode of multi-language extension is replaced by ERLPORT for GRPC communication which is more flexible.

This is the last minor release of 4.x designed to provide users with a rich and stable MQTT server. It is recommended that all 4.x users upgrade to this release, after which the 4.x release will enter maintenance status and the EMQ X Team will move on to the following 5.0 release development.

Visit the EMQ X GitHub repository and click the Watch in the upper right corner to keep track of the latest developments of EMQ X 5.0.

Feature overview: observer_cli looks at Erlang VM health

Upgrade to version 4.3

Check the upgrade guide: upgrade to https://docs.emqx.cn/broker/v4.3/changes/upgrade-4.3.html# – 4-3 – version

Detailed update log: https://docs.emqx.cn/broker/v4.3/changes/changes-4.3.html

Performance improvements

The low-level implementation of multi-language extension function is changed from ERLPORT to GRPC

The multi-language extension allows users to use other programming languages such as Python, Java, etc., to conduct business processing directly to EMQ X mount hooks, receive and process listener byte data packets to achieve the parsing of custom protocols.

The multilingual extension can handle the client connection life cycle in a familiar programming language and quickly integrate into IoT applications. Access to any proprietary protocol and enjoy the many performance and functionality benefits provided by EMQ X.

Previously, EMQ X multilingual extension was based on erlport for cross-language communication, considering that the level of language support was not wide, the performance level was insufficient, and it would cause resource competition with EMQ X’s own functions. In this version, we will replace the low-level mode with GRPC, which can better achieve cross-language and cross-platform work, provide higher performance and clear management ability, and effectively improve the difficulty of user development and maintenance.

Improve the performance of the rules engine

Earlier we added all the hooks available to the rules engine, and EMQ X would try to match the rules even if no rules were created, causing unnecessary overhead when firing any events.

In this version, we have improved the performance of the rules engine by adding only the necessary hooks for the rules that have been created.

Supports routing table compression to reduce memory footprint

Routing table compression is supported to reduce memory footprint and enhance subscription performance, but publish performance is slightly affected, so it provides a shutdown option.

Optimize wildcard subscription performance

Optimized wildcard subscription performance in EMQ X clustering conditions, more than 10 times better than the previous version.

Improved processing performance when a large number of clients are offline at the same time

We fixed the problem of previous offline messages and new online messages piling up in the Broker process, which resulted in a performance improvement.

The new features

The rule engine

  • The rule engine has added update resource logic that allows you to change resources used by actions without deleting rules
  • The rules engine SQL functions support conversion between UNIX timestamps and RFC3339 format times
  • Keep retrying resources that failed to connect after the EMQ X Broker started to avoid rules that cannot be restored after a brief outage of the resource

Operations management

  • Support observer_cli. Enter observer_cli:start() in Console startup mode. You can view the real-time performance details of the Erlang VM
  • Prometheus supports cluster metrics
  • Support single line log output, and support RFC3339 time format
  • Support for IPv6 automatic detection
  • Environment variable override profiles are supported in all distributions (previously only Docker)
  • The open source version supports Dashboard uploading certificate files (previously only available for enterprise version)

MQTT enhancement

  • The starting point is randomly selected when the Shared Subscription Distribution Policy is configured to Round_Robin
  • Shared subscriptions support the distribution of messages as a Hash of the source topic, and there can be a fixed distribution channel between the device and the shared subscription

Other features

  • WebSocket connections support access to real IP and Port
  • The WebSocket listener supports selecting the supported subprotocols from a list of subprotocols
  • Support for MySQL 8.0 default authentication method caching_sha2_password
  • Support for import and export of Mnesia authentication information
  • It is allowed to use the base64-encoded Client certificate or the MD5 value of the Client certificate as the user name or Client ID
  • Reboot listeners are supported, and some listener Settings can be updated without restarting EMQ X
  • Enable data telemetry in the official version only
  • Supports clearing all ACL caches
  • Redis Sentinel mode supports SSL connections
  • Emqx_auth_clientid is merged with emqx_auth_usernmae to become emqx_auth_mnesia. Refer to the documentation to export the data to older versions and import it into 4.3
  • By default, Docker outputs logs to the console. Set EMQX_LOG__TO=file to output logs to a file
  • Log output in JSON format is supported, and some log analysis systems such as ELK can be better configured for use

Safety improvement

  • Protects EMQ X Broker from cross-site WebSocket Hijack attacks
  • SSL supports the verify and server_name_indication configuration items
  • SSL supports maximum length of certificate chain and private key file password configuration items
  • JWT certification supports JWKS

The development of building

  • Support Erlang/OTP 23

    Upgrade to version Erlang/OTP 23, in order to improve the speed feature updates and bug fixes, EMQ X fork and maintenance of the Erlang/OTP project, the default using http://github.com/emqx/otp to build.

  • The new installer only supports MacOS 10.14 and above
  • The project is adjusted to Umbrella structure

    After adjusting the project structure, the complexity of dependency management is reduced, and the atomicity of changes associated with dependencies is guaranteed. At the same time, review and testing are less complex and more developer friendly to the community.

  • Support for compiling plug-ins using Elixir

Bug fix

The MQTT protocol

  • Fix the processing of MQTT heartbeat packets
  • Fixed MQTT message reception count issues
  • The maximum length of the restricted flight window is 65535
  • Fixed an issue where the values of the Keep Alive field in Dashboard were not synchronized when Server Keep Alive was in effect

The gateway

  • Fixed ACL configuration not working in COAP connections
  • Fixed issue where COAP clients using the same ClientID could be accessed at the same time
  • Fixed an issue where MQTT-SN sleep mode was not available
  • Fixed an issue where the MQTT-SN gateway discards Disconnect packets in sleep mode
  • Fixed LWM2M gateway encoding and decoding numbers into unsigned integers
  • Fixed issue with Clean Session false MQTT-SN connection not Posting testate message when abnormally broken

resources

  • Fixed an issue where MySQL authenticated SSL/TLS connection functionality was not available
  • Fixed Redis reconnect failure

Other repair

  • Fixed issue with Ekka_locker where memory could grow indefinitely in extreme conditions
  • Fixed an issue where the max_inflight_size configuration item in the MQTT bridging function was not active
  • Fixed an issue with MQTT bridge flight Windows
  • Fixed metric statistics errors in the MQTT bridging function and multiple unit conversions in the retry_interval field
  • Fixed an issue where the duration of the alarm was calculated incorrectly
  • Fixed an issue where too long Client IDs could not be traced
  • Fixed a crash when querying client information
  • Fixed issue where the theme rewrite was not executed in the same order as the ACL when publishing and subscribing
  • Fixed issue with WebSocket connection not using the counterpart certificate as the user name
  • Fixed an issue where authentication data could not be imported
  • Fixed an issue where EMQ X in Docker might fail to boot
  • Fixed bug MC-115716 – OOM kills connection process quickly

Copyright Notice: This article is original by EMQ, please give credit for reprinting.

The original link: https://www.emqx.cn/blog/emqx-4-3-0-release-notes