What’s New in HiveMQ 4.18?

The HiveMQ team is proud to announce the release of HiveMQ Enterprise MQTT Platform 4.18. This release includes added extension consumer observability in the HiveMQ Control Center, increased extraction, configuration, and access control options in the HiveMQ Enterprise Security extension, and numerous performance enhancements for your HiveMQ Enterprise MQTT broker.

Highlights

• New HiveMQ Control Center view for extension consumers
• Multiple HiveMQ Enterprise Security Extension enhancements:
  • Expanded extraction of fields from X.509 certificates
  • Unlimited customizable general purpose variables
  • File-based authentication and authorization for the REST API

New Extension Consumer view in the HiveMQ Control Center

One of the unique features of the HiveMQ MQTT broker is its flexible extension framework that allows seamless integration with virtually any existing enterprise system. In addition to all the powerful enterprise extensions that come prepackaged in the HiveMQ platform bundle, HiveMQ offers a free open-source extension SDK. The open API of the HiveMQ extension framework gives developers the freedom to create custom extensions to fit their specific business needs.

HiveMQ extensions give you the ability to integrate with cloud providers and streaming services such as Google Cloud Pub/Sub, Amazon Kinesis, Apache Kafka, and many more. The new Extension Consumers view in the HiveMQ Control Center gives you a clear overview of all the extension consumers that are currently active on your HiveMQ deployment.

How it works

The Extension Consumers view in your HiveMQ Control Center gives you the ability to filter for specific consumers in a variety of ways. You can sort information for each extension consumer that is active on your system based on consumer ID, topic filters, and the extension name. The overview shows you the current queue size per consumer as well as the maximum queue size that is allowed:

For more information, visit our HiveMQ Control Center documentation.

How it helps

Occasionally, downstream services cannot keep pace with the message throughput of the HiveMQ broker. For optimal performance, it can be important to periodically check the performance of your extension consumers and make adjustments where needed. The up-to-date information you get on the current and maximum queue size per consumer, topic, and extension can help you quickly identify and rectify any bottlenecks.

HiveMQ Enterprise Security Extension: expanded extraction of fields from X.509 certificates

The X.509 preprocessor of the HiveMQ Enterprise Security Extension can now extract more fields from X.509 certificates and make them available as ESE variables for the processing of user authentication and authorization. The HiveMQ Enterprise Security Extension provides preprocessors for authentication and authorization pipelines that enable sophisticated security use cases.

How it works

You configure the <x509-preprocessor> for your <listener-pipeline>, choose the X.509 fields that you want to extract, and define which ESE variable stores the extracted value. In the following example, the value of the X.509 field serial-number is used as an authentication-key to look up a user in the authentication manager that runs after the preprocessor.

Example extraction of an X.509 certificate value in an X.509 preprocessor:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

<enterprise-security-extension>
    <pipelines>
        <listener-pipeline listener="listener">
            <authentication-preprocessors>
                <x509-preprocessor prefix="{{" postfix="}}">
                    <x509-extractions>
                        <x509-extraction>
                            <x509-field>serial-number</x509-field>
                            <ese-variable>authentication-key</ese-variable>
                        </x509-extraction>
                    </x509-extractions>
                </x509-preprocessor>
            </authentication-preprocessors>
            ...
</enterprise-security-extension>

For a full list of the X.509 certificate fields the HiveMQ Enterprise Security Extension can extract values from, see our X.509 preprocessor documentation.

How it helps

Customer scenarios that use X.509 certificates often need to use fields in a flexible way to generate information for the authentication and authorization of users. HiveMQ 4.18 significantly increases the number of X.509 certificate fields that ESE can extract values from. The additional extraction capabilities make it possible to implement more complex security use cases.

HiveMQ Enterprise Security Extension: customizable general purpose variables

Starting with HiveMQ 4.18, the HiveMQ Enterprise Security Extension allows you to configure an unlimited number of customizable general purpose ESE variables. Previously, the extension offered a limited number of general purpose variables. Now, you can add as many variables as your use case requires and customize the name of the variables. The new customizable general purpose variables give you even more flexibility to configure your HiveMQ Enterprise Security Extension pipelines as desired.

How it works

General purpose string variables need to start with either a string- or byte- prefix and can contain any number of lowercase characters, numbers, and hyphens. In the following example, mqtt-username and mqtt-clientid are concatenated and assigned to a new custom variable called string-my-variable-1 for further processing.

Example use of customized general purpose ESE variables:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

<enterprise-security-extension>
  <pipelines>
    <listener-pipeline listener="listener">
      <authentication-preprocessors>
        <concatenation-preprocessor>
          <from>mqtt-username</from> 
          <from>mqtt-clientid</from> 
          <to>string-my-variable-1</to>
        </concatenation-preprocessor>
      </authentication-preprocessors>
      ...
</enterprise-security-extension>

To learn more about ESE variables and configuration options, see our ESE Variable documentation.

How it helps

Customizable general purpose variables help increase the readability of your configuration. You can now clearly align the names of your ESE variables to your specific use case. The ability to set an unlimited number of general purpose variables can be beneficial for various applications. For example, since it is now possible to extract more fields from X.509 certificates with ESE, you can now copy those values to appropriately-named ESE variables.

HiveMQ Enterprise Security Extension: file-based authentication and authorization for the REST API

The HiveMQ 4.18 release adds file-based authentication and authorization in the HiveMQ Enterprise Security Extension to control access to the HiveMQ REST API. This new feature rounds out your file-based authentication and authorization capabilities across all available realms in your ESE implementation.

How it works

The ESE file realm configuration now includes a section where you can define access to your HiveMQ REST API. The new section contains configuration parameters for users, roles, and permissions that give you the ability to add fine-grained control access to the HiveMQ Rest API.

Example file-based authentication and authorization for the REST API:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ese-file-realm>
  <rest-api>
    <users>
      <user>
        <name>rest-api-user</name>
        <password>rest-api-user-password</password>
        <roles>
          <role>subscriptions</role>
        </roles>
      </user>
    </users>
    <roles>
      <role>
        <id>subscriptions</id>
        <permissions>
          <permission>HIVEMQ_MQTT_CLIENTS_GET</permission>
        </permissions>
      </role>
    </roles>
  </rest-api>
</ese-file-realm>

How it helps

The file realm of the HiveMQ Enterprise Security Extension can now be used to secure all pipelines with both authentication and authorization. This provides greater flexibility for your use cases and can significantly reduce the effort needed to implement security in situations where a file-based realm is sufficient. The file realm also makes it easy to explore features and configurations of the Enterprise Security Extension without having to set up additional external systems such as a database.

More Noteworthy Features and Improvements

HiveMQ Enterprise MQTT Broker

• Improved scaling of the cluster threads used for scheduled tasks based on the instance size to provide increased efficiency.
• Reduced the number of retransmission requests in the cluster transport during periods of high load.
• Fixed an issue that could negatively impact CPU usage during the collection of client session and subscription metrics.
• Reduced heap pressure during topology changes for nodes with a high number of client connections.
• Reduced heap pressure during periodic cleanup tasks.
• Improved observability during OCSP stapling for cases in which a Java KeyStore lacks an issuer certificate.
• Fixed an issue that could cause the client queue for QoS 2 messages to stall when the internal inflight PUBREL configuration is enabled.
• Added system metrics for HiveMQ process uptime and remaining disk space to provide increased observability and deeper insights.
• Fixed a race condition that could cause incorrect subscription metrics.

HiveMQ Enterprise Security Extension

• Fixed an issue that could cause a login failure when OpenID Connect authentication for Control Center users is in use.
• Improved the performance of SQL authorization managers that use REST API permissions.

HiveMQ Extension for MongoDB

• Fine-tuned placeholder handling to retain the original data type of all MQTT PUBLISH properties and ensure efficient processing.

HiveMQ Extension for PostgreSQL

• Added batching support in insert statement configurations that is enabled by default for <mqtt-to-postgresql-routes> using the <insert-statement> processor.

HiveMQ Extension for MySQL

• Added batching support in insert statement configurations that is enabled by default for <mqtt-to-mysql-routes> using the <insert-statement> processor.

MQTT CLI

• Added a command to update an existing policy in the HiveMQ Data Governance Hub.

Get Started Today

To upgrade to HiveMQ 4.18 from a previous HiveMQ version, take a look at our HiveMQ Upgrade Guide. To learn more about all the features we offer, explore the HiveMQ User Guide.

About HiveMQ Team

We love writing about MQTT, IoT protocols and architecture in general. Our experts are here to help, so reach out to us if we can help!

Contact HiveMQ