What’s New in HiveMQ 4.20?

The HiveMQ team is proud to announce the release of HiveMQ Enterprise MQTT Platform 4.20. This special release focuses on the official launch of our new HiveMQ Data Hub product (formerly known as the Data Governance Hub). Along with a new name, our latest release substantially increases Data Hub capabilities with two major new features: Behavior Policies and Data Hub support in the HiveMQ Control Center.

Highlights

• HiveMQ Data Hub behavior models
• New interactive Data Hub Control Center views
• Additional Data Hub functions

HiveMQ Data Hub Behavior Policies for MQTT Clients

Our new behavior policies are designed to give our customers an added level of control over the way their MQTT clients work within the HiveMQ broker. In addition to our existing data policies that ensure that the MQTT message data you accept has the expected structure and format, the new behavior policies ensure that your MQTT clients act in the manner you expect.

Data Hub behavior policies can help protect your deployment against the negative side effects of MQTT clients that do not adhere to your accepted standard behavior. These so-called bad actors can increase costs, decrease performance, or even compromise the integrity of your deployment.

In HiveMQ 4,20 we’ve extended the Data Hub policy engine with new capabilities that allow you to enforce the expected behavior of MQTT clients with behavior models. Now. you can define to which clients a policy applies and specify what action is taken if the policy is not followed.

How it works

Behavior Policies give you the ability to model the behavior of your MQTT clients throughout the entire lifecycle of the client. HiveMQ 4.20, provides three ready-to-use behavior models that a behavior policy can instantiate. Matching schemas let you define which clients you want to control with the policy. Additionally, you can trigger specific actions based on the state transitions and events of the behavior model.

Here’s an example, some use cases expect MQTT clients to send a specific number of MQTT messages (no more and no less). You can use the Publish.quota behavior model to outline your desired client behavior and transition the client to a failure state when the expected conditions are not met. If desired, you can define further actions that the transition event triggers such as logging a message or incrementing a custom metric. You can even use Data Hub to monitor and disconnect the client for sending too many messages.

In other use cases, MQTT clients that repeatedly send an identical message payload at a large scale can cause unnecessarily high resource consumption. Temperature sensors are a good example. Temperature values often remain constant and unchanged readings may not be of interest. The Publish.duplicate behavior model registers repetitive messages that the client sends to the same topic. If a duplicate messages is detected, you can define further actions such as dropping the message.

Example behavior policy with a Publish.duplicate model:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

{
    "id": "example-behavior-policy",
    "matching": {
        "clientIdRegex": ".*"
    },
    "behavior": {
        "id": "Publish.duplicate"
    },
    "onTransitions": [
        {
            "fromState": "Any.*",
            "toState": "Duplicated",
            "Mqtt.OnInboundPublish": {
                "pipeline": [
                    {
                        "id": "count-duplicate-messages",
                        "functionId": "Metrics.Counter.increment",
                        "arguments": {
                            "metricName": "repetitive-messages-count",
                            "incrementBy": 1
                        }
                    },
                    {
                        "id": "drop",
                        "functionId": "Mqtt.drop",
                        "arguments": {}
                    }
                ]
            }
        }
    ]
}

In the example policy, the regular expression (.*) matches every client. The policy instantiates the Publish.duplicate behavior model and the specified action drops every duplicate message.

How it helps

Until now, agreeing upon, documenting, and subsequently enforcing MQTT data handling and MQTT client best practices has relied on extensive communication between engineering teams and time-consuming manual testing. As a result, rules could be easily violated. Behavior policies are an effective safeguard against undesired client activity. Behavior models let teams automatically enforce standard behaviors, flag bad clients, drop the messages they’ve sent, and even disconnect them. Data Hub also improves monitoring of your cluster efficiency and performance, which allows you to reduce waste from unnecessary bandwidth usage and easily compute resource consumption. An added benefit is the enhanced debugging capabilities you gain from end-to-end tracking of client states that can save you time wasted on troubleshooting.

To learn more about Data Hub Behavior Policies, visit the Data Hub documentation.

Data Hub support in the HiveMQ Control Center

The HiveMQ 4.20 release significantly expands the capabilities of the HiveMQ Control Center to make it easier to manage Data Hub schemas and policies. All Data Hub functions available with the Rest API are now provided on a well-designed Control Center user interface.

How it works

The Data Hub view in the HiveMQ Control Center provides everything you need to conveniently create and manage your Data Hub schemas, data policies, and behavior policies. The new features even include an overview of the overall health status of your pipelines.

Our new Data Hub dashboard provides a clear snapshot of your current data pipeline quality. The dashboard visualizes the health of your data pipeline to show the percentage of data policy execution that results in failed and successful validations. The list of clients that violated policies also references the associated policy. You can use the dashboard to quickly understand the state of your data pipeline and act to resolve issues with any problematic clients. Naturally, Data Hub also provides a full set of metrics for further integration into your monitoring system.

How it helps

Control Center support for Data Hub simplifies usage for users that prefer graphical user interfaces over command line tools. Our policy creation wizards provide an intuitive way to create new policies with context help and immediate feedback on policy validity. The overview and list views help you efficiently manage your schemas and policies.

For more information, visit our HiveMQ Control Center documentation.

New Data Hub Drop and Disconnect Functions

An important aspect in both data policy and behavior policy creation is the option to trigger an action based on certain events and outcomes. HiveMQ 4.20 adds new functions that make it possible to drop MQTT messages or disconnect MQTT clients. The ability to utilize the new functions in actions in the Data Hub policy engine further increases the capability of the product to maintain data pipeline quality.

How it works

The new functions are available to use for both data policies and behavior policies.

• Mqtt.drop is a function that drops any MQTT packet handled by a policy. A typical use case is to drop MQTT messages with payloads that do not adhere to your defined schema. You can even provide a user-defined reason string to the client.
• Mqtt.disconnect is a function that disconnects a client from a broker. This option is useful when a client does not behave in the agreed upon manner.

How it helps

The new functions allow you to preemptively sort out bad data and the bad clients that produce it. Our MQTT drop and disconnect functions are an effective option for use cases where it is absolutely necessary to keep the data pipeline healthy.

Additional Features and Improvements

HiveMQ Enterprise MQTT Broker

• Trimmed the processing overhead of PUBLISH messages in the Consumer Service to improve performance.

HiveMQ Data Hub

• Fixed an issue that prevented upgrading an existing policy in free mode.

HiveMQ Enterprise Extension for Amazon Kinesis

• Fixed an issue that could cause a QoS 0 PUBLISH message to be unnecessarily resent to Amazon Kinesis Data Streams after the initial delivery attempt fails.

HiveMQ Enterprise Extension for Google Cloud Pub/Sub

• Fixed an issue that could cause a QoS 0 PUBLISH message to be unnecessarily resent to Google Cloud PubSub after the initial delivery attempt fails.

MQTT CLI

• Added support for Data Hub behavior policies
• Added support for Data Hub client state retrieval of a behavior model
• Adapted the Data Hub command names from policies to data-policies and behavior-policies and from schemas to schema.

Get Started Today

To upgrade to HiveMQ 4.20 from a previous HiveMQ version, take a look at our HiveMQ Upgrade Guide. To learn more about all the features we offer, explore the HiveMQ User Guide.

About HiveMQ Team

We love writing about MQTT, IoT protocols and architecture in general. Our experts are here to help, so reach out to us if we can help!

Contact HiveMQ