Skip to content

HiveMQ Data Hub

Enhance the Value of Your IoT Data

Integrated policy engine that validates, enforces, and manipulates data in motion to ensure data integrity and quality across your MQTT deployment.

HiveMQ Data Hub Overview

The HiveMQ Data Hub provides mechanisms to define how MQTT data is handled in the HiveMQ broker. This ensures that data quality is assessed at an early stage in the data supply chain, eliminating the need for subscribers to perform resource-intense validation before data reaches downstream devices or upstream services.

HiveMQ Data Hub Capabilities

The HiveMQ Data Hub provides the following capabilities to help you enforce data integrity and quality.

Enforce Behavior Policies

Define a policy that dictates agreed upon behaviors for how devices should work with the MQTT broker by logging, stopping, or transforming the behaviors. Flow-control validates in-flight message flow patterns, and the scripting engine allows you to implement behaviors tailored to your specific needs via JavaScript functions.

Transform Data

Convert or manipulate raw or structured data into the desired format while it moves through the MQTT broker and before it reaches consumers. Transformations move more operations to the edge so data can be standardized. For example, convert Fahrenheit to Celsius, or something more sophisticated such as data manipulation using JavaScript.

Visualize Data

Utilize the simple user interface to manage schemas, data, and behavioral policies. The dashboard provides an overview of overall quality metrics, making it easy to locate bad actors and bad data sources. Visualize the MQTT data further in tools like Grafana.

Data Schemas

Create the blueprint for how data is formatted. Both JSON and Protobuf formats are currently supported.  

Data Policies

Define the set of rules and guidelines to enforce how data and messages should be expected, like requiring a temperature between 100-1200.

RestAPI

The HiveMQ REST API provides an interface for applications to interact programmatically with the HiveMQ Enterprise MQTT broker.

HiveMQ allows us to securely move business critical data between devices and the cloud at scale. HiveMQ Data Hub now allows us to automatically validate and standardize that data to build a more reliable data pipeline and further grow our business.

Klaas Jan Koopman

/

Director Video Back Office & Service Platform, Liberty Global

Data Schema

Create the schema of how data should be formatted. Both JSON and Protobuf formats are currently supported. These can vary from simple to complex. The example on the right is a schema that validates GPS coordinates.

{
  "$id": "https://example.com/geographical-location.schema.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Longitude and Latitude Values",
  "description": "A geographical coordinate.",
  "required": ["latitude", "longitude"],
  "type": "object",
  "properties": {
    "latitude": {
      "type": "number",
      "minimum": -90,
      "maximum": 90
    },
    "longitude": {
      "type": "number",
      "minimum": -180,
      "maximum": 180
    }
  }
}
policy-gps.json

Data Policy

An appropriate policy tells HiveMQ how to handle incoming MQTT messages to enforce the rules and structure that the schema outlines. The example on the right drops the message and logs the result, but these can be arbitrarily complex and could re-queue the message. On the re-queue, the system could also apply transformation functions to fix bad data.

{
  "id": "com.hivemq.policy.coordinates",
  "matching": {
    "topicFilter": "coordinates"
  },
  "validation": {
    "validators": [
      {
        "type": "schema",
        "arguments": {
          "strategy": "ALL_OF",
          "schemas": [
            {
              "schemaId": "gps_coordinates",
              "version": "latest"
            }
          ]
        }
      }
    ]
  },
  "onFailure": {
    "pipeline": [
      {
        "id": "logFailure",
        "functionId": "log",
        "arguments": {
          "level": "WARN",
          "message": "${clientId} sent invalid coordinates on topic '${topic}' with result '${validationResult}'"
        }
      }
    ]
  }
}
workflow.json
Grafana Dashboard to Visualize MQTT Data

Visualization

A demo use case introduces a quality metric and visualizes it in a Grafana dashboard. An addition to the quality metric is the list of bad clients queried from a PostgreSQL database. A screenshot of the dashboard includes the data quality on the left hand side and list of the top 10 bad clients on the right hand side.

HiveMQ Data Hub Key Benefits

Maximize the business value of data being transported by defining data policies, transformations, and validation requirements to ensure the data is accurate and meets the standards your organization requires.

Faster Business Insights with HiveMQ MQTT Platform

Faster Business Insights

Provide faster business insights on validated data - stop acting on rogue data and generating more noise than insights.

Increased Data Quality with HiveMQ MQTT Platform

Increased Data Quality

Ensure data quality standards are centrally defined and enforced across all devices and messages.

Operational Efficiency with HiveMQ MQTT Platform

Operational Efficiency

Stop bad-acting devices from misusing MQTT connections, sending bad data, and monopolizing resources.

/sb-assets/f/243938/134x134/905daaa987/cost-savings.png

Reduce Costs

Reduce redundant processing and storage costs by only acting on good data. Reduce the impact of acting on bad data.

Improve Data Integrity with HiveMQ MQTT Platform

Improve Data Integrity

Quarantine and further investigate bad data to prevent it from contaminating your systems and ultimately ensuring your data is accurate and reliable.

Seamless Data Management with HiveMQ MQTT Platform

Data Management

Manage everything in a single system, allowing data to be processed faster and negating the need to manage another standalone system to ensure data quality.

Learn more about HiveMQ Data Hub

HiveMQ Data Hub Frequently Asked Questions

Recommended Next Steps

Schedule Demo

Contact Us

Get HiveMQ

Free Download

Read the Docs

Documentation
HiveMQ logo
Review HiveMQ on G2