HiveMQ Logo

HiveMQ Kafka Solution

Streaming IoT Data and MQTT Messages to Kafka

Apache Kafka is a popular open source streaming platform that makes it easy to share data between enterprise systems and applications. The HiveMQ Enterprise Extension for Kafka makes it possible to send and receive IoT device data with a Kafka cluster.

NEW: HiveMQ Cloud now integrates with Confluent Cloud making it easy to stream IoT data into Confluent Cloud.
With the 4.12 release, the Enterprise Extension for Kafka now has the following additional features:
  • Added tracing support in the HiveMQ Enterprise Extension for Kafka for MqttToKafkaTransformer customizations that create multiple Kafka records from a single MQTT message.
  • Added the capability to export information about the HiveMQ Enterprise Extension for Kafka in a diagnostic archive to further facilitate support and troubleshooting.
Kafka Solution for HiveMQ Confluent Integration in HiveMQ Cloud


HiveMQ and Kafka Architecture

HiveMQ solves the issues of Kafka for IoT by seamlessly integrating MQTT messages into the Kafka messaging flow. Conversely, Kafka messages can be distributed to HiveMQ and MQTT clients. The HiveMQ broker is extended to include a native implementation of the Kafka protocol. This allows HiveMQ to transpose MQTT messages into the Kafka protocol and vice versa. Our architecture allows for full MQTT support of IoT data plus complete integration with Kafka.

Architectural diagram of HiveMQ and Kafka

Why Is HiveMQ & MQTT Needed for IoT Use Cases


Kafka is well suited for sharing data between enterprise systems and applications located in a data center or in the cloud. However, for IoT use cases there are a number of reasons Kafka is not well suited:
More Resources

Detailed Features
Implements the Kafka protocol so MQTT messages can be written to a Kafka topic and conversely Kafka messages can be distributed to multiple MQTT clients.
Supports all MQTT 3 and MQTT 5 features, including all QoS levels
Select multiple MQTT topic filters per Kafka topic (MQTT topic filters come with full support of MQTT wildcards).
Buffer messages at the broker to ensure high-availability and failure tolerance if a Kafka cluster is not available.
Monitor MQTT messages that are written to Kafka on the HiveMQ Control Center.
Distribute Kafka messages to multiple MQTT clients
Map one to many Kafka topics to multiple MQTT topics.
Programmatic Customization API to specify MQTT to Kafka topic mapping and custom message transformation.
MQTT PUBLISH packets can be generated dynamically at runtime with variable replacement using parts of the Kafka message
Validate messages read from Kafka with the help of a schema registry
Use the predefined default configurations or the possibility to set MQTT PUBLISH metadata configuratively
Added tracing support in the extension for MqttToKafkaTransformer customizations; that can create multiple Kafka records from a single MQTT message.
Export information about the extension in a diagnostic archive to further facilitate support and troubleshooting.
... more will follow!

Try Now

Try HiveMQ Extension for Kafka

Download the Kafka Extension for HiveMQ


How can we help you?

Contact us to discuss how HiveMQ can help extend Kafka to IoT device data.