Skip to content

Real-time Data Processing with Amazon Kinesis Integration on HiveMQ Cloud

by Shashank Sharma
5 min read

We are pleased to announce the addition of Amazon Kinesis integration to HiveMQ Cloud Starter. Amazon Kinesis integration enhances our MQTT platform by providing IoT data integration and real-time processing capabilities to HiveMQ Cloud users. This is valuable for individuals and organizations focusing on IoT and data-driven decision-making because it enables users to effortlessly collect, process, and analyze data from many IoT devices in real-time, leading to faster, more informed decision-making and a boost in operational efficiency for AWS-based processes. 

Amazon Kinesis Integration  

What is Amazon Kinesis? 

Amazon Kinesis is a comprehensive suite of tools on AWS to facilitate real-time data streaming and analytics. Amazon Kinesis Data Streams enables the continuous capture and storage of data streams such as IoT device data, supporting ingestion and enabling real-time analytics on this streamed data. As such, there is often a need for data transport from MQTT broker to AWS data pipeline using Kinesis Data Streams. 

What is Included in the Integration? 

This integration supports communication between IoT devices to Amazon Kinesis Data Streams. Once the data is integrated, it can be processed, stored, analyzed, or visualized for AI/ML projects — all within AWS.

The Amazon Kinesis integration is available under the Integrations tab in your Starter Cluster. 

Amazon Kinesis Integration on HiveMQ Cloud Starter

What are the Benefits of Using Amazon Kinesis Integration with HiveMQ Cloud?

The Amazon Kinesis integration with HiveMQ Cloud is particularly valuable for individuals and organizations focused on IoT and data-driven decision-making. As a user, it enables you to effortlessly transport data from your cloud broker instance to your AWS services. This allows you to process and analyze data in near real-time, leading to faster, more informed decision-making and a boost in operational efficiency for AWS-based processes. 

Configuring Amazon Kinesis Integration 

Configuring Amazon Kinesis integration consists of three easy steps: 

1. AWS Authentication 

You need an AWS credentials file. Your file should contain aws_access_key_id and aws_secret_access_key 

AWS Authentication

2. MQTT to Topic Filter 

The next step is to create a mapping from MQTT to AWS Kinesis. Currently, HiveMQ Cloud Starter supports unidirectional mapping from MQTT to the Kinesis stream. You can select the MQTT topics you want to stream data to Amazon Kinesis by defining an MQTT topic filter. You can utilize MQTT wildcards here. 

MQTT to Topic Filter3. Amazon Kinesis Stream and Region Configuration 

Finally, add details such as your Amazon Kinesis data stream name and your Kinesis region to complete your configuration. This helps you to connect your broker to the right Kinesis data stream.

Amazon Kinesis Stream and Region ConfigurationHere is a video that walks you through the integration: 

Walk through of Amazon Kinesis Integration on HiveMQ Cloud Starter

Chapters

    Summary 

    The Amazon Kinesis integration provides a simple and easy way to connect your HiveMQ Cloud broker with AWS workflows, leading to faster, more informed decision-making. This integration is a simplified version of the Enterprise Kinesis extension which is available with Cloud Enterprise Self-hosted plans.

    Begin your MQTT journey with no risks attached. Get a complete MQTT platform offering unlimited connections, 99.95% uptime SLA, 24/7 support, data streaming integrations, and more. Sign up for a free 15-day trial.

    Try HiveMQ Cloud Starter for FREE

    Shashank Sharma

    Shashank Sharma is a product marketing manager at HiveMQ. He is passionate about technology, supporting customers, and enabling developer-centric workflows. He focuses on the HiveMQ Cloud offerings and has previous experience in application software tooling, autonomous driving, and numerical computing.

    • Contact Shashank Sharma via e-mail
    HiveMQ logo
    Review HiveMQ on G2