TL;DRThis blog demonstrates how to enable real-time environmental data monitoring by extending Sensor.Community IoT sensor kits with a custom containerized FastAPI service and HiveMQ Cloud MQTT broker. The solution bridges Sensor.Community's lack of native MQTT support, enabling real-time streaming, Node-RED visualization, SQL storage, and air quality alerting, all within HiveMQ's free tier.
Who is this blog for: IoT developers, data engineers, and environmental monitoring enthusiasts interested in MQTT, real-time data streaming, and open environmental data solutions. "
Why Real-Time Environmental Data Matters
With the HiveMQ Cloud Fully Managed MQTT Broker, we can have open environmental data streamed in real time and available for visualization, downstream analytics, and real-time air quality index (AQI) alerting. We can achieve this functionality with a custom developed containerized API and the Sensor.Community pre-flashed IoT sensor kit. The solution enhances the data collection and reporting of open environmental data. If you ever wanted to have your own end-to-end solution tracking temperature, humidity, pressure and particulate matter concentration, this guide is for you.
Problem: Sensor.Community’s Limited Real-Time Streaming Capabilities
Sensor.Community is a contributor-driven global sensor network that tracks and collects Open Environmental Data. Every enthusiast can install a sensor kit to begin monitoring air data in real-time. The pre-flashed kit sends data to an open-source API. Eventually the data is used for visualizations in a publicly available Grafana instance. If you are a bit more of a data enthusiast, you may wonder: how can you implement real-time monitoring of the incoming data? How can you store the data for historical analysis? The problem is further constrained by the default sensor firmware—it lacks native MQTT integration.
Solution: Extending Sensor.Community with HiveMQ Cloud and a Custom API
The standard IoT device configuration offers the option to send data readings to a custom API:
Therefore, to enable real-time monitoring using a cloud MQTT broker and to add custom data collection, I decided to build a compatible API service. The idea is that at every measurement interval, my API will get the data the IoT sensor kit sends. From there, I can publish the data to an MQTT broker and store it in my own database.
Hardware and Software Requirements for the End-to-End Setup
Hardware:
- pre-flashed sensor kit. Here is what a standard kit looks like:
- You can buy the kit as a complete set or source each component separately.
Software:
- Visual Studio Code
- Docker Desktop or similar containerization engine
- An account with a cloud provider. I used Azure, but the solution can be adapted for any cloud provider. You will need the following services, or their equivalents:
- Container Registry
- App Service Plan & Azure Function
- Azure SQL Server & Database
- A HiveMQ Cloud account for creating a cloud MQTT broker. If you have an existing account, that is fine. If not, you can sign up for the free tier using this link.
- Node-RED for visualizing the data. The easiest way is to also run it under Docker.
Hardware Setup: Getting Started with the Sensor.Community Kit
To get started installing and using the kit, follow the guide on the Sensor.Community site. This document will focus on the software side of the setup.
Software Setup: Building the API-to-MQTT Integration Layer
On the software side, the task is to develop a solution that will do the following:
- Create a REST API endpoint that can accept requests from the IoT sensor with data payload
- Connect to HiveMQ MQTT broker and publish the payload
- Connect to and store the data in a database
- Wrap all functionality in a container
- Run the container in an Azure function app
The full solution code is available at this GitHub repo. In this document, I will only provide a high-level overview of the components of the solution. Here is the overall namespace structure:
└── 📁sensor-community-api-to-mqtt
└── 📁.venv
└── 📁api
└── 📁models
├── __init__.py
├── air_data.py
├── db_model.py
└── 📁routers
├── __init__.py
├── sensor_readings.py
├── __init__.py
├── .env
├── dependencies.py
├── function_app.py
├── main.py
├── publisher.py
├── rds.py
├── sql_client.py
├── transformer.py
├── .gitignore
├── Dockerfile
├── README.md
└── requirements.txtHere is a breakdown of the modules in the solution:
After building the container and deploying it to the function app, I can see the API service running:
At this point, the API service is ready to accept data payloads from the sensor. From the configuration menu available at a local IP on the network, we must configure the device to send requests to the custom API alongside the default one:
Data Flow: From Sensor to HiveMQ Cloud in Real Time
Having applied the configuration, I can navigate to the HiveMQ Cloud console, where I can see the output from my sensor. All the way from my balcony to the cloud!
Data Visualization
Next, we can use Node-RED or a similar platform supporting MQTT as data source to develop a dashboard displaying the data in real-time. This is a huge benefit enabled by the publish-subscribe nature of the MQTT protocol. I used the handy Node-RED “MQTT in” node to subscribe to the broker in real time. The flow definition is also available in the repo. Here is what the dashboard looks like:
Having everything connected and all software configurations in place, let us take a step back and see what the benefits are of publishing the data to HiveMQ.
Why HiveMQ? Benefits of a Cloud MQTT Broker for Environmental Data Streaming
To create a solid foundation for advanced analytics and monitoring, I chose to publish the data to a cloud-based MQTT broker. While my initial API implementation ran as a Windows service with a locally hosted broker, migrating to a cloud solution offers significant advantages in flexibility and accessibility. A cloud broker provides universal data access from anywhere with internet connectivity, eliminates the need for local infrastructure maintenance, and enables seamless integration with downstream analytics platforms and third-party services.
HiveMQ's cloud offering proved ideal for these requirements—the setup process took less than a minute, and I had a production-ready broker with built-in security, monitoring dashboards, and guaranteed uptime. This cloud-first architecture not only simplifies the current implementation but also creates a foundation for scaling to multiple sensors, implementing distributed data processing, and building collaborative environmental monitoring networks without the constraints of local hardware dependencies.
The usage data is also quite compelling: each payload has a size of approximately 800 bytes, and with the default publishing frequency of 145 seconds, this translates to just 596 publish events per day—roughly half a megabyte of daily traffic. This minimal footprint falls well within the range of HiveMQ's free tier limits, making it a cost-effective solution for individual enthusiasts while maintaining the infrastructure needed for future expansion to multi-sensor networks:
Conclusion: Unlocking New Possibilities for Open Environmental Data
This project successfully demonstrates how a cloud-based MQTT broker can transform the capabilities of Sensor.Community's IoT environmental monitoring system by bridging the gap between limited sensor connectivity firmware and real-time data infrastructure. By developing a containerized FastAPI service that intercepts sensor readings and publishes them to HiveMQ, the solution unlocks critical functionality that wasn't previously available: real-time data visualization through Node-RED dashboards, persistent storage for historical analysis in SQL databases, and a scalable foundation for advanced use cases like machine learning anomaly detection and air quality forecasting.
The architecture can be particularly practical for hobbyists and environmental enthusiasts, as it operates well within HiveMQ's free tier limits while maintaining enterprise-grade security through TLS encryption and authentication. With modest cloud resources and open-source tools, any data developer can build end-to-end IoT monitoring solutions that contribute to global environmental awareness.
Future Enhancements
With the core infrastructure now operational, I am planning several enhancements to further leverage the real-time data capabilities that HiveMQ enables:
- Implement dynamic data quality validation that references historical data to identify anomalous data readings
- Integrate an LLM agent or a machine learning model for predictive air quality forecasting or anomaly detection
- Develop pre-aggregated data products to accelerate downstream analytics and reporting workflows.
