Coordinating Edge Microservices with HiveMQ and MQTT
Edge computing has shifted critical decision-making closer to where data originates. Yet most architectures still struggle to synchronize microservices running at the edge with those operating at building, campus, and enterprise layers, especially when physical devices and real-world systems are part of the environment. This challenge becomes even more complex in distributed systems, where decisions, data, and actions are spread across devices, buildings, and campuses.
In this article, we discuss the process of coordinating physical systems in a distributed environment. In typical software systems, constructs like API gateways, service meshes, and other logically centralized software components are used to mediate, secure, control, and access to other software services. This concept needs to be extended for situations where there is not a pure software environment, but also physical devices. Specifically, we’ll examine a building management system (BMS) used in a multi-campus setting, such as with a large multi-site organization or university campus. BMSs need to collect data and manage systems close to the devices, floor, or building where they are located. Data must be shared between devices at a local and campus level. Finally, this data and information must be managed, processed, collected, and aggregated locally and at a whole-organization level.
Microservices Overview
Before we dive into details about MQTT-centric systems to control real-world physical systems, we will investigate microservices and outline some important concepts. We highlight where MQTT and its paradigm are aligned with event-driven architecture generally and BMS use case specifically.
Microservices Defined
Microservice architecture is a software architecture that divides a problem domain into a set of smaller areas of responsibility. Microservices have been a popular software architecture for much of the last decade. These are a powerful way of reducing monolithic software complexity while allowing software engineers and other IT staff the flexibility of independently upgrading bits and pieces of the system in a stepwise fashion. The flexibility that microservices provide to software architects also shows benefits to system operators and maintainers. Upgrades can be done independently as needed.
In microservices, the service boundaries are well-defined. Typically, microservices only implement functionality for one relatively small area of responsibility. For example in a building management system (BMS) we will discuss later, a microservice may be responsible for detecting people in security camera video streams. Another microservice may be solely responsible for ventilation control. Yet another microservice may be responsible for relaying command messages from a centralized building control room.
General Benefits of Microservices
Microservices help organizations efficiently manage workload demand and utilize compute resources effectively. In a distributed system, this means that a smaller computing resource is required at edge locations. The compute resource can be deployed physically where it is needed. Doing so lets you match scale, energy, and network resource consumption to the needs of the service. Some complexity comes along with microservices in deployment and management due to having more components. However, when well managed, microservices provide a reasonable level of complexity with many added advantages that often outweigh the complexity.
Edge Coordination - MQTT is a Perfect Match
Edge coordination is a crucial facet of managing distributed systems. Often, large volumes of data are generated at the edge. This data is critical, but not all of it is required to be centrally stored and managed. It is not efficient nor practical to transmit all of this voluminous data.
Rather, the value, efficiency, and effectiveness come from making intelligent decisions close to the data source, then conveying the necessary information to interested components. This is again where MQTT’s Publish/Subscribe event-driven model comes into play.
The colocality of data generation and actions that need to be taken on the data often form a crisp boundary that aligns with microservices boundary definitions. In other words, the MQTT-based microservices can be built to act intelligently on the data near to where it is generated.
In case more elements of data are required to formulate context and control systems, MQTT supports flexible solutions to transmit data over dynamically changing topic trees. Additionally, the options in routing architecture using technologies like HiveMQ Data Hub and HiveMQ Bridge Extension give designers flexibility. Data Hub can inspect MQTT payload and metadata, rewrite topics and payloads, and republish onto existing or new topics.
Dynamic routing and easy, novel topologies coupled with a sound microservices architecture provide the best of both worlds where data flow is optimized. Information is retained locally and coordinated across services and segments of the topology as needed. Locality of reference can be maximized resulting in optimized bandwidth utilization, reduced computational resources at intermediate hubs like buildings and campus-level MQTT Brokers. Centralized Cloud compute resources are optimized in turn. Simply put, data is acted upon closer to where it is generated and components that need to know are informed quickly.
Ease of Use of MQTT to Coordinate Microservices
The HiveMQ MQTT Broker can act in many ways like an API gateway in software-only based microservices. MQTT is designed for high scale, fan-out, and event-driven processing. These qualities make it easy to use an MQTT broker as a central point of control. Data can be processed and acted upon locally as needed. And, derived information can be triggered and transmitted to where it is needed.
Flexible MQTT bridging is supported out of the box with HiveMQ Platform in the MQTT broker, letting architects design topologies to transmit information dynamically where it is needed.
Scenario
We start with an overview of a common use of MQTT. And, this makes a great case for the use of microservices at the edge. The scenario is a multi-campus building management system (BMS). The following diagram shows the multi-campus topology depicting buildings, HiveMQ Edge for protocol conversion, a campus-level HiveMQ Broker, and a centralized HiveMQ Broker.
Here, there is a need for each campus and each building at the campus to be able to operate semi-autonomously. The following diagram depicts a representative campus-level diagram. The HiveMQ broker at the center is the main point of coordination at the campus level.
Campus Detail
The chart below provides a representative, zoomed-in view on part of the campus connectivity from various building systems. HiveMQ Edge performs MQTT protocol conversion from non-MQTT and legacy systems. It serves as the first level of data aggregation. Then, the MQTT traffic is directed over an MQTT bridge to the campus HiveMQ Broker. As seen in the diagram, the campus broker acts as a centralized collection point for other, MQTT-native systems.
Data is needed at the campus level and the top level to provide insights and support decision-making. In this BMS scenario, intelligence must be distributed across multiple levels, including at the edges and devices, at the building level, at the campus level, and at top management levels.
BMS Components and Function
In the BMS use case, there are many sensors and systems generating data, including:
Security cameras
Motion detectors
Fire and CO alarms
Temperature and PIR sensors
Security device services - building alarms, door locks, motion detectors, and local police
Some data that is collected, processed, and tracked includes the purposes of:
Activity and Usage Statistics
Track the number of people
Maximum occupancy
Use data to plan building hours based on usage trends
Identification of:
Persons
Vehicles
Environmental conditions - like cold, heat, fire, and ventilation
Situational awareness
Energy and resource consumption
Outcomes
This data supports objectives that include outcomes of:
Enhanced comfort, safety, security
Support LEED goals
Optimize Energy and resource usage
Use data to optimize environmentals
Optimize the cost associated with resource consumption and operations
Improve physical security, reduce crime
BMS Microservices and UNS
The following table shows some of the microservices in this building management system scenario, their function, and key MQTT components. A Unified Namespace (UNS) is key to optimizing microservices for edge coordination in this BMS and other scenarios.
Note: the MQTT topics are shown abbreviated for space considerations.
Service | Function | MQTT Role |
|---|---|---|
Image Detection | Detects people, objects, or motion in camera images | Publishes detections (+/+/+/ |
Security Controller | Activates alarms, lights, locks | Subscribes to detections, publishes commands (+/+/+/ |
Image Recorder | Starts/stops video recording | Subscribes to commands (+/+/+/ |
Image Storage | Stores camera images | Subscribes to detection events, uploads images |
Activity Logger | Logs events into a time-series database | Subscribes to topics (topic filters and wildcard patterns including #, +) |
Edge Coordinator | Orchestrates workflows and retries | Subscribes/publishes to multiple services as needed |
The following sections show a sample subset of Unified Namespace (UNS) for this multi-campus BMS. This subset focuses on the security and environmental aspects of the scenario.
UNS Structure
The unified namespace typically follows the format:
{enterprise}/{site}/{building_id}/{subsystem}/{device_type}/{device_id}/{metric}/{attribute}With some sample topic filter patterns for the cameras, hvac, and doors for the fictional acmeorg:
acmeorg/+/building/+/floor/+/camera/#
acmeorg/+/building/+/floor/+/hvac/#
acmeorg/+/building/+/floor/+/door/+/lockTopic Structure
The partial topic structure for this BMS UNS may look something like:
campus/+/buidling/+/floor/+/camera/{camera_id}/detect
+/+/+/camera/{camera_id}/frame
+/+/+/security/{zone}/cmd
+/+/+/record/{camera_id}/cmd
+/+/+/log/{service}/status
+/+/+/system/healthMicroservices Diagram for BMS
The following diagram shows how the various physical elements of the building management system such as cameras, DVRs, and locks, work together with the microservices to coordinate activities.
It is helpful to look at some of the sample events emitted by various sensors, cameras, and constituent systems within the BMS. For example the following sequence of events might be triggered when an unauthorized person approaches a building entrance after hours.
Example messages and event payloads
There are many possible event flows in this building management system. For simplicity, we’ll look at an example where an unauthorized person is detected in the building and the doors are locked along with video recording.
A security camera detects event in a camera at entrance 001 that is most likely a person:
acmeorg/campusB/building01/camera/entrance001/detect → { "detectedObject": "person", "confidence": 0.92 }Leads to the following command to lock the lobby door:
acmeorg/campusB/building01/security/lobby/cmd → { "cmd": "lockDoor", "reason": "person_detected" }Which issues a further command to record video:
acmeorg/campusB/building01/record/entrance001/cmd → { "cmd": "startRec", "duration": 10 }With a follow-on status event showing the door locked:
acmeorg/campusB/building01/log/security/status → { "event": "lockEngaged", “entrance001”, "timestamp":"..." }Edge Microservices Patterns
MQTT Broker Single Entry Point
This pattern is similar to using an API Gateway in software microservices. Here all events are coordinated through a centralized broker. In the case of HiveMQ with its true, masterless clustering, this central broker cluster eliminates many single points of failure and provides a highly scalable, performant point of control and coordination.
Topics Enable Inter-service Communication: MQTT topics form the essential communication backbone for interaction between the microservices. Since these are decoupled and topic-based, the system is freed from many of the burdens of state management. Message payloads carry most of the state changes. The remainder of state is held by the HiveMQ MQTT Broker in the form of Retained Messages.
Event-Driven Triggering
At the heart of the MQTT-centric building management system is MQTT with its publish/subscribe (PUB/SUB) pattern which is perfect for building event-driven systems. As events are produced, they can be published to a scalable centralized HiveMQ broker. They do not need to be concerned with the consumers of the data, the state of that consumer or its underlying infrastructure. With HiveMQ's seamless masterless clustering, the resilience of the system is increased.
In a similar way, the consumers or subscribers of the data do not need to be concerned with the state of the physical systems producing the data. This is the power of the coupling that comes with the publish/ subscribe paradigm. If more than one service or client needs to be informed about a particular event, it can simply subscribe to that topic, or by using topic filters, a whole tree of topics of interest. Since MQTT topics are dynamic, new functionality can be added to the system with very little to no overhead or increasing complexity.
Request-Response Pattern
MQTT version 5.0 defines an optional request-response pattern that is facilitated by an option response topic and correlation data. It is recommended to use QoS 1 with this pattern for important operations and commands.
Like the name says the request-response pattern is used to initiate a request and get a response. This is a one-to-one pattern from the requestor to the requestee. It differs from the typical decoupled MQTT pattern where a published message could be received by any number of interested subscribers. MQTT version 5.0 added this request-response pattern to provide some optional, but important transaction semantics.
The request-response pattern is very useful in the scenario we consider in this article. In building management, it is often important to initiate commands to devices and ensure proper response. Several important benefits and selection criteria for using Request-Response pattern include the following:
Correlation
Correlation data is carried in the Correlation Data property of the MQTT 5 metadata. The request-response pattern is best used when you need to correlate multiple events and command messages such as locking doors in response to a control signal or automated detection of unauthorized access.
Immediate Feedback
Request-response style is good for situations where you need to provide immediate feedback from the results of a control message. For example consider the case of a command to turn on sprinklers in response to detection of a fire. The request-response pattern helps ensure positive confirmation that the sprinklers have, indeed, been activated.
Reliability and Delivery Confirmation are Critical
For critical operations where message delivery must be guaranteed and confirmed by the recipient, Request-Response offers a mechanism for end-to-end acknowledgment. Again this differs in important ways from typical MQTT use. You might think that using QoS 1 or QoS 2 will offer what you need. However, the standard PUBSUB model with QoS 1 or 2 will only guarantee delivery to the broker at best. Due to the decoupling of the publisher from the subscribers in normal MQTT semantics, there is no way to associate the publisher to any intended subscribers.
Targeted/Scoped Data Retrieval or Configuration Changes Needed
If a microservice needs to actively request specific data from an edge device or initiate a configuration change, the Request-Response pattern provides a structured way to do so; for instance, actively requesting sensor data at a specific interval or updating device settings.
Error Handling and Real-time Feedback
Feedback on the success or failure of an operation is crucial for maintaining system stability and responsiveness in edge environments. In these scenarios, real-time control of BMS equipment depends on real-time feedback and robust error handling.
Retained Messages for Persistent State
MQTT protocol offers retained messages. These messages are held by a broker so that any subscriber can know certain things upon connection. These retained messages are an effective way to communicate a persistent state to either newly connected subscriber clients or those clients that were offline or intermittently connected. Persistent messages can efficiently communicate state in a distributed system at high scale since they eliminate the need to poll devices.
Quality of Service (QoS) and Reliability
QoS 0
QoS 0 is helpful for devices and services that need frequent telemetry—temperature, camera status, water pressure, air-quality. Often these sensors and their associated data Qos 0 is lightweight and has the lowest overhead. For data like ambient temperature in a BMS, the lack of message guarantee is generally not a downside since the data does not change often and will be acquired on the next successful publish and subscribe sequence.
QoS 1
QoS 1 is best for critical events and commands like door alarms, fire alarms, CO levels, and other gas sensors that need to convey priority or urgent information. QoS 1 offers delivery guarantees with the possibility of repeating the data “at least once”. For such important events, getting extra copies is generally not a problem. We prefer getting the message with assurance of delivery and can deal with repetitions.
Persistent Sessions
Persistent Sessions are important for key services in the microservices architecture. This helps to retain state and subscriptions in case of disconnects. These key services in our multi-campus building management can include activity loggers and security controllers. These clients need to receive every message.
Edge-Specific Benefits of Microservices
Microservices at the edge provide a number of benefits including: logical organization; code isolation; stable, well-known interface contracts; independent scalability; flexibility in lifecycle operations; and more.
Logical Alignment
Microservices are organized by function and align well with various subsystems in a BMS.There are many device types within a building management system that comprise various aggregated systems like heating and ventilation air conditioning HVAC, security services, maintenance functions, fire and safety, lighting, and occupancy management.
Clean Encapsulation Boundaries
Microservices effectively encapsulate these device data and functions into logical and manageable groups. The well-defined interfaces and functions of the microservices lend themselves to further composition into larger scale services. These services can then be designed appropriately to manage the building operations functions in a natural way.
IT/OT Operations
Microservices can be updated, upgraded, and even changed ownership provided they adhere to their clear concise service boundaries and interface contract design. In this way various physical systems may be upgraded or entirely replaced with little to no outage or downtime. These clear service boundaries offer flexibility by virtue of their isolation. This is the black-box effect, other service consumers cannot know and do not know any internal information about the service. Thus, the service can be swapped out easly.
For example, an entire HVAC system could be swapped out and the microservices and MQTT infrastructure as well as any consuming applications need not know about the change. Such flexibility would not be possible in a point-to-point hardwired interface. Overall the flexibility of microservices contributes to long-term reduced cost of ownership and reliability.
Autonomy
In an IIoT Edge microservices architecture, each discrete BMS function is encapsulated as its own service to maximize team autonomy and enforce clear service boundaries. This service decomposition enables independent scaling based on workload characteristics and SLOs, allowing compute, memory, and I/O to be tuned per service without coupling to adjacent components.
Release cadence and ownership are decoupled at the service level. An edge microservices approach offers isolation and black‑box behavior that is enforced through a stable, versioned service interface contract, a unified namespace (UNS), and event schemas. These features preserve potential backward compatibility and insulation from effects of change. Microservices architecture minimizes cross-team coordination and service-impact radius during change.
Conclusion
In this article we looked at a building management system and showed how a combination of microservices architecture and MQTT features can provide the basis of an optimal highly scalable and highly performant system. The HiveMQ MQTT broker cluster can act as a functional replacement to a microservices API gateway. Additionally, you have the most options with flexible MQTT bridging and features like HiveMQ Data Hub. You can reap all of the benefits of an API gateway for centralized control and coordination of your event-driven infrastructure.
MQTT on HiveMQ supports powerful publish/subscribe mechanisms right alongside request response patterns to give the best combination of scalability and guaranteed control for devices and systems in a multi-campus building management system. You can choose the appropriate semantics for event-driven transactions while retaining the flexibility to grow your system with minimal overhead or complexity. The microservices architecture lets you click clear boundaries around logical groupings of data and apply that down to a physical level within a building management system.
A well structured, carefully thought-out unified namespace paired with microservices with clear service boundaries are essential requirements for a flexible, scalable system based on microservices architecture. Upfront design considerations yield easy coordination of events in a large distributed system. Composability and extensibility emerge from good microservice design. MQTT, with its event-driven publish-subscribe and request-response patterns, is an ideal way to coordinate activities at the device and edge levels. Finally, always remember to keep security, resilience, redundancy, and fault isolation in mind when you're designing and operating your event-based systems.
Resources
Bill Sommers
Bill Sommers is a Technical Account Manager at HiveMQ, where he champions customer success by bridging technical expertise with IoT innovation. With a strong background in capacity planning, Kubernetes, cloud-native integration, and microservices, Bill brings extensive experience across diverse domains, including healthcare, financial services, academia, and the public sector. At HiveMQ, he guides customers in leveraging MQTT, HiveMQ, UNS, and Sparkplug to drive digital transformation and Industry 4.0 initiatives. A skilled advocate for customer needs, he ensures seamless technical support, fosters satisfaction, and contributes to the MQTT community through technical insights and code contributions.
