- A raw sensor value like 87.3 is useless until it has context: asset identity, process state, units and thresholds.
- HiveMQ Data Intelligence enriches data in flight at the MQTT broker. It validates schemas, normalizes units, adds asset metadata and routes messages based on content, all before subscribers receive them.
- Enrichment rules are declarative policies, not code. They’re version-controlled, auditable, testable and scale across thousands of topics with wildcards.
- Doing this once at the broker replaces the separate normalization logic in your historian, MES and analytics tools. It removes extra network hops and gives downstream models consistent data.
- For a Unified Namespace, HiveMQ Platform policies act as the quality gate, and they’re the foundation for governance and semantic modeling in HiveMQ Data Intelligence.
- Start small: audit your busiest topics, define schemas, run validation-only policies, then add enrichment one layer at a time.
Who this blog is for: This article is for IT/OT architects and solutions architects designing manufacturing data infrastructure, especially those building or planning a Unified Namespace.
Raw sensor data from a CNC machine or PLC tells you almost nothing on its own. A temperature reading of 87.3 without asset context, process state, or unit metadata is noise, not information. In-flight data enrichment is the practice of adding context, structure, and meaning to operational data as it flows through the MQTT broker, before it reaches downstream consumers. For IT/OT architects designing manufacturing data infrastructure, this capability eliminates an entire class of middleware and transforms the broker from a simple message relay into the first stage of a data intelligence pipeline.
This article walks through how HiveMQ Platform’s Contextualize layer enables in-flight enrichment, how to design policy-driven shaping strategies for manufacturing data, and why contextualizing data at the broker produces measurably higher-value downstream outputs.
Why does raw manufacturing data fall short without context?
A typical brownfield manufacturing environment generates telemetry from hundreds or thousands of sources: temperature sensors, vibration monitors, motor controllers, quality inspection stations, and environmental sensors. Each publishes data to MQTT topics, often in inconsistent formats, with no standardized schema, and without the metadata needed to make the data actionable.
Consider what a downstream system actually needs to act on a vibration reading from a spindle motor:
- Asset identity: Which machine, which spindle, which production line?
- Process context: What operation was running? What material was being cut?
- Temporal context: Where in the production cycle was this reading captured?
- Engineering units and ranges: Is 4.2 mm/s normal for this asset class, or is it an anomaly?
Without this context, every consuming application (your historian, your MES, your analytics platform) must independently resolve these questions. That means each system maintains its own mapping tables, its own normalization logic, and its own assumptions. The result is inconsistency, duplicated effort, and a fragile architecture where changing one asset's metadata requires updates across five systems.
Industry estimates suggest data scientists spend as much as 60% of their time cleaning data, according to a Rockwell Automation executive quoted in IndustryWeek. Moving that work upstream, to the point of ingestion, is one of the highest-leverage architectural decisions an IT/OT solutions architect can make.
How does in-flight enrichment work at the MQTT broker?
In-flight enrichment means the broker itself performs transformations on messages as they transit from publisher to subscriber. Rather than forwarding raw payloads unchanged, the broker applies rules that validate structure, attach metadata, normalize formats, and route based on content.
HiveMQ Platform implements this through a declarative policy engine that operates on messages in real time. Here is how the core enrichment pipeline works:
- Schema validation: Incoming messages are validated against a defined schema (JSON Schema or protobuf). Messages that fail validation are routed to error topics or dropped, depending on policy.
- Data transformation: Payloads are reshaped, fields are renamed or converted, and engineering units are normalized. A temperature published in Fahrenheit by a legacy sensor can be converted to Celsius before any subscriber sees it.
- Context injection: Static or semi-static metadata (asset identifiers, location hierarchies, process parameters) is attached to messages based on the publishing topic or client identifier.
- Conditional routing: Messages are directed to different subscribers or topic branches based on their content, enabling quality-of-data-based routing where only enriched, validated data reaches production analytics.
All of this happens within the broker process, at message transit time. There is no external microservice to deploy, no additional network hop, and no separate state store to manage.
A practical example: Enriching CNC machine vibration data
Suppose a vibration sensor on a milling machine publishes to site/chicago/line-3/mill-07/vibration:
{
"v": 4.2,
"ts": 1719403200
}A Data Intelligence policy can transform this into:
{
"measurement": "vibration_rms",
"value": 4.2,
"unit": "mm/s",
"timestamp": "2025-06-26T16:00:00Z",
"asset": {
"id": "MILL-07",
"type": "CNC_5AXIS",
"line": "LINE-3",
"site": "CHICAGO"
},
"process": {
"operation": "roughing",
"material": "AL-6061"
},
"thresholds": {
"warning": 5.0,
"critical": 8.0
}
}The downstream consumer, whether it is a real-time dashboard, a historian, or an anomaly detection model, receives a self-describing message. No lookup tables needed. No ambiguity about units or asset identity.
Embedding context directly into MQTT messages can simplify downstream consumption, but repeating static metadata in every payload also increases data volume. For strategies that preserve context without unnecessary payload overhead, read Contextual Modeling Strategies to Reduce Manufacturing Data Volume.
What does policy-driven data shaping look like in practice?
The key architectural advantage of HiveMQ Platform's approach is that enrichment logic is defined as policies, not code. Policies are declarative configurations that specify what happens to data matching certain criteria.
This distinction matters for IT/OT architects because:
- Policies are version-controlled and auditable. Every change to data shaping logic is traceable. In regulated manufacturing (pharmaceutical, automotive, food and beverage), this traceability is not optional.
- Policies scale horizontally. A single policy can apply to thousands of topics using topic filters and wildcards. Adding a new production line does not require new code; it requires a new asset entry.
- Policies are testable in isolation. You can validate a policy against sample payloads before deploying it to production, reducing the risk of breaking downstream consumers.
Designing a policy strategy for a manufacturing UNS
If you are building toward a Unified Namespace (UNS) architecture, platform policies become the enforcement layer that ensures data entering the namespace meets your standards. A practical policy strategy for a multi-site manufacturer might include:
| Policy Layer | Purpose | Example Rule |
|---|---|---|
| Schema enforcement | Guarantee structural consistency | All messages on +/+/+/temperature must include value, unit, and timestamp fields |
| Unit normalization | Eliminate unit ambiguity | Convert all temperature readings to Celsius; all pressure readings to kPa |
| Asset tagging | Attach ISA-95 hierarchy metadata | Inject site, area, line, and cell fields derived from topic path |
| Quality flagging | Mark data reliability | Add quality: "good" or quality: "uncertain" based on value range checks |
| Topic remapping | Standardize namespace structure | Remap legacy topic patterns to ISA-95-aligned UNS topics |
This layered approach means each policy handles one concern. Policies compose; they do not create monolithic transformation scripts.
How does broker-level enrichment improve downstream data quality?
The architectural payoff of enriching data at the broker is measurable in three dimensions:
Reduced integration complexity
Without broker-level enrichment, every downstream consumer implements its own normalization. In a typical manufacturing stack with a historian, MES, ERP connector, analytics platform, and one or two machine learning pipelines, that means five or six independent implementations of the same enrichment logic. Each is a maintenance burden and a source of inconsistency.
With HiveMQ Platform handling enrichment centrally, downstream systems receive ready-to-consume data. Integration effort drops significantly. Organizations running HiveMQ in production have reported reducing integration middleware by 40-60% after centralizing data shaping at the broker.
Lower latency for real-time use cases
Every external enrichment hop adds latency. A message that must travel from broker to enrichment microservice and back before reaching a subscriber adds 5-20ms per hop in a well-optimized environment, more in congested networks. For real-time quality control or closed-loop process optimization, those milliseconds matter.
In-flight enrichment at the broker adds sub-millisecond processing time because the transformation happens in-process, with no network round-trip.
Higher trust in analytics outputs
Analytics models trained on inconsistent data produce inconsistent results. When a vibration anomaly model receives readings in mixed units (some in mm/s, some in in/s, some with asset context, some without), its predictions are unreliable. Centralizing normalization and context injection at the broker ensures every data point reaching the analytics layer is structurally identical and semantically consistent.
This consistency is the foundation for the Data Intelligence capabilities in HiveMQ Intelligence, where semantic modeling and data governance build on top of already-contextualized streaming data. You cannot govern or model data that is inconsistent at the source.
Consistent data at the broker is only one part of building trustworthy manufacturing analytics. To see how contextualized MQTT data supports reliable OEE, throughput, and quality dashboards, read Building Trustworthy Manufacturing Dashboards on Contextualized MQTT Data with HiveMQ.
What is the relationship between data enrichment and a Unified Namespace?
A UNS is only as good as the data that flows through it. The design pattern calls for a single source of truth for operational data, but if that data is raw, unvalidated, and context-free, the namespace becomes a centralized dumping ground rather than a source of truth.
HiveMQ Platform policies serve as the quality gate at the UNS boundary. They ensure that:
- Only structurally valid data enters the namespace
- Every message carries the metadata needed for self-description
- Topic structures conform to the organizational hierarchy
- Data consumers can trust what they receive without independent verification
For architects evaluating the path from data streaming to data intelligence, this is the critical insight: enrichment at the broker is not a nice-to-have optimization; it is a prerequisite for building a trustworthy UNS and, eventually, for layering governance and semantic modeling on top of it.
How do you get started with Data Intelligence policies?
If you are running HiveMQ Broker, HiveMQ Platform is available as a built-in capability. Here is a practical starting path:
- Audit your highest-volume topics. Identify the 10-20 topics that carry the most traffic or serve the most consumers. These are your highest-leverage enrichment targets.
- Define schemas for those topics. Use JSON Schema to formalize what a valid message looks like. Start descriptive (documenting what exists) before becoming prescriptive (enforcing what should exist).
- Implement validation-only policies first. Before transforming data, deploy policies that validate and log violations without blocking. This reveals the gap between actual and expected data quality.
- Add enrichment incrementally. Once validation is stable, layer on unit normalization, asset tagging, and topic remapping. One policy layer at a time.
- Connect enriched data to downstream consumers. Point your historian, analytics platform, or Kafka integration at the enriched topics and retire the legacy normalization logic in those systems.
The HiveMQ Platform documentation provides schema examples, policy syntax references, and a sandbox environment for testing policies against sample payloads.
Ready to see how policy-driven enrichment works in your environment? Explore the HiveMQ Platform documentation to build your first validation policy in minutes.
