MQTT Shared Subscriptions – MQTT 5 Essentials Part 7
Shared subscriptions, a core feature of MQTT 5, enable multiple MQTT clients to share a single subscription on the broker. In essence, this feature allows messages on a topic to be distributed among multiple clients, thereby improving load balancing and fault tolerance in an MQTT system.
Shared subscriptions: Bridging the MQTT Gap with v5
MQTT 5 was ingeniously designed to connect the gap between the functionality offered by MQTT 3.1.1 and users’ expectations for the Internet of Things’ de facto standard protocol. With the integration of sought-after features such as Shared Subscriptions and Session and Message Expiry Intervals, MQTT 5 is poised to consolidate MQTT’s standing as the go-to IoT protocol for the foreseeable future.
Due to the high demand for shared subscriptions, HiveMQ introduced this feature before the MQTT 5 specification was released. As a result, all MQTT brokers striving for 100% compatibility with the MQTT 5 specification must support this feature.
How MQTT Shared Subscriptions Work?
In a standard MQTT subscription, each subscribing client is privy to a copy of each message broadcasted to that topic. With shared subscriptions, clients sharing a subscription in the same group receive messages in rotation, a process sometimes referred to as client load balancing. The message load of a single topic is distributed across all subscribers.
MQTT clients can subscribe to a shared subscription using standard MQTT mechanisms. Any standard MQTT clients, such as Eclipse Paho, can participate without requiring any modifications on the client side. It’s important to note that shared subscriptions employ a unique topic syntax for subscribing.
Shared subscriptions use the following topic structure:
The shared subscription consists of 3 parts:
- A static shared subscription identifier ($share)
- A group identifier
- The actual topic subscriptions (may include wildcards)
A concrete example for such a subscriber would be:
How do MQTT Shared Subscriptions Work in HiveMQ MQTT Broker?
A shared subscription group can be conceptually envisaged as a virtual client acting as a proxy for numerous subscribers simultaneously. HiveMQ selects one subscriber from the group and delivers the message to that client. It typically uses a round-robin approach for distribution. The following picture demonstrates the principle:
For instance, a HiveMQ deployment might feature multiple shared subscription groups. These groups can have the same subscription but different group identifiers. Whenever a publisher sends a message with a matching topic, one unique client from each group receives the message.For example, the following scenario is possible:
In the given scenario, we have two distinct groups, each encompassing two subscribing clients within their shared subscription group. Although these groups subscribe to the same topics, they are differentiated by unique group identifiers. When a publisher issues a message that aligns with a particular topic, a solitary client from each group— and only one client— is selected to receive the message.
MQTT Shared Subscription Use Cases
Shared subscriptions have a multitude of applications, particularly in high-scalability scenarios. These include:
- Client load balancing for MQTT clients that cannot handle the load on subscribed topics.
- Worker (backend) applications that ingest MQTT streams must scale horizontally.
- Optimizing subscriber node-locality for incoming publishes to reduce HiveMQ intra-cluster node traffic.
- The delivery semantics employ QoS 1 and 2 though there’s no necessity for guarantees on ordered-topic.
- Addressing hot topics causing scalability bottlenecks due to higher message rates.
How to Subscribe with Shared Subscriptions in MQTT?
Engaging your clients with shared subscriptions is a straightforward process. Below, we illustrate how to accomplish this using the MQTT CLI. The given command line execution code allows two MQTT clients to subscribe to the same subscription group and topic:
With these commands executed, both MQTT clients now have a shared subscription to the ‘my-share-topic’ (part of the virtual group ‘group1’). In this configuration, each client is allocated half of the MQTT messages dispatched over the MQTT broker on the topic ‘my-share-topic’.
Remember, the MQTT clients can join or depart from the subscription group whenever they prefer. For instance, should a third client decide to join the group, the distribution of relevant MQTT messages becomes equally divided among all three clients, each receiving one-third of the total.
For a deeper understanding of the semantics of shared subscriptions, the official HiveMQ Documentation is an excellent resource.
Scaling MQTT Subscribers with Shared Subscriptions
Shared subscriptions provide a simple way to integrate backend systems with MQTT, especially when it is not feasible to use HiveMQ’s extension system or dynamic scaling is necessary. With the shared subscriptions, you can quickly add subscribers as needed, distributing work in a push fashion.
Shared subscriptions prove immensely valuable for scaling and load-balancing MQTT clients. Furthermore, HiveMQ clusters offer additional benefits in terms of latency and scalability through internal optimization of message routing.
For comprehensive details on shared subscriptions and HiveMQ clusters, reference the official HiveMQ Documentation.
Shared subscriptions provide a compelling way to distribute messages across various MQTT subscribers using standard MQTT mechanisms. This feature simplifies implementing MQTT-client load balancing without the need for any proprietary modifications to your MQTT clients. This is especially beneficial for backend systems or “hot-topics” that might quickly overwhelm a single MQTT client. In the next article, we will explore Payload Format Description.
Sign up for our newsletter to get regular updates. Subscribe to our RSS feed here to stay updated. We encourage you to visit our MQTT Glossary for an in-depth understanding of the essential MQTT terminologies. It will equip you with the necessary vocabulary to grasp the complexities of MQTT and its various versions. Watch the video below that complements the concepts discussed in this article.
FAQs on MQTT User Properties
If all subscribers to a shared subscription in MQTT disconnect simultaneously, the behavior depends on the MQTT broker implementation and configuration. The broker may or may not retain any messages for the shared subscription topic. Some brokers may keep the shared subscription active even if there are no active subscribers, while others may automatically unsubscribe or clean up the shared subscription if all subscribers disconnect.
The MQTT specification does not define any specific restrictions on the number of shared subscriptions that can be used in an MQTT system. The number of shared subscriptions you can use in your MQTT system would typically depend on factors such as the capabilities and limitations of your MQTT broker, the resources available in your system, and the scalability requirements of your application.
No, a client cannot subscribe to multiple shared subscriptions simultaneously in MQTT. The MQTT protocol specification allows a client to subscribe to multiple topics using a single SUBSCRIBE packet, but it does not support subscribing to multiple shared subscriptions in a single operation.
Yes, shared subscriptions can be used in a clustered setup with load balancers in MQTT. The combination of shared subscriptions and load balancers can help distribute the workload across multiple MQTT brokers or broker instances, providing scalability and fault tolerance in an MQTT-based system.
About Florian Raschbichler
Florian serves as the head of the HiveMQ support team with years of first hand experience overcoming challenges in achieving reliable, scalable, and secure IoT messaging for enterprise customers.Contact Florian