MQTT is the standard protocol for messaging and data exchange for the Internet of Things. The protocol uses a publish/subscribe architecture. The technology provides a scalable and cost-effective way to connect devices over the Internet. It is able to deliver data over the Internet in near real-time and with guarantees of delivery. MQTT is designed for IoT devices - lightweight, which enables low-cost device communication.
MQTT is used by many major companies, especially in the automotive, industry 4.0, transport, and entertainment sectors. MQTT is used for data exchange between constrained devices and server applications. It keeps bandwidth requirements to an absolute minimum, handles unreliable networks, requires little implementation effort for developers, and is, therefore, ideal for machine-to-machine (M2M) communication.
MQTT follows the Publish/Subscribe paradigm. The sender (Publisher) and receiver (Subscribers) of messages communicate via so-called topics and are decoupled from each other. The connection between them is handled by the broker. The task of the broker is to filter all incoming messages and distribute them correctly to the subscribers. A client doesn’t have to pull the information it needs, the broker pushes the information to the client whenever something new is available.
An MQTT client is any device (from a microcontroller to a full-fledged server) that operates an MQTT library and connects to an MQTT broker over a network. Each MQTT client can be both publisher and/or subscriber.
An MQTT broker is at the heart of any MQTT deployment. Depending on the implementation, a broker can handle up to millions of concurrently connected MQTT clients. The broker is responsible for receiving all messages, filtering the messages, determining who is interested in each message, and sending the message to these subscribed clients.
Communication in MQTT is based on the topic principle. An MQTTtopic is a UTF-8 string that the broker uses to filter messages for each connected client. To receive messages, the client must subscribe to the topic. A topic can have one or more topic levels. Each topic level is separated by a slash (Topic Level Separator).
MQTT is designed to allow very secure communication. As an application layer protocol, it introduces extensive device authentication and authorization possibilities. The underlying TCP/IP transport protocol can add additional security via TLS encryption.
MQTT is an open protocol that is standardized by OASIS and ISO (ISO/IEC 20922:2016).
MQTT is a data-centric, extremely lightweight, binary protocol. Due to its minimal packet overhead, MQTT excels when transferring data over the wire in comparison to document-centric protocols such as HTTP. Unlike HTTP, which is based on a request/response pattern, MQTT provides push-based communication. This push is made possible via standing TCP connections.
MQTT 5 and HiveMQ 4 were released at about the same time. MQTT 5 - released March 2019 - is a binary protocol for transferring data. The centerpiece of every MQTT communication is the MQTT broker. HiveMQ itself is a first-class MQTT broker. Since release version 4 all MQTT 5 features are fully integrated in HiveMQ. HiveMQ 4 provides full MQTT 5 support.
AMQP is bi-directional, synchronous peer-to-peer messaging protocol that has high demands for implementation complexity and a larger network overhead. It is a binary wire protocol, build as replacement for message middleware with main focus on interoperability between different vendors via rich featuring and several exchange patterns. MQTT is a binary protocol with strength in simplicity to be ideal for mobile IoT app and M2M. It provides the pub/sub messaging pattern and is designed for resource-constrained devices, low bandwidth and high latency networks. MQTT is specified by the official OASIS Standard.
With the right broker implementation that supports native WebSockets, MQTT with 100% of its features set can be used via WebSockets. As they provide a bi-directional, ordered, and lossless communication channel via TCP.
MQTT uses TCP. Due to ordering requirements MQTT over UDP is not possible.
For a detailled answer, watch the MQTT Q&A Video:
Yes, to send or receive messages, the MQTT client must establish a TCP connection to the broker. However, MQTT comes with features specifically designed to cope with unstable network connections, like the broker buffering incoming messages for disconnected clients.
Yes, MQTT and Kafka can be integrated with one another. The most efficient way to do so is to use the HiveMQ Enterprise Kafka Extension.
No. MQTT is not stateless and does not provide a request/response pattern. It is a publish/subscribe application layer messaging protocol that requires a standing TCP connection and transmit messages in an instantaneous, push way.
Yes, when used in a cluster, HiveMQ can handle Network Splits very well. Its self healing mechanisms will rebuild the cluster over time and no data will be lost.
Each sub-cluster will persist new subscriptions during the network split and after the self healing mechanisms rebuilt the cluster, the subscriptions persisted by all the sub-clusters will be UNION merged. This means that subscriptions are not lost.
In case of a partition happening, HiveMQ's true distributed and masterless cluster combined with its self healing mechanisms will ensure the cluster stays available and consistency will be reached, once the disconnected nodes can reconnect.
Yes, an MQTT client can connect to any cluster node and resume its session.
The client will resume its session seamlessly.
Yes, all prevalent load balancing solutions, both hardware and software, are fully supported by HiveMQ.
The rest of the cluster will overtake that node's duties. By default at least one other node replicates all the relevant information.
Yes, HiveMQ supports Rolling Upgrades, which means that zero-downtime upgrades are possible when using a HiveMQ cluster. All nodes can be upgraded one-by-one.
HiveMQ is designed with a true distributed and masterless cluster architecture. All persistent data and session information is distributed between the cluster nodes. Messages are being handled by the node a client is connected to and distributed to the other nodes when needed.
HiveMQ implements a sophisticated replication system. By default each node has 1 replicate within the cluster but that value can be configured. As long as a node's replicate(s) doesn't go down at the same time, the node goes down, no messages will be lost.
Sessions, including queued messages, are stored by multiple nodes. The number of replicates is configurable. Every node in the cluster can access that information.
Significant performance issues are not to be expected in this case. HiveMQ will empty the messages one by one and make sure that consumer and broker can handle the speed of delivery. We recommend anticipating an overall additional load of roughly 10-15% for such cases when setting up your system.
- Major HiveMQ versions do not have a predefined release cadence.
- HiveMQ releases three minor versions per year and provides one maintenance release per month.
- The length of support HiveMQ provides for a version is based on the minor version type:
- All HiveMQ minor-version releases are supported for at least one year.
- HiveMQ minor-version releases that have the designation LTS (long-term support) provide two years of support with the option to extend the support for a third year.
How do you know if the start up of your HiveMQ machine can be considered slow? Before that question can be answered we have to consider the two possible scenarios for the HiveMQ startup process:
- 1. The HiveMQ machine has no persistent data from previous sessions.
- 2. The HiveMQ machine has persistent data from previous sessions, which has to be loaded beforehand.
For the first scenario we consider a startup process of over 60 seconds to be slow. You can see how long your HiveMQ needed to boot in the terminal you started HiveMQ from or in the log file. The line should look similar to:
2019-01-12 13:58:01,980 INFO - Started HiveMQ in 7454ms
For the second scenario we can give no answer but you can test/fix it anyway, because our test and fix ignore the mentioned scenarios.
One possible cause of the slow start up process of HiveMQ can be the method:
In order to bind the HiveMQ server to a local address this method is called. In a normal case this method will finish after some milliseconds have passed. But sometimes an event (e.g. OS-Update) can lead to a malfunction of the method, though it still works but now instead of some milliseconds this method runs up to several seconds.
How can you test if this method is responsible for the slow start up process?Antonio Troina has written a little program that you can find here. First you need to download inetTester.jar in the bin folder. Then you run the jar file on the computer you are running the HiveMQ server with the command:
java -jar Path/To/inetTester.jarSee how much time elapsed to call getLocalHost() once. An elapsed time in the range from 0ms - 30ms is perfect. If you see a result of 500ms or more we recommend that you fix this issue.
How can I fix this problem?The issue can be fixed by adding the hostname to your local addresses in the 'hosts' file (see here to locate the hosts file for your OS). Following are the two entries you should add:
127.0.0.1 localhost hostname ::1 localhost hostname
To find your hostname type ‘hostname’ in the command line:
pc01:~ marysue$ hostname pc01.hivemq.example
For our example the fixed hosts file looks like:
127.0.0.1 localhost pc01.hivemq.example ::1 localhost pc01.hivemq.example