MQTT Essentials

The standardized and improved version of the MQTT specification that the OASIS technical committee released in 2014.

The latest version of the MQTT specification. The OASIS MQTT 5 release features significant enhancements for scalability and large-scale systems.

An MQTT broker is a server that receives all messages from the MQTT clients and then routes the messages to the appropriate destination clients. The broker is at the heart of any publish/subscribe protocol.

The MQTT 5 feature that allows the MQTT client to define whether the MQTT broker continues a previous session for the client or starts a fresh session. The clean start flag is set in the CONNECT package of the client.

Any device (from a microcontroller up to a fully-fledged server) that runs an MQTT library and connects to an MQTT broker over a network. Both publishers and subscribers are MQTT clients.

The client identifier (ClientId) identifies each MQTT client that connects to an MQTT broker. The broker uses the client ID to identify the client and the current state of the client. This ID should be unique per client and broker.

The connection between an MQTT client and the MQTT broker. MQTT clients never connect directly. Once the MQTT connection is established, the broker maintains the connection until the client sends a disconnect command or the connection breaks.

Refers to the growing network of Internet-connected devices around the world that can send and receive data over a wireless network with little or no human assistance.

A subset of IoT that enhances manufacturing and industrial processes. IIoT brings together smart devices, machine learning, advanced analytics, computers, and people to create intelligent industrial operations.

Keep-Alive number of seconds that the broker permits between when a client finishes sending one MQTT packet and starts to send the next.

The last will message is part of the Last Will and Testament (LWT) feature of MQTT. The will message notifies other clients when a client disconnects ungracefully.

See Packet.

Message Expiry is an MQTT 5 feature that allows the MQTT client to set the period of time that the MQTT broker stores a specific PUBLISH message for any matching subscribers that are not currently connected. For more details, read our Session and Message Expiry Intervals blog post.

A protocol that defines the rules, formats, and functions used to send messages between machines.

The MQTT protocol defines a series of predefined MQTT control packets (messages) to manage the communication between devices and the broker.

MQTT Packet Types

Each MQTT packet contains a mandatory 2-byte header with an optional variable header and/or payload. The minimum MQTT packet size is 2 bytes. The maximum packet size is 256MB.

AUTH: A packet that MQTT brokers and clients can send after connection establishment to realize advanced authentication methods such as challenge/response or OAuth. (MQTT 5 only)

CONNECT: The packet the MQTT client sends to the MQTT broker to request a connection.

CONNACK: The packet the MQTT broker sends to the MQTT client to acknowledge receipt of the connection request.

DISCONNECT: The packet the MQTT broker or client sends upon disconnection.

PINGREQ: The packet the MQTT client sends to the MQTT broker to establish the status of the server connection.

PINGRESP: The packet the MQTT broker sends in response to a PINGREQ message from an MQTT client to indicate that the server is alive.

PUBLISH: The packet the MQTT broker or client sends to publish a message to a topic.

PUBACK: The packet the MQTT broker or client sends in response to a PUBLISH packet with QoS 1.

PUBREC: The packet the MQTT broker or client sends in response to a PUBLISH packet with QoS 2. PUBREC is the second packet of the QoS 2 protocol exchange.

PUBREL: The packet the MQTT broker or client sends in response to a PUBREC message. PUBREL is the third packet of the QoS 2 protocol exchange.

PUBCOMP: The packet the MQTT broker or client sends in response to a PUBREL message. PUBCOMP is the fourth and final message of the QoS 2 protocol exchange.

SUBSCRIBE: The packet that the MQTT client sends to the MQTT broker to subscribe to a topic. A SUBSCRIBE message can contain multiple subscriptions for a client. Each subscription is made up of a topic and a QoS level.

SUBACK: The packet the MQTT broker sends to the MQTT client in response to a SUBSCRIBE packet. The message contains the packet identifier of the original subscribe message and a list of return codes.

UNSUBSCRIBE: The packet the MQTT client sends to the MQTT broker to delete a subscription to a specific topic on the broker.

UNSUBACK: The packet that the MQTT broker sends to the MQTT client to acknowledge the deletion of a subscription. The UNSUBACK message contains the packet identifier of the original UNSUBSCRIBE message.

The packet identifier uniquely identifies a message as it flows between the MQTT client and broker. The client library and/or the broker are responsible for setting this internal MQTT identifier.

The application-specific content of an MQTT packet. The payload contains the actual content of the message. MQTT is data-agnostic. It is possible to send images, text in any encoding, encrypted data, and virtually every data in binary.

The encryption of application-specific data on the application level (typically, the MQTT PUBLISH packet payload or the CONNECT LWT payload). Payload encryption allows end-to-end encryption of application data and adds another layer of security for topics that deliver confidential information.

A client session for which the MQTT broker saves all relevant client information. When the client reconnects, information such as subscriptions is immediately available. The session is identified by the client ID that the client provides during connection establishment. To know more, read our Persistent Session and Queuing Messages blog post.

An architectural pattern that decouples the clients that send messages (the publishers) from the clients that receive messages (the subscribers). In the pub/sub model, publishers and subscribers never contact each other directly. The connection is handled by a broker that filters all incoming messages and distributes them correctly to subscribers

The Quality of Service (QoS) level is an agreement between the sender of a message and the receiver of a message that defines the guarantee of delivery for a specific message. There are 3 QoS levels in MQTT.

Quality of Service levels

QoS 0: This service level guarantees a best-effort delivery. There is no guarantee of delivery. The recipient does not acknowledge receipt of the message and the message is not stored and re-transmitted by the sender.

QoS 1: This service level guarantees that a message is delivered at least one time to the receiver. The sender stores the message until it gets a PUBACK packet from the receiver that acknowledges receipt of the message. The message can be sent or delivered multiple times.

QoS 2: This service level guarantees that each message is received only once by the intended recipients. QoS 2 is the safest and slowest service level. The guarantee requires at least two request/response flows (a four-part handshake) between the sender and the receiver.

Information the MQTT protocol supplies to indicate the result of the numerous operations. The SUBACK and UNSUBACK packets provide a list of reason codes per topic in the payload. Since MQTT v5, reason codes for CONNACK, PUBACK, PUBREC, PUBREL, PUBCOMP, DISCONNECT, and AUTH packets appear in the variable header.

Optional human-readable content that can be provided with a specific MQTT reason code to further explain the outcome.

A normal MQTT message with the retained flag set to true. The MQTT broker stores the last retained message and the corresponding QoS for the topic. Each client that subscribes to a topic pattern that matches the topic of the retained message receives the retained message immediately after they subscribe. The broker stores only one retained message per topic.

The MQTT protocol is designed for very secure communication. As an application layer protocol, MQTT offers extensive device authentication and authorization possibilities. The underlying TCP/IP transport protocol of MQTT can add additional security via TLS encryption.

See MQTT broker.

Session Expiry is an MQTT 5 feature that allows the MQTT client to define how long after the client disconnects until the session of the client expires. When a session expires, the broker removes all subscriptions, queued messages, and incomplete message transmissions that are associated with the session.

A special type of subscription that allows multiple MQTT clients to share the same subscription. The clients in the shared subscription group receive messages in an alternating fashion. This MQTT 5 feature balances the message load of a single topic across all subscribers.

An open-source software specification for MQTT. Sparkplug is designed to help the manufacturing industry seamlessly integrate data from their applications, sensors, devices, and gateways.

Topic is an MQTT-specific form of addressing that allows MQTT clients to share information.

An MQTT 5 feature that allows MQTT clients to define an integer value that the client can use as a substitute for the name of a topic. Use of a topic alias in place of the UTF-8 string topic name can reduce the size of the PUBLISH package.

A label with UTF-8 encoding that the publisher of the MQTT message assigns to the MQTT message. The MQTT broker matches this label against the subscription requests that MQTT clients submit.

A text string with UTF-8 encoding that MQTT clients use to subscribe to topics and receive publications. The MQTT broker uses to filter to distribute matching messages to interested MQTT clients. An MQTT topic filter consists of one or more topic levels. Each topic level is separated by a forward slash (topic level separator). MQTT topic filters support the use of wildcards.

Topic Filters

MQTT Wildcards: Predefined symbols that can be used in an MQTT topic filter to subscribe to multiple topics simultaneously.

Plus sign (+): A single-level wildcard that matches any name for a specific topic level.

Hash (#): A multi-level wildcard that matches all topics that match the topic levels that are specified in front of the # symbol in the topic filter.

MQTT topics that start with a $ symbol are reserved for internal statistics of the MQTT broker. Clients cannot publish messages to $ topics and $ topics are excluded from wildcard subscriptions.

An MQTT 5 feature that appends UTF-8 string key-value pairs to MQTT packets (excluding PINGREQ and PINGRESP). Within the limits of message size, an unlimited number of user properties can be added as metadata to MQTT messages to pass information between the MQTT publisher, broker, and subscriber.

MQTT is the standard protocol for messaging and data exchange for the Internet of Things. The protocol uses a publish/subscribe architecture. The technology provides a scalable and cost-effective way to connect devices over the Internet. It is able to deliver data over the Internet in near real-time and with guarantees of delivery. MQTT is designed for IoT devices - lightweight, which enables low-cost device communication.

MQTT is used by many major companies, especially in the automotive, industry 4.0, transport, and entertainment sectors. MQTT is used for data exchange between constrained devices and server applications. It keeps bandwidth requirements to an absolute minimum, handles unreliable networks, requires little implementation effort for developers, and is, therefore, ideal for machine-to-machine (M2M) communication.

MQTT follows the Publish/Subscribe paradigm. The sender (Publisher) and receiver (Subscribers) of messages communicate via so-called topics and are decoupled from each other. The connection between them is handled by the broker. The task of the broker is to filter all incoming messages and distribute them correctly to the subscribers. A client doesn’t have to pull the information it needs, the broker pushes the information to the client whenever something new is available.

An MQTT client is any device (from a microcontroller to a full-fledged server) that operates an MQTT library and connects to an MQTT broker over a network. Each MQTT client can be both publisher and/or subscriber.

An MQTT broker is at the heart of any MQTT deployment. Depending on the implementation, a broker can handle up to millions of concurrently connected MQTT clients. The broker is responsible for receiving all messages, filtering the messages, determining who is interested in each message, and sending the message to these subscribed clients.

Communication in MQTT is based on the topic principle. An MQTTtopic is a UTF-8 string that the broker uses to filter messages for each connected client. To receive messages, the client must subscribe to the topic. A topic can have one or more topic levels. Each topic level is separated by a slash (Topic Level Separator).

MQTT is designed to allow very secure communication. As an application layer protocol, it introduces extensive device authentication and authorization possibilities. The underlying TCP/IP transport protocol can add additional security via TLS encryption.

MQTT is an open protocol that is standardized by OASIS and ISO (ISO/IEC 20922:2016).

MQTT is a data-centric, extremely lightweight, binary protocol. Due to its minimal packet overhead, MQTT excels when transferring data over the wire in comparison to document-centric protocols such as HTTP. Unlike HTTP, which is based on a request/response pattern, MQTT provides push-based communication. This push is made possible via standing TCP connections.

MQTT 5 and HiveMQ 4 were released at about the same time. MQTT 5 - released March 2019 - is a binary protocol for transferring data. The centerpiece of every MQTT communication is the MQTT broker. HiveMQ itself is a first-class MQTT broker. Since release version 4 all MQTT 5 features are fully integrated in HiveMQ. HiveMQ 4 provides full MQTT 5 support.

AMQP is bi-directional, synchronous peer-to-peer messaging protocol that has high demands for implementation complexity and a larger network overhead. It is a binary wire protocol, build as replacement for message middleware with main focus on interoperability between different vendors via rich featuring and several exchange patterns. MQTT is a binary protocol with strength in simplicity to be ideal for mobile IoT app and M2M. It provides the pub/sub messaging pattern and is designed for resource-constrained devices, low bandwidth and high latency networks. MQTT is specified by the official OASIS Standard.

With the right broker implementation that supports native WebSockets, MQTT with 100% of its features set can be used via WebSockets. As they provide a bi-directional, ordered, and lossless communication channel via TCP.

MQTT uses TCP. Due to ordering requirements MQTT over UDP is not possible.

For a detailled answer, watch the MQTT Q&A Video:

Yes, to send or receive messages, the MQTT client must establish a TCP connection to the broker. However, MQTT comes with features specifically designed to cope with unstable network connections, like the broker buffering incoming messages for disconnected clients.

Yes, MQTT and Kafka can be integrated with one another. The most efficient way to do so is to use the HiveMQ Enterprise Kafka Extension.

No. MQTT is not stateless and does not provide a request/response pattern. It is a publish/subscribe application layer messaging protocol that requires a standing TCP connection and transmit messages in an instantaneous, push way.

Yes, when used in a cluster, HiveMQ can handle Network Splits very well. Its self healing mechanisms will rebuild the cluster over time and no data will be lost.

Yes, an MQTT client can connect to any cluster node and resume its session.

Yes, all prevalent load balancing solutions, both hardware and software, are fully supported by HiveMQ.