MQTT Essentials Part 3: Client, Broker and Connection Establishment
Welcome to the third part of the MQTT Essentials. A blog series about the core features and concepts in the MQTT protocol. In this post, we’ll discuss the role of MQTT client and broker and the parameters and options available, when connecting to a broker.
In the last post, we explained how the publish/subscribe pattern works and how it is applied in MQTT. The following is a quick recap of the essence: Publish/Subscribe decouples a client, which is sending a particular message (called publisher) from another client (or more clients), which is receiving the message (called subscriber). In order to determine, which message gets to which client, MQTT uses topics. A topic is a hierarchical structured string, which is used for message filtering and routing (More details).
The last post was really more academical nature as we examined what publish/subscribe is about and how it can be differentiated from a message queuing approach. This post will be way more practical and stuffed with basic knowledge about MQTT. Some topics we discuss are definition of MQTT client & broker, basics of an MQTT connection, the Connect Message with its parameters and the establishing of the connection by the acknowledgement of the broker.
As we have seen MQTT decouples publisher and subscriber, so a connection of any client is always with the broker. Before we start diving into the connection details, let’s make clear what we mean by client and broker.
The counterpart to a MQTT client is the MQTT broker, which is the heart of any publish/subscribe protocol. Depending on the concrete implementation, a broker can handle up to thousands of concurrently connected MQTT clients. The broker is primarily responsible for receiving all messages, filtering them, decide who is interested in it and then sending the message to all subscribed clients. It also holds the session of all persisted clients including subscriptions and missed messages (More details). Another responsibility of the broker is the authentication and authorization of clients. And at most of the times a broker is also extensible, which allows to easily integrate custom authentication, authorization and integration into backend systems. Especially the integration is an important aspect, because often the broker is the component, which is directly exposed on the internet and handles a lot of clients and then passes messages along to downstream analyzing and processing systems. As we described in one of our early blog post subscribing to all message is not really an option. All in all the broker is the central hub, which every message needs to pass. Therefore it is important, that it is highly scalable, integratable into backend systems, easy to monitor and of course failure-resistant. For example HiveMQ solves this challenges by using state-of-the-art event driven network processing, an open plugin system and standard providers for monitoring.
The MQTT protocol is based on top of TCP/IP and both client and broker need to have a TCP/IP stack.
The MQTT connection itself is always between one client and the broker, no client is connected to another client directly. The connection is initiated through a client sending a CONNECT message to the broker. The broker response with a CONNACK and a status code. Once the connection is established, the broker will keep it open as long as the client doesn’t send a disconnect command or it looses the connection.
MQTT connection through a NAT
It is a common use case that MQTT clients are behind routers, which are using network address translation (NAT) in order to translate from a private network address (like 192.168.x.x, 10.0.x.x) to a public facing one. As already mentioned the MQTT client is doing the first step by sending a CONNECT message. So there is no problem at all with clients behind a NAT, because the broker has a public address and the connection will be kept open to allow sending and receiving message bidirectional after the initial CONNECT.
Client initiates connection with the CONNECT message
So let’s look at the MQTT CONNECT command message. As already mentioned this is sent from the client to the broker to initiate a connection. If the CONNECT message is malformed (according to the MQTT spec) or it takes too long from opening a network socket to sending it, the broker will close the connection. This is a reasonable behavior to avoid that malicious clients can slow down the broker.
A good-natured client will send a connect message with the following content among other things:
Additionally there are other informations included in a CONNECT message, which are more a concern to the implementer of a MQTT library than to the user of a library. If you are interested in the details have a look at the MQTT 3.1.1 specification.
So let’s go through all these options one by one:
The client identifier (short ClientId) is an identifier of each MQTT client connecting to a MQTT broker. As the word identifier already suggests, it should be unique per broker. The broker uses it for identifying the client and the current state of the client. If you don’t need a state to be hold by the broker, in MQTT 3.1.1 (current standard) it is also possible to send an empty ClientId, which results in a connection without any state. A condition is that clean session is true, otherwise the connection will be rejected.
The clean session flag indicates the broker, whether the client wants to establish a persistent session or not. A persistent session (CleanSession is false) means, that the broker will store all subscriptions for the client and also all missed messages, when subscribing with Quality of Service (QoS) 1 or 2. If clean session is set to true, the broker won’t store anything for the client and will also purge all information from a previous persistent session.
MQTT allows to send a username and password for authenticating the client and also authorization. However, the password is sent in plaintext, if it isn’t encrypted or hashed by implementation or TLS is used underneath. We highly recommend to use username and password together with a secure transport of it. In brokers like HiveMQ it is also possible to authenticate clients with an SSL certificate, so no username and password is needed.
The will message is part of the last will and testament feature of MQTT. It allows to notify other clients, when a client disconnects ungracefully. A connecting client will provide his will in form of an MQTT message and topic in the CONNECT message. If this clients gets disconnected ungracefully, the broker sends this message on behalf of the client. We will talk about this in detail in an individual post.
The keep alive is a time interval, the clients commits to by sending regular PING Request messages to the broker. The broker response with PING Response and this mechanism will allow both sides to determine if the other one is still alive and reachable. We’ll talk about this in detail in a future post.
That are basically all information that are necessary to connect to a MQTT broker from a MQTT client. Often each individual library will have additional options, which can be configured. They are most likely regarding the specific implementation, for example how should queued message be stored.
Broker responds with the CONNACK message
When a broker obtains a CONNECT message, it is obligated to respond with a CONNACK message. The CONNACK contains only two data entries: session present flag, connect return code.
Session Present flag
The session present flag indicate, whether the broker already has a persistent session of the client from previous interactions. If a client connects and has set CleanSession to true, this flag is always false, because there is no session available. If the client has set CleanSession to false, the flag is depending on, if there are session information available for the ClientId. If stored session information exist, then the flag is true and otherwise it is false. This flag was added newly in MQTT 3.1.1 and helps the client to determine, whether it has to subscribe to topics or if these are still stored in his session.
Connect acknowledge flag
The second flag in the CONNACK is the connect acknowledge flag. It signals the client, if the connection attempt was successful and otherwise what the issue is.
In the following table you see all return codes at a glance.
|Return Code||Return Code Response|
|1||Connection Refused, unacceptable protocol version|
|2||Connection Refused, identifier rejected|
|3||Connection Refused, Server unavailable|
|4||Connection Refused, bad user name or password|
|5||Connection Refused, not authorized|
A more detailed explanation of each of these can be found in the MQTT specification.
You maybe ask, how MQTT keeps the connection open, even when there are no messages send? Or how to know when a connection is lost? You have to be patient, but we will devote a whole blog inside of the essentials series to that topic later on.
So that’s the end of part three in our MQTT Essentials series. We hope you learned at least one new thing about MQTT and looking forward to the next post about how publishing, subscribing and unsubscribing works in MQTT.
If you want to be notified as soon as the next part is released, simply sign up for our newsletter below. This brings you fresh content about MQTT and HiveMQ once a week. If you prefer RSS, you can subscribe to our RSS feed here.