MQTT Essentials Part 3: Client, Broker and Connection Establishment

MQTT Essentials Part 3

Welcome to the third part of the MQTT Essentials. A blog series about the core features and concepts in the MQTT protocol. In this post, we’ll discuss the role of MQTT client and broker and the parameters and options available, when connecting to a broker.

In the last post, we explained how the publish/subscribe pattern works and how it is applied in MQTT. The following is a quick recap of the essence: Publish/Subscribe decouples a client, which is sending a particular message (called publisher) from another client (or more clients), which is receiving the message (called subscriber). In order to determine, which message gets to which client, MQTT uses topics. A topic is a hierarchical structured string, which is used for message filtering and routing (More details).

The last post was really more academical nature as we examined what publish/subscribe is about and how it can be differentiated from a message queuing approach. This post will be way more practical and stuffed with basic knowledge about MQTT. Some topics we discuss are definition of MQTT client & broker, basics of an MQTT connection, the Connect Message with its parameters and the establishing of the connection by the acknowledgement of the broker.


As we have seen MQTT decouples publisher and subscriber, so a connection of any client is always with the broker. Before we start diving into the connection details, let’s make clear what we mean by client and broker.


When talking about a client it almost always means an MQTT client. This includes publisher or subscribers, both of them label an MQTT client that is only doing publishing or subscribing. (In general a MQTT client can be both a publisher & subscriber at the same time). A MQTT client is any device from a micro controller up to a full fledged server, that has a MQTT library running and is connecting to an MQTT broker over any kind of network. This could be a really small and resource constrained device, that is connected over a wireless network and has a library strapped to the minimum or a typical computer running a graphical MQTT client for testing purposes, basically any device that has a TCP/IP stack and speaks MQTT over it. The client implementation of the MQTT protocol is very straight-forward and really reduced to the essence. That’s one aspect, why MQTT is ideally suitable for small devices. MQTT client libraries are available for a huge variety of programming languages, for example Android, Arduino, C, C++, C#, Go, iOS, Java, JavaScript, .NET. A complete list can be found on the MQTT wiki.


The counterpart to a MQTT client is the MQTT broker, which is the heart of any publish/subscribe protocol. Depending on the concrete implementation, a broker can handle up to thousands of concurrently connected MQTT clients. The broker is primarily responsible for receiving all messages, filtering them, decide who is interested in it and then sending the message to all subscribed clients. It also holds the session of all persisted clients including subscriptions and missed messages (More details). Another responsibility of the broker is the authentication and authorization of clients. And at most of the times a broker is also extensible, which allows to easily integrate custom authentication, authorization and integration into backend systems. Especially the integration is an important aspect, because often the broker is the component, which is directly exposed on the internet and handles a lot of clients and then passes messages along to downstream analyzing and processing systems. As we described in one of our early blog post subscribing to all message is not really an option. All in all the broker is the central hub, which every message needs to pass. Therefore it is important, that it is highly scalable, integratable into backend systems, easy to monitor and of course failure-resistant. For example HiveMQ solves this challenges by using state-of-the-art event driven network processing, an open plugin system and standard providers for monitoring.

MQTT Connection

The MQTT protocol is based on top of TCP/IP and both client and broker need to have a TCP/IP stack.


The MQTT connection itself is always between one client and the broker, no client is connected to another client directly. The connection is initiated through a client sending a CONNECT message to the broker. The broker response with a CONNACK and a status code. Once the connection is established, the broker will keep it open as long as the client doesn’t send a disconnect command or it looses the connection.

MQTT Connection Flow

MQTT connection through a NAT

It is a common use case that MQTT clients are behind routers, which are using network address translation (NAT) in order to translate from a private network address (like 192.168.x.x, 10.0.x.x) to a public facing one. As already mentioned the MQTT client is doing the first step by sending a CONNECT message. So there is no problem at all with clients behind a NAT, because the broker has a public address and the connection will be kept open to allow sending and receiving message bidirectional after the initial CONNECT.

Client initiates connection with the CONNECT message

So let’s look at the MQTT CONNECT command message. As already mentioned this is sent from the client to the broker to initiate a connection. If the CONNECT message is malformed (according to the MQTT spec) or it takes too long from opening a network socket to sending it, the broker will close the connection. This is a reasonable behavior to avoid that malicious clients can slow down the broker.
A good-natured client will send a connect message with the following content among other things:

MQTT Connect message content

Additionally there are other informations included in a CONNECT message, which are more a concern to the implementer of a MQTT library than to the user of a library. If you are interested in the details have a look at the MQTT 3.1.1 specification.
So let’s go through all these options one by one:


The client identifier (short ClientId) is an identifier of each MQTT client connecting to a MQTT broker. As the word identifier already suggests, it should be unique per broker. The broker uses it for identifying the client and the current state of the client. If you don’t need a state to be hold by the broker, in MQTT 3.1.1 (current standard) it is also possible to send an empty ClientId, which results in a connection without any state. A condition is that clean session is true, otherwise the connection will be rejected.

Clean Session

The clean session flag indicates the broker, whether the client wants to establish a persistent session or not. A persistent session (CleanSession is false) means, that the broker will store all subscriptions for the client and also all missed messages, when subscribing with Quality of Service (QoS) 1 or 2. If clean session is set to true, the broker won’t store anything for the client and will also purge all information from a previous persistent session.


MQTT allows to send a username and password for authenticating the client and also authorization. However, the password is sent in plaintext, if it isn’t encrypted or hashed by implementation or TLS is used underneath. We highly recommend to use username and password together with a secure transport of it. In brokers like HiveMQ it is also possible to authenticate clients with an SSL certificate, so no username and password is needed.

Will Message

The will message is part of the last will and testament feature of MQTT. It allows to notify other clients, when a client disconnects ungracefully. A connecting client will provide his will in form of an MQTT message and topic in the CONNECT message. If this clients gets disconnected ungracefully, the broker sends this message on behalf of the client. We will talk about this in detail in an individual post.


The keep alive is a time interval, the clients commits to by sending regular PING Request messages to the broker. The broker response with PING Response and this mechanism will allow both sides to determine if the other one is still alive and reachable. We’ll talk about this in detail in a future post.

That are basically all information that are necessary to connect to a MQTT broker from a MQTT client. Often each individual library will have additional options, which can be configured. They are most likely regarding the specific implementation, for example how should queued message be stored.

Broker responds with the CONNACK message

When a broker obtains a CONNECT message, it is obligated to respond with a CONNACK message. The CONNACK contains only two data entries: session present flag, connect return code.

Session Present flag

The session present flag indicate, whether the broker already has a persistent session of the client from previous interactions. If a client connects and has set CleanSession to true, this flag is always false, because there is no session available. If the client has set CleanSession to false, the flag is depending on, if there are session information available for the ClientId. If stored session information exist, then the flag is true and otherwise it is false. This flag was added newly in MQTT 3.1.1 and helps the client to determine, whether it has to subscribe to topics or if these are still stored in his session.

Connect acknowledge flag

The second flag in the CONNACK is the connect acknowledge flag. It signals the client, if the connection attempt was successful and otherwise what the issue is.

MQTT Connack contents

In the following table you see all return codes at a glance.

Return Code Return Code Response
0 Connection Accepted
1 Connection Refused, unacceptable protocol version
2 Connection Refused, identifier rejected
3 Connection Refused, Server unavailable
4 Connection Refused, bad user name or password
5 Connection Refused, not authorized

A more detailed explanation of each of these can be found in the MQTT specification.

Loose ends

You maybe ask, how MQTT keeps the connection open, even when there are no messages send? Or how to know when a connection is lost? You have to be patient, but we will devote a whole blog inside of the essentials series to that topic later on.

So that’s the end of part three in our MQTT Essentials series. We hope you learned at least one new thing about MQTT and looking forward to the next post about how publishing, subscribing and unsubscribing works in MQTT.

If you want to be notified as soon as the next part is released, simply sign up for our newsletter below. This brings you fresh content about MQTT and HiveMQ once a week. If you prefer RSS, you can subscribe to our RSS feed here.


  1. Ahmed Al-Haddad says:

    Wow you guys. Awesome work seriously!!!

  2. C-: says:

    Can the following sentence be improved?

    “Once the connection is initiated MQTT will keep it open as long as the client doesn’t send a disconnect command or it looses the connection (More details on how this is recognizes is in a later post).”


    1. Hi,

      thanks for the feedback! We fixed that sentence and it should be easier to read now.

      Dominik from the HiveMQ Team

  3. Sundeep Saluja says:

    Great Link for a quick go thru the MQTT essential.. Kudos,, Thanks

  4. vj_time says:

    very good work! guys. thank you!

  5. vj_time says:

    great work! Guys! thank you!

  6. sumee says:

    Thanks for the information:-)

  7. vincent says:

    Useful info for new learners.Thanks for the materials.

  8. Nhat Dang says:

    Hi HiveMQ Team, It’s a great topic.
    I’m studying about ESP8266 sending mqtt message to server
    Here is example to send HTTP get message with ESP8266:
    GET /api/xxx HTTP/1.1\r\nHost:\r\nContent-Type: application/json\r\nCache-Control: no-cache\r\nConnection: close\r\n\r\n

    so I read the MQTT packet format in, I see the packet is built from bytes, I wonder if I could send a MQTT message format like this:
    CONNECT\r\nclientId:”xxx”\r\ncleanSession:true\r\n ….

    1. Hi Nhat Dang,

      this won’t work, but there is the PubSubClient by Nick O’Leary which is a MQTT client that supports the ESP8266.
      You can read a quick introduction here:

      Hope that helps,
      Christoph from the HiveMQ Team

    2. Nhat Dang says:

      Hi Christoph ,
      I’m using TM4C123GXL to communicate with esp8266
      and I’m still looking for some library for IDE like ENERGIA or Code Composer Studio (CCS) to program with TM4C
      Do you know any library ???

      Thank you very much

    3. I am not familiar with TM4C.

  9. David says:

    Hiya, thanks for the great post. I am relatively new to MQTT. Actually i’m trying to set a reconnection loop using MQTT and actually i don’t have any clues about how to do it. Example: I’m connected to MQTT broker and sending messages normally, then my connection falls appart, so i lose connection and i need to restart my script again. Is there a way to set a loop so i’ll keep trying to reconnect until my connection is fine again? Thanks in advance!

    1. Hi David,

      Thanks for your comment. Of course that is possible, but that is depending on the MQTT library you are using.

      You can learn more about MQTT libraries and if reconnecting is supported out of the box in our MQTT client encyclopedia ( If there is no reconnection functionality you need to build it by yourself or contact the developer of the library.

      Hope that helps,
      Christian from the HiveMQ Team

  10. Jason says:

    What is the best design pattern for connecting and sending sensor data in an IoT application with “10s of thousands” of sensors?

    a.) Connect and stay connected while continually pushing sensor data to the Broker server
    b.) On a timed basis, say every 30 seconds: connect, push data, disconnect right away

    I would think option a.) would overburden the Broker host with too many connections, while b.) can burden it with continuous re-authentication/authorizations, etc. each time it reconnects. Please advise.

    1. Hi Jason,

      In general I would recommend option a). Keeping connections open is generally quite cheap in comparison to opening new connections (especially if SSL/TLS is in use). And if your sensor data is coming from small restricted devices, which is usually the case, a TLS handshake can be a lot more expensive in terms of CPU, memory and bandwidth than sending a MQTT PING packet every once in a while. Also MQTT brokers are prepared to keep a lot of connections open at the same time.

      Hope that helps,

    2. Stefano says:

      Hi everybody,
      I’ve the same problem: I don’t have any problems of CPU or memory or bandwidth, so I would to implement the ‘b’ pattern (when my sensor produces data, I create a task which connect, push data, disconnect right away and then dies).

      Unfortunately, in this sequence, data never arrive!
      Instead, if I delay the Mqtt disconnect from broker, data arrive with success to broker.

      Am I doing something wrong?
      Thanks in advance.

    3. Hi Stefano,

      it seems like you are closing your client’s connection before the publish is finished.

      You need to know that all MQTT operations are handled asynchronous (Synchronization decoupling). If you want to disconnect right after a publish, you need to implement a callback like “deliveryComplete” on Paho Java client to make sure that the publish is completed.

      Hope this helps,
      Abdullah from the HiveMQ Team.

  11. ravi teja says:

    I’m sending CONNECT to ip> with port 1883 directly through my socket progarm but it is not getting connected .please send the format which i need to send to server through tcp get CONNACK
    and the data format all commands..

    1. Hallo Ravi,

      an exact definition of the data needed for a connect message can be found in the OASIS MQTT Spec.
      We did experience some difficulties with connectivity in general yesterday, so the error could have been on the broker’s side. These difficulties have been resolved by now and you can connect to it again.

      I hope this helps,
      Florian from the HiveMQ Team

  12. venkat says:

    hai i want to develop a small chat application using mqtt in python programing . how to connect mqtt client using javascript

    1. Hallo venkat,

      We actually have a dedicated blogpost on this matter.

      Good luck with your chat application.

      Kind regards,
      Florian, from The HiveMQ Team.

  13. Issam says:

    How can I change the MQTT broker hostname?

    1. Hallo Isam,

      The hostname is the address used, to reach the machine, the broker is running on.
      So it’s either an IP-Address (for example if you want to connect to a broker, running on your local machine) or a domain that will resolve to the IP-Address. (localhost in the described example)

      Hope that helps.
      Best regards,
      Florian, from The HiveMQ Team.

  14. Amit Kumar says:

    Hi Guys,

    Thanks for the explanation, I want to know how can I run MQTT broker on my own system and accept requests from clients which is nothing but different sensors data?

    Kind Regards

    1. Hallo Amit,

      See the installation guide for the HiveMQ broker.
      You can download the broker here.

      Kind regards,
      Florian from The HiveMQ Team.

  15. Maria says:

    I can’t connect to broker.
    the command pronpt keeps showing this message;
    “Connection failed to broker”.
    What’s happening?
    I am new at this, and I’m trying to connect a NodeMCU version 1.0 to the IoT node-red.

    1. Hallo Maria,

      Glad to see you’re taking an interest in MQTT.
      The message you quoted is very generic. My best guess would be that either the broker you connecting to is not running or that you are using a wrong IP-Adress or Port for connection.

      Best regards,
      Florian from the HiveMQ Team.

  16. Sayed says:

    Hi team HiveMQ,
    This is the best mqtt tutorial i have ever found. It clears so many doubts. Still one thing is unclear in my mind is how broker identify the client? For instance i have my esp8266 as a client behind the router and a cluod as mqtt broker. As esp8266 is having a dynamic ip which changes everytime when router get restarted and router’s public ip too. So if i publich a message for this esp8266 how broker will identify if it is the same client? Is there any dependency on client’s ip address? Or is it just the topic on which broker identify the client.


    1. Hallo Sayed,

      Thank you very much for the kind words. The client is identified by the so called “clientID”, that can be set in the connection options. It is not dependant on the IP-address.

      Kind regards,
      Florian, from The HiveMQ Team

  17. Sherif says:

    Thank you for the post. What happens when two clients with the same clientID connect? My use case is many threads listening to the same topic, do I need a unique ID for each?

    1. Hello Sherif,

      When a second connects, using a clientID that is already in use by a connected client, the ‘old’ client will get disconnected. This mechanism is called “client take over” and is covered in depth on the 10th part of this blog series.
      So yes – unique IDs are a good idea.

      Kind regards,
      Florian from the HiveMQ Team.

  18. Amit Ghodke says:

    how to publish birth message from android mqttclient to kapua server?
    i am able to publish message to kapua server which is using mqtt but on server side my device is not appearing in device list only showing in connections and not message is received at server side following is code .
    can anybody help.

    String topic = “test/”+clientId;
    String payload = “the payload”;
    byte[] encodedPayload = new byte[0];
    try {
    encodedPayload = payload.getBytes(“UTF-8″);
    MqttMessage message = new MqttMessage(encodedPayload);

    client.publish(topic, message);
    } catch ( Exception e) {
    showToast(clientId+” Error\n”+e.getMessage());

  19. Yogesh Bhatia says:


    I have a question….if i enter a Broker URL, which is not pingable via the MQTT client and MQTT client calls the connection, then my clinet is waiting for 120 seconds before giving error that broker is unreachable. My keepalive time is 60 seconds. Can this 120 seconds time is standard for can i configure it ?

    1. Hi Yogesh,

      The keepAlive value is designed, so the broker can recognise broken connections. When a client does not sent any packets for and and a half times the configured keepAlive value, the broker will disconnect the connection. This value has no impact on the client. The waiting time for a client before it will view a connection as faulty is depending on each individual client implementation.
      There is no MQTT standard time for that.

      Kind regards,
      Florian from The HiveMQ Team.

  20. Yogesh Bhatia says:

    Hi Florian,

    I am presenting my previous question in a different way. If my Mqtt client is trying to connect to a broker url (which doesnt not exist ), mosquitto_connect() is blocked for 120 second (which is too much for my system). Can i somehow reduce that ? Because i cant find any parameter in MQTT api to reduce or remove this blocking call. If i use mosquitto_connect_async(), then i get return value of 0 (which means connection with broker is SUCCESS), even if broker url is invalid.

    1. Hi Yogesh,

      Thank you for clarifying your query.
      Sounds like you are using the mosquitto lib.
      For detailed questions about that library please refer to mosquitto.

      Sorry I can’t be of more help and kind regards,
      Florian from The HiveMQ Team.

  21. Sandhya says:

    Is there a way I can have different authentication implementation for different set of clients? I need one kind of authentication for devices which publish data and another set for the consumers. How do I differentiate between these two?

    1. Hi there,

      nice to see you’re taking an interest in MQTT and HiveMQ.
      There is multiple ways of differentiating between clients. Good practise is to use the clientID for this.

      Kind regards,
      Florian from The HiveMQ Team.

  22. Anuprit says:


    Firstly thanks a lot for this post. I want to set up a local mqtt server and clients connected to it with no internet connection, it will be a local network. Can i use HIVEMQ for this purpose and how will the connection between clients and local server will be a secure connection using HIVEMQ.

    1. Hi Anpurit,

      Thank you for the kind words. It is great to see you are taking an interest in MQTT and HiveMQ.
      You can absolutely use HiveMQ in a local network. Just make sure your to use to HiveMQ’S private IP address, when connecting the clients.
      Regarding security concerns and possibilities I would recommend reading the Security Fundamentals, a blog post series dedicated to the subject.

      Kind regards,
      Florian from The HiveMQ Team.

  23. Biliyaminu Umar says:

    Hello, Good work you are doing. Please how can i use the number of CONNECT and DISCONNECT packets to obtain the number of active, inactive, total and expired clients of the broker. This is assuming i cant get the number of clients directly.


    1. Hi Biliyaminu,

      Nice to see you’re taking an interest in MQTT.

      If you’re using HiveMQ as your MQTT broker than you can use this guide to setup a monitoring dashboard to see all metrics.

      For any other broker solutions we recommend contacting the support of the broker solution your choice.

      Kind regards,
      Abdullah from The HiveMQ Team.

Leave a Reply

Your email address will not be published. Required fields are marked *