MQTT Essentials Part 9: Last Will and Testament

mqttessentials_part9

Welcome to the ninth part of the MQTT Essentials, a blog series about the core features and concepts in the MQTT protocol. In this post we will cover the Last Will and Testament feature of MQTT.

MQTT is often used in scenarios were unreliable networks are very common. Therefore it is assumed that some clients will disconnect ungracefully from time to time, because they lost the connection, the battery is empty or any other imaginable case. Therefore it would be good to know, if a connected client has disconnected gracefully (which means with a MQTT DISCONNECT message) or not, in order to take appropriate action. The Last Will and Testament feature provides a way for clients to do exactly that.

Last Will and Testament

The Last Will and Testament (LWT) feature is used in MQTT to notify other clients about an ungracefully disconnected client. Each client can specify its last will message (a normal MQTT message with topic, retained flag, QoS and payload) when connecting to a broker. The broker will store the message until it detects that the client has disconnected ungracefully. If the client disconnect abruptly, the broker sends the message to all subscribed clients on the topic, which was specified in the last will message. The stored LWT message will be discarded if a client disconnects gracefully by sending a DISCONNECT message.

disconnect

LWT helps to implement strategies when the connection of a client drops or at least to inform other clients about the offline status.

How to specify a LWT message for a client?

The LWT message can be specified by each client as part of the CONNECT message, which serves as connection initiation between client and broker.

MQTT Connect message content

More details about establishing a connection between client and broker can be found in part 3 of the MQTT Essentials series.

When will a broker send the LWT message?

According to the MQTT 3.1.1 specification the broker will distribute the LWT of a client in the following cases:

  • An I/O error or network failure is detected by the server.
  • The client fails to communicate within the Keep Alive time.
  • The client closes the network connection without sending a DISCONNECT packet first.
  • The server closes the network connection because of a protocol error.

We will hear more about the Keep Alive time in the next post.

Best Practices – When should you use LWT?

LWT is ideal for notifying other interested clients about the connection loss. In real world scenarios LWT is often used together with retained messages, in order to store the state of a client on a specific topic. For example after a client has connected to a broker, it will send a retained message to the topic client1/status with the payload “online“. When connecting to the broker, the client sets the LWT message on the same topic to the payload “offline” and marks this LWT message as a retained message. If the client now disconnects ungracefully, the broker will publish the retained message with the content “offline“. This pattern allows for other clients to observe the status of the client on a single topic and due to the retained message even newly connected client now immediately the current status.


So that’s the end of part nine in our MQTT Essentials series. We hope you enjoyed it. In the next and last post we’ll cover the MQTT heartbeat mechanism and how the broker knows a client is online or offline.

Have a great week and we’ll hope to see you on the next MQTT Monday!

You like the MQTT Essentials? Then sign up for our newsletter and get notified on each new post as soon as its available. If you prefer RSS, you can subscribe to our RSS feed here.

27 comments

  1. Andy says:

    Hi, I am confused that if one client connected to broker I, then this client disconnected from the broker I unexpectedly, at this time this client connected to the broker I again. could the broker send one “last will message”? thanks.

    1. Hi Andy,

      when a client gets disconnected unexpectedly, the LWT will always be sent, except if a client take-over takes place (a client with the same clientId connects and the ‘old’ client gets disconnected).

      If you really want to send a LWT on every disconnect (even graceful disconnects), then you can use a feature of HiveMQ which allows to do this. This blog post explains how to enable this: http://www.hivemq.com/hivemq-2-1-0-hivemq-artifacts-maven-central/

      Hope this helps!
      Dominik from the HiveMQ Team

  2. Sherry says:

    Hi
    I really liked your series regarding MQTT. It provides practical insight to the MQTT protocol and use cases.
    One query regarding the LWT scenario which you have given. It is said that “After the client is connected, the client sets LWT message on the same topic as “offline” and marks this LWT message as retained message “.

    My Query is : If the retained message is changed, then the newly connected clients will still get the retianed message (which is now same as LWTmessage) for when the client is still online. Can you please clairfy this, when the LWT message is made as retained message. Is it practical because of the above scenario.

    Regards
    Sherry

    1. Hi Sherry,

      thanks for the kind words!

      The newly connected clients which subscribe to that topic will receive the retained LWT message, so they can see that the client for that topic is “offline”. If this client comes back again and sets it to “online” again, then all subscribers will receive the message that the clients are online. All new subscribers will also receive the “online” retained message. The broker always sets the LWT message to “offline” if the client disconnects ungracefully. In case the client wants to disconnect gracefully, it needs to set the “offline” message manually (with a separate PUBLISH message), since LWT is only for ungraceful disconnections.

      I hope this clarifies passage.

      Best,
      Dominik from the HiveMQ Team

  3. Per Westermark says:

    I think it would have been better if your description had shown the use of a three-state solution instead of the two-state solution “online” + “offline”.

    A three-state example would be “running”, “stopped”, “dead”.

    On startup, the application registers a last will “dead”.
    It then publishes a persistent state “running”.

    So anyone interested can check the state and see that it is running.

    If the program decides to gracefully end, then it publishes a persistent state “stopped”.

    So anyone interested will know that the program has been run at least once, but currently isn’t available. And the program ended gracefully so there are no technical issues why it isn’t up and running. If the service isn’t available, then it’s probably planned maintainance or similar.

    If the program dies, or something happens with the network, then the broker itself will “replay” the registered last will, changing the persistent state topic to “dead”.

    So anyone interested will know that the program has been run at least once. But something went wrong and the program has died or in other ways failed to communicate. So maybe time to call support and report the failure.
    So anyone interested

    1. Hi Per,

      great example, the three-state solution also works beautifully with retained messages and LWT. Thanks for sharing!

      Best,
      Dominik from the HiveMQ Team

  4. Gerrit says:

    What is the behavior if cluster node has crashed or must be restarted? All client connections to this node are lost. Are LTW messages sent out after restart or from other still active nodes? Otherwise the last retained message on the client status topic is still “online”.

    1. Hi Gerrit,

      this is a great question! The MQTT specification does not cover this case.

      A HiveMQ 3.1 cluster does not send out LWT messages if a node crashes, since the node with the connected clients is responsible for sending out LWTs. The reason for that is, that it’s not possible in distributed systems to detect if a node crashed or if a split-brain scenario occurs. In case of split-brain, sending out LWT messages by other nodes would create wrong state in the cluster in split brain scenarios, because the client is actually connected. In the future there are plans to make this behaviour customizable, since there are some cases where such a split brain behaviour could potentially be beneficial. If a HiveMQ cluster node shuts down gracefully, LWT messages are sent out as expected.

      Best,
      Dominik from the HiveMQ Team

  5. Ronald says:

    Hi,

    Thanks for the nice article, I especially like the ‘online/offline’ best practice, using it now.

    I do not really get the purpose of the last will QoS. The preferred QoS is determined by the subscriber, isn’t it?

    Regards,
    Ronald

    1. Hi Ronald,

      QoS messages are never “upgraded”, so if the original publisher sent a message with QoS 0, a QoS 2 subscriber will still receive the message as QoS 0. You can find more details here: http://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels

      Hope this helps,
      Dominik

    2. Ronald says:

      Hi Dominik,

      Thanks for your quick response.

      OK, I was not aware that QoS messages are never upgraded. I wonder what the reason for this behavior is. Suppose I have a subscriber for which it is a matter of life and dead that it would receive every message exactly once. The system might even get in trouble if it receives duplicates. In that case it would subscribe at QoS level 2. Is it not strange that the publisher can determine that the QoS level of the communication between the broker and the subscriber should be any lower?

      Given the fact that QoS messages are never upgraded, would it be a best practice to always set the last will QoS to level 2? It will have no effect on the communication between the publisher and the broker, but it will at least not stop the subscribers to reach their preferred QoS level.

      Regards,
      Ronald

    3. Hi Ronald,

      awesome question!

      The main idea is that the initial publisher of a message decides on the importance of a message. If you have a life-or-death scenario, then the publisher should really use QoS 2 (+ timestamps + data integrity checks as discussed in http://www.hivemq.com/blog/mqtt-security-fundamentals-mqtt-message-data-integrity). If you have business logic which needs the message to get upgraded to a different QoS level, you could use the HiveMQ plugin system for doing this.

      Another thing you should bear in mind is, that when the connection does not drop between client and broker, each message, independent of QoS level, will be transmitted exactly once due to TCP guarantees. Only in edge cases (like the client disconnects while messages are transmitted) or you have networks that can ‘blackhole’, the QoS semantics are needed. So please use persistent sessions when you need these guarantees!

      We are strong advocates of QoS 1 instead of QoS 2 whenever possible, but if your scenario absolutely requires QoS 2 then it may be a good idea to always send QoS 2 messages. Unfortunately there is no silver bullet here :-)

      Hope this helps,
      Dominik from the HiveMQ Team

  6. NHBYK says:

    Dear mqtt users

    If a client subscribed to a topic goes offline, can mqtt send him all data that he missed while he was offline

    best regards

    1. Hi,

      yes this is possible. This blog post discusses all you need to know about it: http://www.hivemq.com/blog/mqtt-essentials-part-7-persistent-session-queuing-messages

      All the best,
      Dominik from the HiveMQ Team

  7. David says:

    Is it possible to update the last will message after a connect, so that state information may be sent that would reflect the state of client at the moment of the ungraceful disconnect.

    connect ( … last_will conatins state init )

    … 10 seconds in the future …

    update LWT

    … network outage …

    updated LWT sent to topics, not the one setup on connect.

    1. Hi David,

      currently there is no consideration of manipulating the LWT messages after the connect message in the mqtt specification.

      Regards,
      Christoph

  8. Ralf says:

    Great article, keep up the awesome work! I do have a question, though: You’re writing

    “[…] the client sets the LWT message on the same topic to the payload “offline” and marks this LWT message as a retained message.”

    I don’t see a retain field in the CONNECT packet. So how does the client set the LWT message to be retained?

    1. Ralf says:

      From how I understand the spec, the retain field of the CONNECT packet is ignored. Can you please clarify!

    2. The spec says:
      “If the Will Flag is set to 1, the Will QoS and Will Retain fields in the Connect Flags will be used by the Server, and the Will Topic and Will Message fields MUST be present in the payload [MQTT-3.1.2-9].”
      “If the Will Flag is set to 0 the Will QoS and Will Retain fields in the Connect Flags MUST be set to zero and the Will Topic and Will Message fields MUST NOT be present in the payload [MQTT-3.1.2-11].”

      This means that the retain field is only ignored if the Will Flag is set to 0.

      Hope that helps,
      Christoph from the HiveMQ Team

    3. Thanks for pointing this out, the retain field is present in the CONNECT packet, but it was missing from our image in the Connect-Part of the series.
      We updated the image to also show the retained field

  9. Gaddour Mohamed says:

    Hi,
    I used mqttws31.js MQTT javascipt client to connect to mosquitto broker with websockets.
    When the client connect to the broker it set WILL message to “offline” and publich message ‘online’ to all subscribers.
    on page onload.
    ths problem is when the page refrech or user click a link in the page and goes to other url. a disconnect event is detected by broker and offline message is send until next page load and then the ‘online’ message is published again.

    So haw to distiguich between Page closed by user (user disconnected) or the user is just navigating from one page to another ?

    1. Hi Mohamed,
      unfortunately when using an Javascript MQTT client in your browser the connection gets lost when loading a new page.
      When a new page is loaded the existing Javascript context gets destroyed and the MQTT client will ungracefully disconnect, sending it’s Last Will message.
      A possible solution to this problem would be to adjust your application in a way so the client gracefully disconnects, when a user navigates to a different page. This way no Last Will message will be sent.

      Kind regards,
      Florian from the HiveMQ Team

  10. Gaddour Mohamed says:

    Hi
    Thanks you for your response, Indeed I can set client.disconnect() on beforeunload js event and The WILL offline message will not be send..
    but haw I can detect that the user has closed the page and it became offline in this case. the status off user will remain always online. I can not find solution for this problem.

    1. Hi Mohamed,
      you will have to specify the event of a user clicking one of your links more detailed, than by the beforeunload event. This way you can have a clean disconnect for users loading a new page, while the Last Will message will be sent by clients, closing the window.

      Hope that is helpful,
      Florian from the HiveMQ Team

  11. Ervin says:

    Hi,
    Thanks for the great site. I am confuse though as to how Retained LWT Messages are updated.
    When a client connects and sets it’s willMessage as online, who’s responsibility is it to update the LWT message e.g. to “offline”? ATM my implementation using M2MQtt client always shows as online.

    1. Hallo Ervin,

      Thank you for the compliment.
      As to your question:

      The LWT message can be specified by each client as part of the CONNECT message, which serves as connection initiation between client and broker.

      In the example mentioned, the LWT is “offline”. Setting that status to “online” can not be done by the LWT, since you can only set the LWT once when connecting.

      I hope that clarifies things.
      Best regards,
      Florian, from The HiveMQ Team.

Leave a Reply

Your email address will not be published. Required fields are marked *