A Quick Guide to HiveMQ and Syslog

Logging is a key ingredient in the diagnosing, monitoring and trouble shooting of applications. Logging lets you see what your application is actually doing and helps you debug unwanted behavior.

MQTT brokers like HiveMQ are critical infrastructure components that should be monitored and connected to your company’s central logging system. When it comes to choosing a centralized logging system for your MQTT broker, there are more options available today than ever before. For example, the ELK Stack (Elasticsearch, Logstash, Kibana) and Splunk. Solutions such as Syslog continue to be very popular and command a sizable market share. Naturally, HiveMQ integrates seamlessly with a Syslog based logging infrastructure.

Customized logging

To give our customers the best possible logging experience, HiveMQ has implemented the powerful logback logging framework. Logback offers a variety of features that let you tailor your HiveMQ logging to meet your individual needs.

Consolidating your log files

HiveMQ is designed to scale to millions of concurrently connected devices and to handle a throughput of hundreds of thousands of messages per second. To reach these numbers in your MQTT deployment, you need to scale out horizontally by taking advantage of the HiveMQ cluster feature. Many HiveMQ customers run demanding deployments with dozens of HiveMQ broker nodes. When you work with such a high number of nodes, consolidating the log files from all your brokers is a must. If you don’t consolidate your log files centrally, every debugging scenario creates the necessity to manually sift through dozens of log files. This situation becomes worse if your MQTT deployment uses a load balancer (which is usually the case in ambitious MQTT deployments). If a load balancer is in use, it is not possible to identify which HiveMQ node a single client was connected to when the incident that needs to be debugged occurred.

Syslog advantages

Syslog has long been considered the defacto standard for message logging. Syslog decouples the software that generates messages from the system that stores the messages. Use Syslog to consolidate the log files from multiple HiveMQ broker nodes into a single log file that is easier to manage and analyze. A configurable prefix in the log statements ensures that each statement can be associated with the HiveMQ node on which it was created.

Enable Syslog in your HiveMQ deployment

Syslog can be enabled easily by adding a Syslog appender to the logback.xml file. Here’s an example of an appender:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

<configuration>
...

    <appender name="SYSLOG" class="ch.qos.logback.classic.net.SyslogAppender">

        // IP-Address of your syslog server
        <syslogHost>$IP-Address</syslogHost>

        <facility>user</facility>
        // replace X with the actual node
        <suffixPattern>[nodeX] %-30(%d %level)- %msg%n%ex</suffixPattern>
    </appender>

    <root level="DEBUG">
        <appender-ref ref="SYSLOG" />
    </root>

'''
    
</configuration>

Simply replace $IP-Address with the actual address of the Syslog server that you are using.

That’s it.

When you add the appender to the logback.xml configuration file, you enable the use of a Syslog server and can start benefitting from consolidated HiveMQ log files.

Have a great day,
The HiveMQ Team

About Florian Raschbichler

Florian serves as the head of the HiveMQ support team with years of first hand experience overcoming challenges in achieving reliable, scalable, and secure IoT messaging for enterprise customers.

Contact Florian