Using Syslog with HiveMQ
Logging is a key ingredient in the diagnosing, monitoring and trouble shooting of applications. Logging lets you see what your application is actually doing and helps you debug unwanted behavior.
MQTT brokers like HiveMQ are critical infrastructure components that should be monitored and connected to your company’s central logging system. When it comes to choosing a centralized logging system for your MQTT broker, there are more options available today than ever before. For example, the ELK Stack (Elasticsearch, Logstash, Kibana) and Splunk. Solutions such as Syslog continue to be very popular and command a sizable market share. Naturally, HiveMQ integrates seamlessly with a Syslog based logging infrastructure.
To give our customers the best possible logging experience, HiveMQ has implemented the powerful logback logging framework. Logback offers a variety of features that let you tailor your HiveMQ logging to meet your individual needs.
Consolidating your log files
HiveMQ is designed to scale to millions of concurrently connected devices and to handle a throughput of hundreds of thousands of messages per second. To reach these numbers in your MQTT deployment, you need to scale out horizontally by taking advantage of the HiveMQ cluster feature.
Many HiveMQ customers run demanding deployments with dozens of HiveMQ broker nodes.
When you work with such a high number of nodes, consolidating the log files from all your brokers is a must. If you don’t consolidate your log files centrally, every debugging scenario creates the necessity to manually sift through dozens of log files. This situation becomes worse if your MQTT deployment uses a load balancer (which is usually the case in ambitious MQTT deployments). If a load balancer is in use, it is not possible to identify which HiveMQ node a single client was connected to when the incident that needs to be debugged occurred.
Syslog has long been considered the defacto standard for message logging. Syslog decouples the software that generates messages from the system that stores the messages. Use Syslog to consolidate the log files from multiple HiveMQ broker nodes into a single log file that is easier to manage and analyze. A configurable prefix in the log statements ensures that each statement can be associated with the HiveMQ node on which it was created.
Enabling Syslog in your HiveMQ deployment
Syslog can be enabled easily by adding a Syslog appender to the
Here’s an example of an appender:
<appender name="SYSLOG" class="ch.qos.logback.classic.net.SyslogAppender">
// IP-Address of your syslog server
// replace X with the actual node
<suffixPattern>[nodeX] %-30(%d %level)- %msg%n%ex</suffixPattern>
<appender-ref ref="SYSLOG" />
Simply replace $IP-Address with the actual address of the Syslog server that you are using.
When you add the appender to the logback.xml configuration file, you enable the use of a Syslog server and can start benefitting from consolidated HiveMQ log files.