Introduction

This Guide is an addendum to the official documentation and shows all upgrade steps from one HiveMQ version to another.

HiveMQ uses Semantic Versioning for its version numbers. Every HiveMQ version uses the following scheme:

major.minor.patch
major

Major versions are not downward compatible with previous versions. These are major feature releases and manual upgrade instructions need to be applied. Customers with professional support may be provided with tools for smooth migration. The plugin system interfaces may change and customers may need to upgrade their plugins to new APIs. The upgrade path is always clearly documented. Deprecated APIs may be removed in major releases.

minor

Minor versions are typically feature releases with downward compatible changes. The plugin system may receive upgrades and new APIs. Old APIs may be deprecated and may be removed in a later major version. Deprecated plugin APIs stay in place and won’t get removed.

patch

Patch releases (also known as bugfix releases) don’t need manual upgrade steps and are drop-in replacements for the current minior version.

Hotfix Releases
Customers with professional support can receive hotfix releases which don’t use semantic versioning. These hotfix releases are always customer specific and are typically downward compatible. All customers which receive hotfix versions get detailled instructions if manual upgrade steps are needed.

3.2.x to 3.3.x Migration Guide

This is a major HiveMQ upgrade. HiveMQ 3.3 is a drop in replacement for HiveMQ 3.2.x

No manual steps are required for upgrading from HiveMQ 3.2 to HiveMQ 3.3.

HiveMQ Upgrade

  • Create a backup of the whole HiveMQ installation folder

  • Install HiveMQ 3.3 as described in the HiveMQ Installation Guide

  • Migrate the configuration file contents of your old HiveMQ installation

  • To migrate the persistent data, copy all contents in the data folder from your backup to the new HiveMQ 3.3 data folder. On first start HiveMQ 3.3 will migrate the persistent data from your previous installation into its new persistent storage.

Configuration File Upgrade

HiveMQ 3.3 introduces some changes in the configuration file for:

Persistent Data Migration

HiveMQ 3.3 automatically migrates all data in your data folder from previous versions to updated file storage formats.

To allow HiveMQ to migrate the persistent data from previous versions you have to copy the contents of your data folder from the previous installation to the data folder of the new installation.

Linux example
cp -r /opt/hivemq-3.2.7/data/* /opt/hivemq-3.3.0/data/

On first start HiveMQ 3.3 will automatically migrate the persistent data from your previous installation into its new persistent storage.

Cluster Upgrade

Rolling upgrades are supported and it’s possible to have HiveMQ version 3.2 and version 3.3 simultaneously in the same cluster. When all nodes are upgraded to the new version, all new cluster features are enabled by default by the HiveMQ cluster. No manual intervention is needed.

Please upgrade node for node and give some time for data synchronization after a node is up and running.

Please see the HiveMQ Clustering Documentation for more information.

Web UI

HiveMQ 3.3 has a built-in Web UI for advanced analysis and administrative tasks.

The Web UI listens on localhost and port 8080.

If you need to access the Web UI from another machine, make sure to configure the Web UI to listen on a public network interface and configure your firewall rules to allow incoming TCP traffic on port 8080.

Please see the HiveMQ Web UI Documentation for more information.

When using a load balancer like HAProxy, nginx or NetScaler for managing traffic to HiveMQ ensure that HTTP load balancing for access to the Web UI with sticky-sessions (also known as session affinity) is configured.

Plugin System Upgrade

The plugin system has an improved isolation strategy.

The sharing of state through objects or logic through classes across plugin boundaries is now prohibited.

Cross plugin object or class injection is actively prevented by HiveMQ to ensure that conflicting dependencies do not influence other plugins. The plugin system now provides each plugin with its own completely separate injector.

Existing plugins are compatible with HiveMQ 3.3 without changes.

Upgrading your plugins

In order to upgrade your plugin, execute the following steps:

  • update your pom.xml

  • remove code depending on shared objects or classes across plugins

Adapt your pom.xml file

Change your HiveMQ SPI version to 3.3.0 in your pom.xml file

<dependency>
    <groupId>com.hivemq</groupId>
    <artifactId>hivemq-spi</artifactId>
    <version>3.3.0</version>
</dependency>

3.2.x to 3.2.4 Migration Guide

This is a minor HiveMQ upgrade.

No manual steps are required for upgrading from HiveMQ 3.2.x to HiveMQ 3.2.4.

HiveMQ Upgrade

  • Create a backup of the whole HiveMQ installation folder

  • Install HiveMQ 3.2.4 as described in the HiveMQ Installation Guide

  • Migrate the configuration file contents of your old HiveMQ installation

  • To migrate the persistent data, copy all contents in the data folder from your backup to the new HiveMQ 3.2.4 data folder. On first start HiveMQ 3.2.4 will migrate the persistent data from your previous installation into its new persistent storage.

Configuration File Upgrade

HiveMQ 3.2.4 introduces some changes in the configuration file:

  • Deprecated client-bind-port for tcp transport

This setting will now be ignored and can be removed from your configuration file.

Persistent Data Migration

HiveMQ 3.2 automatically migrates all data in your data folder from previous versions to updated file storage formats.

To allow HiveMQ to migrate the persistent data from previous versions you have to copy the contents of your data folder from the previous installation to the data folder of the new installation.

Linux example
cp -r /opt/hivemq-3.2.1/data/* /opt/hivemq-3.2.4/data/

On first start HiveMQ 3.2.4 will automatically migrate the persistent data from your previous installation into its new persistent storage.

Cluster Upgrade

Rolling upgrades are supported and it’s possible to have HiveMQ version 3.2.x and version 3.2.4 simultaneously in the same cluster. When all nodes are upgraded to the new version, all new cluster features are enabled by default by the HiveMQ cluster. No manual intervention is needed.

Please upgrade node for node and give some time for data synchronization after a node is up and running.

Please see the HiveMQ Clustering Documentation for more information.

Plugin System Upgrade

The plugin system update has new features that can be used. HiveMQ 3.2.4 is completely backward compatible with older versions, so you are not required to update your plugins if you don’t want to use the new plugin system features.

Upgrading your plugins

In order to upgrade your plugin, execute the following steps:

  • update your pom.xml

Adapt your pom.xml file

Change your HiveMQ SPI version to 3.2.1 in your pom.xml file

<dependency>
    <groupId>com.hivemq</groupId>
    <artifactId>hivemq-spi</artifactId>
    <version>3.2.1</version>
</dependency>

3.1.x to 3.2.x Migration Guide

This is a major HiveMQ upgrade. HiveMQ 3.2 is a drop in replacement for HiveMQ 3.1.x

No manual steps are required for upgrading from HiveMQ 3.1 to HiveMQ 3.2.

HiveMQ Upgrade

  • Create a backup of the whole HiveMQ installation folder

  • Install HiveMQ 3.2 as described in the HiveMQ Installation Guide

  • Migrate the configuration file contents of your old HiveMQ installation

  • To migrate the persistent data, copy all contents in the data folder from your backup to the new HiveMQ 3.2 data folder. On first start HiveMQ 3.2 will migrate the persistent data from your previous installation into its new persistent storage.

Configuration File Upgrade

HiveMQ 3.2 introduces some changes in the configuration file:

Persistent Data Migration

HiveMQ 3.2 automatically migrates all data in your data folder from previous versions to updated file storage formats.

To allow HiveMQ to migrate the persistent data from previous versions you have to copy the contents of your data folder from the previous installation to the data folder of the new installation.

Linux example
cp -r /opt/hivemq-3.1.6/data/* /opt/hivemq-3.2.0/data/

On first start HiveMQ 3.2 will automatically migrate the persistent data from your previous installation into its new persistent storage.

Cluster Upgrade

Rolling upgrades are supported and it’s possible to have HiveMQ version 3.1 and version 3.2 simultaneously in the same cluster. When all nodes are upgraded to the new version, all new cluster features are enabled by default by the HiveMQ cluster. No manual intervention is needed.

Please upgrade node for node and give some time for data synchronization after a node is up and running.

Please see the HiveMQ Clustering Documentation for more information.

Plugin System Upgrade

The plugin system update has new features that can be used. HiveMQ 3.2 is completely backward compatible with older versions, so you are not required to update your plugins if you don’t want to use the new plugin system features.

Upgrading your plugins

In order to upgrade your plugin, execute the following steps:

  • update your pom.xml

Adapt your pom.xml file

Change your HiveMQ SPI version to 3.2.0 in your pom.xml file

<dependency>
    <groupId>com.hivemq</groupId>
    <artifactId>hivemq-spi</artifactId>
    <version>3.2.0</version>
</dependency>

3.0.x to 3.1.x Migration Guide

This is a major HiveMQ upgrade. HiveMQ 3.1 is not a drop in replacement for HiveMQ 3.0.x

Manual steps are required for upgrading from HiveMQ 3.0 to HiveMQ 3.1.

HiveMQ Upgrade

  • Create a backup of the whole HiveMQ installation folder

  • Install HiveMQ 3.1 as described in the HiveMQ Installation Guide

  • Migrate the configuration file contents of your old HiveMQ installation

  • To migrate the persistent data, copy all contents in the data folder from your backup to the new HiveMQ 3.1 data folder. On first start HiveMQ 3.1 will migrate the persistent data from your previous installation into its new persistent storage.

Configuration File Upgrade

HiveMQ 3.1 introduces some changes in the configuration file:

Cluster configuration

HiveMQ 3.1 introduces a much simpler way to configure a cluster directly in HiveMQ’s config.xml.

This configuration is not compatible with the previous way of configuring a HiveMQ cluster.

Please see the HiveMQ Clustering Documentation on how to configure your cluster for HiveMQ 3.1.

Persistent Data Migration

HiveMQ 3.1 introduces automatic migrations for persistent data between versions.

To allow HiveMQ to migrate the persistent data from previous versions you have to copy the contents of your data folder from the previous installation to the data folder of the new installation.

Linux example
cp -r /opt/hivemq-3.0.3/data/* /opt/hivemq-3.1.0/data/

On first start HiveMQ 3.1 will automatically migrate the persistent data from your previous installation into its new persistent storage.

Cluster Upgrade

In HiveMQ 3.1 the clustering underwent significant changes. HiveMQ clustering is not full-replication by default anymore. This allows for better distribution and a lot better scalability.

The cluster of HiveMQ 3.0 and HiveMQ 3.1 are not compatible. Rolling upgrades are not supported.

Please see the HiveMQ Clustering Documentation for more information on the new clustering options.

Plugin System Upgrade

The plugin system significantly improved in HiveMQ 3.1.

New Features

The HiveMQ 3 plugin systems has the following new features and improvements:

  • New Blocking and Async APIs for services. The previous services are deprecated but still work. They will be removed in HiveMQ 4.0. You should consider updating to the Async APIs for better performance.

  • The SharedSubscription service is deprecated, please use the new shared subscription logic introduced in HiveMQ 3.1 instead.

  • MetricService is now able to get Metrics from all cluster nodes.

Upgrading your plugins

In order to upgrade your plugin, execute the following steps:

  • update your pom.xml

Adapt your pom.xml file

Change your HiveMQ SPI version to 3.1.0 in your pom.xml file

<dependency>
    <groupId>com.hivemq</groupId>
    <artifactId>hivemq-spi</artifactId>
    <version>3.1.0</version>
</dependency>

2.3.x to 3.0.x Migration Guide

This is a major HiveMQ upgrade. HiveMQ 3.0 comes with many new features and improvements. HiveMQ 3.0 is not compatible with HiveMQ 2.x and is not a drop in replacement.

HiveMQ Upgrade

  • Create a backup of the whole HiveMQ 2 installation folder

  • Install a recent Java 7 version (or newer)

  • Install HiveMQ 3 as described in the HiveMQ Installation Guide

  • Migrate the configuration file contents of your old HiveMQ installation

  • Delete all contents in the data folder. This will delete all persistent data of HiveMQ 2.x.

Configuration File Upgrade

Manual steps are required for upgrading the configuration file of HiveMQ 2.3 to HiveMQ 3.0.

HiveMQ 3.0 comes with a new, single, XML based configuration file. HiveMQ 2.x uses two plain properties files, so the migration from HiveMQ 2.3 needs to be done manually.

Removed Configurations

The following table shows all removed configurations which are not available in HiveMQ 3 anymore. Configurations ending with a star (*) mean, that all configurations which start with that name were removed.

Table 1. Removed HiveMQ 2.3 configurations
Properties name Deprecation reason Alternative

plugin.performanceMonitoring

Plugin monitoring is done automatically by HiveMQ 3, so this configuration is not necessary anymore

None

graphite.*

Graphite integration is not in the HiveMQ core anymore. A more powerful Graphite plugin is available for free.

Use the free Graphite plugin from the plugin store

measureResourceConsumption.*

Resource consumption logfile reporting was too inaccurate for real monitoring and was too resource intensive.

The usage of a dedicated monitoring tool like collectl, collectd, or any other monitoring tool of your choice is recommended

mqtt.lwt.onEveryDisconnect

This configuration wasn’t very popular and caused a lot of confusion.

None

plugin.LowLevelCallbacksEnabled

In HiveMQ 3 low level callbacks are always enabled, without performance loss

None

ssl.hybridMode

HiveMQ 3 allows multiple listeners, so this setting isn’t needed anymore

Use different listeners simultaneously.

statistics.*

All HiveMQ statistics and metrics are enabled by default and are available instantly.

None, statistics are always enabled

sys.*

The $SYS Topics were removed from the core and are now available as plugin, delivered with HiveMQ.

Use the SYS-Topic plugin (also delivered with HiveMQ)

Upgrading Configurations

HiveMQ 3 uses a XML based file format, so manual migration of configurations is necessary. In case you’re using a default HiveMQ configuration, no configuration changes are needed, just use the HiveMQ 3 default config.xml.

HiveMQ 3 comes with various popular example configurations which are located in the conf/examples folder.

The following instructions show how to upgrade manually. It’s not necessary to follow all steps, you can pick the steps you need for your concrete configuration.

Listener Configuration

HiveMQ 3 uses a new listener concept, which allows to add as many listeners to different network interfaces and ports as needed. This also deprecates the HiveMQ 2 Hybrid Mode. The following listener types are available:

  • TCP Listener

  • TLS-TCP Listener

  • Websocket Listener

  • TLS Websocket Listener

The following guide shows how to use the new listener concept: Listener Documentation

MQTT Configuration

Most of the MQTT configurations are now organized under the mqtt XML element.

The following table shows the equivalent of the old properties to the new XML elements.

Table 2. MQTT Config Migration
Old properties name New XML element Migration notes

global.retryInterval

<retry-interval>

None

global.maxClientIdLength

<max-client-id-length>

None

messages.maxInflight

None

Max In-flight window is always at most 1 per topic subscription per client in order to give Ordered Topic guarantees.

messages.maxQueued

<max-queued-messages>

None

Throttling Configuration

All throttling configurations are now organized under the throttling XML element.

The following table shows the equivalent of the old properties to the new XML elements.

HiveMQ 3 does not support reloading of throttling configuration via the XML file at runtime. You can still use custom plugins for runtime throttling.

Table 3. Throttling Config Migration
Old properties name New XML element Migration notes

global.maxConnections

<max-connections>

None

global.maxMessageSize

<max-message-size>

None

global.outgoingLimit

<outgoing-limit>

The old value was in KB/s, the new value is in B/s.

global.ingoingLimit

<incoming-limit>

The old value was in KB/s, the new value is in B/s.

Plugin System Upgrade

The plugin system significantly improved in HiveMQ 3.0. The goal with HiveMQ 3 was to be as backwards compatible as possible while providing the foundation for even better plugin integration for the future. Most of the plugin services are now cluster-aware, which means information from the cluster can be gathered at runtime by plugins.

New Features

The HiveMQ 3 plugin systems has a lot of new features and improvements, the most noteworthy are the following:

  • A PluginExecutorService, managed by HiveMQ, was added, so plugins don’t need to bring their own Executors for asynchronous processing

  • A completely new RESTService is now included in HiveMQ. This allows to hook in your own Java Servlets or JAX-RS resources for creating HTTP APIs inside of HiveMQ

  • Most of the configuration can be changed at runtime, thanks to the new Configuration Services

  • Additional informal annotations (like @NotNull,@Nullable, @Threadsafe) are now available

  • Annotations for creating automated Metrics for your plugins are now available (@Metered, @Counted, @ExceptionMetered)

Removed Functionality

  • Removed MqttMessageEstimationUtil

  • Removed HiveMQStatistics (these got replaced by the more powerful HiveMQMetrics)

  • Removed OnStatisticsUpdate Callback (the HiveMQ statistics are now up to date all the time)

  • Removed Apache Commons Configuration. Plugins are now responsible for providing their information themselves.

  • PathUtils was deprecated, the SystemInformation interface can be injected instead for retrieving relevant information.

Upgrading your plugins

In order to upgrade your plugin, execute the following steps:

Adapt your pom.xml file

Change your HiveMQ SPI version to 3.0.0 in your pom.xml file

<dependency>
    <groupId>com.hivemq</groupId>
    <artifactId>hivemq-spi</artifactId>
    <version>3.0.0</version>
</dependency>

You also need to upgrade your HiveMQ Maven plugin to the newest version in order to develop against HiveMQ 3.0:

<profile>
    <id>RunWithHiveMQ</id>
    <build>
        <plugins>
            <plugin>
                <groupId>com.hivemq</groupId>
                <artifactId>hivemq-maven-plugin</artifactId>
                <version>3.0.1</version>
                <executions>
                    <execution>
                        <id>hivemq</id>
                        <phase>package</phase>
                        <goals>
                            <goal>hivemq</goal>
                        </goals>
                        <configuration>
                            <hiveMQDir>INSERT_HERE</hiveMQDir>
                            <debugMode>SERVER</debugMode>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

Change the HiveMQ namespace

The namespace of all SPI classes was changed from com.dcsquare.hivemq to com.hivemq.

An easy way to fix this is to use your favourite text editor or IDE and bulk replace com.dcsquare. with com..

Upgrading the META-INF/services file

Rename the com.dcsquare.hivemq.spi.HiveMQPluginModule file to com.hivemq.spi.HiveMQPluginModule.

Upgrading MQTT Messages

The MQTT message classes received a general clean-up to represent the MQTT specification better and to avoid misunderstandings. The following changes may affect your plugins:

  • All messages: duplicateDelivery, retain and QoS flags are now only supported for PUBLISH messages.

  • CONNECT: the getPassword() method now returns a byte[] instead of a String to conform the MQTT specification. For convenience there’s now a getPasswordAsUTF8String().

  • CONNECT: The getWillMessage() and setWillMessage methods now also return/take a byte[] instead of a String to conform the MQTT specification.

Upgrade Subscription Store API

The Subscription Store now has methods for receiving cluster wide state of subscriptions. Methods with the old semantics (get subscription data from local node) are still available, with a slightly different name.

The Subscription Stores main API change is, that methods now return a ListenableFuture instead of the result directly, so plugin developers can decide when to wait for the results or if a callback-based approach should be used.

The following table shows the new name of the 'old' methods:

Table 4. Subscription Store API Migration
Old method New Method

getSubscriptions()

getLocalSubscriptions()

getSubscribers(String topic)

getLocalSubscribers(String topic)

getTopics(String clientID)

getLocalTopics(String clientID)

Upgrade ClientService API

The ClientService APIs are now capable of collecting cluster-wide data. Methods with the old semantics (get data only from the current HiveMQ node) are still available with another name.

The ClientServices main API change is, that methods now return a ListenableFuture instead of the result directly, so plugin developers can decide when to wait for the results or if a callback-based approach should be used.

The following table shows the new name of the 'old' methods:

Table 5. ClientService API Migration
Old method New Method

getConnectedClients()

getLocalConnectedClients()

getDisconnectedClients()

getLocalDisconnectedClients()

isClientConnected(String clientId)

isClientConnectedLocal(String clientId)

getClientDataForClientId(String clientId)

getLocalClientDataForClientId(String clientId)

Upgrading Authorization and Authentication

While HiveMQ 2.x only allowed blacklists, HiveMQ 3 now allows a white and black list authorization approach.

Each OnAuthorizationCallback needs to override the getDefaultBehaviour() method.

MqttTopicPermission has been improved and can be used to allow or deny a specific actions on a topic.

In order to upgrade the MqttTopicPermissions to the new API with semantics from HiveMQ 2, use ALLOW as Permission Type.

The following additional authentication and authorization changes are included in HiveMQ 3:

  • RestrictionsAfterLoginCallback is now an asynchronous callback, not a synchronous one.

  • ClientCredentialsData now offers a getPasswordBytes() method which allows to get the raw byte array of a password instead of a UTF-8 representation.

  • Added an additional SslClientCertificate wrapper for client certificates with many utility methods for dealing with X509 client certificates. It’s also possible to directly obtain the java.security.cert.Certificate object via this wrapper.

  • ClientData now also allows to get the IP Address of a client via getInetAddress().

Executor Services

HiveMQ 3 offers a HiveMQ-managed PluginExecutorService for all asynchronous needs. All plugins share this managed executor service.

In the past some HiveMQ plugin developers used custom CachedThreadPools which drained all system resources and HiveMQ got unresponsive due to excessive thread spawning in the plugins. To avoid this problem the PluginExecutorService was introduced in HiveMQ 3.0.

Upgrading is easy: Remove all your executor services and inject PluginExecutorService.

The PluginExecutorService can be used to schedule your Runnables and returns ListenableFutures, so you can use callback-based, non-blocking programming paradigms.

REST Service

The old experimental REST Service was removed and a new, more sophisticated REST Service was introduced. The Plugin Developer Guide shows how to use the new REST Service

Configurations

HiveMQ 3 does not support the old Apache Commons Configuration approach anymore. In the past this allowed to break classloader isolation since properties were shared between plugins.

HiveMQ 3 requires plugin developers to bring their own configuration mechanism. The following configuration libraries are known to work very with HiveMQ:

If you want to use (reloadable) standard Java Properties files for configuration, you can take a look at this example plugin: Graphite Example Plugin