Introduction

HiveMQ offers a free, open-source extension SDK. The HiveMQ extension framework provides an open API that allows developers to create custom extensions that suit their specific infrastructures. Use our extension framework to extend HiveMQ with custom business logic or to integrate virtually any system into HiveMQ.

The HiveMQ extension framework can be used to add many different types of functionality:

  • Write messages to a database,

  • integrate other services,

  • collect statistics,

  • add fine-grained security,

  • and much more.

Once you understand the core concepts, extension development for HiveMQ is as easy as writing a Java main method. This guide covers all the basics that you need to get started with extension development for HiveMQ.

HiveMQ extensions can interact with the MQTT broker in various ways:

  • Use our extension API to interact with HiveMQ and connected MQTT clients

  • Register callbacks with HiveMQ that HiveMQ calls when a specific event occurs

  • Inspect and manipulate MQTT-related data such as sessions, retained messages, subscriptions, and more

  • Implement fine-grained authentication and authorization for MQTT clients

  • Add a custom cluster-discovery mechanism

  • Add custom logic to HiveMQ

HiveMQ 3 Plugins are not compatible with HiveMQ 4.

For a list of all the HiveMQ extenstions that are currenlty available, visit the HiveMQ Marketplace.


Get Started

The best way to begin extension development for HiveMQ is to follow our Quick Start Guide for HiveMQ Extension Developers.

If you prefer to learn from real-world extensions, visit the HiveMQ Github page to access a variety of current extensions.

For in-depth information about the HiveMQ Enterprise SDK, advanced extension developers can jump directly to the API JavaDoc or the detailed Concepts, Services, and Registries chapters of our documentation.


Quick Start Guide for HiveMQ Extension Developers

Extension Structure

An extension is a folder that contains at least a Java archive (.jar) and an XML descriptor file (hivemq-extension.xml).

Example content of an extension folder
|- hello-world-extension/  (1)
|   |- hello-world-extension-1.0.0.jar  (2)
|   |- hivemq-extension.xml (3)
1 The name of the extension folder. This name must match the ID in your XML descriptor file
2 The Java archive that contains the Java classes that implement the logic of the extension
3 The XML descriptor file that contains additional information and metadata

The Java archive contains all the compiled code of the extension. This code implements the logic that HiveMQ loads and executes. HiveMQ extension structures are preconfigured with everything that is needed to automatically create the XML-descriptor file from the source code of the extension.


Extension Information

The HiveMQ Extension SDK uses the hivemq-extension.xml to store meta information and the priority level of the extension.

The priority that you assign to an extension defines the order in which HiveMQ executes the extension. If multiple extensions are installed, the extension with the highest priority executes first.

We strongly recommend that you fill in all of the metadata that is requested in the hivemq-extension.xml file. HiveMQ logs the information when the extension loads and the information is useful for monitoring and debugging purposes.
Example hivemq-extension.xml content
<hivemq-extension>
    <id>hello-world-extension</id> (1)
    <name>Hello World Extension</name> (2)
    <version>1.0-SNAPSHOT</version> (3)
    <priority>1000</priority> (4)
    <author>dc-square-GmbH</author> (5)
</hivemq-extension>
1 The extension identifier. This identifier must be unique across all of the HiveMQ extensions that are installed on a HiveMQ installation.
2 The name of the extension. HiveMQ displays this name for the extension throughout the HiveMQ installation.
3 The version of the extension.
4 The priority of the extension. Extensions with the lowest priority execute last. The default setting is 0. This setting is optional.
5 The author of the extension. This setting is optional.


Prerequisites

All you need to start developing a custom HiveMQ extension is a basic understanding of the Java programming language and the following two components:

  1. OpenJDK 11 or newer

  2. Maven

To facilitate your development process, we strongly recommend that you use a Java-development IDE such as Eclipse or IntelliJ.


Create Your Java Project

There are many different ways to create your first extension for HiveMQ. In this guide, we use the HiveMQ Maven-extension archetype to create a base for the extension.

To begin, run the following command on the command line:

mvn archetype:generate -DarchetypeGroupId=com.hivemq -DarchetypeArtifactId=hivemq-extension-archetype -DarchetypeVersion=4.2.0

When prompted, define the following Maven identifiers for your new project: GroupId, ArtifactId, Version, and Package.

Example Output of the Maven Archetype Command
$ mvn archetype:generate -DarchetypeGroupId=com.hivemq -DarchetypeArtifactId=hivemq-extension-archetype -DarchetypeVersion=4.2.0

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.apache.maven:standalone-pom >-------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] >>> maven-archetype-plugin:3.0.1:generate (default-cli) > generate-sources @ standalone-pom >>>
[INFO]
[INFO] <<< maven-archetype-plugin:3.0.1:generate (default-cli) < generate-sources @ standalone-pom <<<
[INFO]
[INFO]
[INFO] --- maven-archetype-plugin:3.0.1:generate (default-cli) @ standalone-pom ---
[INFO] Generating project in Interactive mode
[INFO] Archetype repository not defined. Using the one from [com.hivemq:hivemq-extension-archetype:4.2.0] found in catalog remote
Define value for property 'groupId': com.hivemq.extensions
Define value for property 'artifactId': hello-world-extension
Define value for property 'version' 1.0-SNAPSHOT: : 1.0.0
Define value for property 'package' com.hivemq.extensions: : com.hivemq.extensions.helloworld
Confirm properties configuration:
groupId: com.hivemq.extensions
artifactId: hello-world-extension
version: 1.0.0
package: com.hivemq.extensions.helloworld
 Y: : Y
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype: hivemq-extension-archetype:4.2.0
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: com.hivemq.extensions
[INFO] Parameter: artifactId, Value: hello-world-extension
[INFO] Parameter: version, Value: 1.0.0
[INFO] Parameter: package, Value: com.hivemq.extensions.helloworld
[INFO] Parameter: packageInPathFormat, Value: com/hivemq/extensions/helloworld
[INFO] Parameter: package, Value: com.hivemq.extensions.helloworld
[INFO] Parameter: groupId, Value: com.hivemq.extensions
[INFO] Parameter: artifactId, Value: hello-world-extension
[INFO] Parameter: version, Value: 1.0.0
[INFO] Project created from Archetype in dir: /Users/cschaebe/Projects/play/docs/hello-world-extension
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 59.512 s
[INFO] Finished at: 2019-02-01T14:57:36+01:00
[INFO] ------------------------------------------------------------------------

The Maven-archetype command creates a new folder that contains a fully functional HiveMQ extension. This folder has the same name as the ArtifactId that you defined for the Maven-extension archetype of your project. The Java project for a HiveMQ extension is a basic Maven project that includes the hivemq-extension-sdk as a dependency. The SDK is available on Maven Central and is automatically downloaded and added to your project by Maven.

As an alternative to the Maven archetype, you can clone the hivemq-hello-world-extension on github.

Deploy Your Extension

At this point in the process, the extension is a fully functional HiveMQ extension project that can be packaged and deployed to HiveMQ.

To package your extension, simply execute the following Maven goal in the extension folder:

mvn package

This command creates an <artifactId>-<version>-distribution.zip archive in the target folder. The zip archive contains your ready-to-deploy HiveMQ extension.

To deploy your HiveMQ Extension to your HiveMQ instance, unzip the archive and move the extracted folder to the extensions folder of HiveMQ.

Install Extension

HiveMQ automatically recognizes and starts the extension.

HiveMQ Console
2018-12-11 12:18:00,248 INFO  - Started Hello World Extension:1.0.0
2018-12-11 12:18:00,248 INFO  - Extension "Hello World Extension" version 1.0.0 started successfully.
HiveMQ 4 and the HiveMQ Extension SDK support Extension Hot Reload. This functionality allows you to add and remove extensions to HiveMQ during runtime.

Develop Your Extension

Now that you can package and deploy your extension to HiveMQ, it’s time to delve deeper into the development of custom HiveMQ extensions.
Let’s start by exploring the essential parts of the Java project structure.


Java Project Structure

HiveMQ Extension Structure

Project Structure

A minimal HiveMQ extension consists of a Java class, a text file, and an XML file:

  1. A Java class that implements com.hivemq.extension.sdk.api.ExtensionMain is the starting point of every HiveMQ extension.

  2. A text file in the META-INF/services folder named com.hivemq.extension.sdk.api.ExtensionMain tells HiveMQ where to find the main class of the extension.

  3. An XML file in the resources folder named hivemq-extension.xml contains the metadata of the extension.

Using the Maven Archetype
The Maven Archetype automatically generates all of these files with the desired content.


ExtensionMain

The starting point of every extension development is a class that implements the ExtensionMain class of the HiveMQ Extension SDK. The interface ExtensionMain consists of the methods extensionStart and extensionStop.

HiveMQ calls the extensionStart method when you start the extension. This method is also the starting point for extension development, since this is also the point where an extension can register callbacks and implementations with HiveMQ.

Example

The following example shows the minimal content of a main class for a HiveMQ extension:

public class HelloWorldMain implements ExtensionMain {

    @Override
    public void extensionStart(ExtensionStartInput input, ExtensionStartOutput output) {
        //Code to run when extension is started
    }

    @Override
    public void extensionStop(ExtensionStopInput input, ExtensionStopOutput output) {
        //Code to run when extension is stopped
    }
}
When you rename the main class (in the example, HelloWorldMain), remember to change the reference to the class in the src/main/resources/META-INF/services/com.hivemq.extension.sdk.api.ExtensionMain file.

Popular Use Cases

A HiveMQ extension has virtually infinite possibilities. This guide focuses on some of the most popular use cases. Throughout the guide, we provide links to help you dive deeper into the details of the extension SDK and associated APIs.

Here are some typical uses for the HiveMQ Extension SDK:


Authentication

Authentication is the mechanism that verifies the identity of your communication partner.
Air travel provides a classic example of authentication. Before you are permitted to board a plane, airport security checks your passport. Presentation of a passport that identifies you as the person whose name is on the ticket, allows you to proceed.
In the context of MQTT, authentication is used to check if an MQTT client is allowed to connect to a broker and if it is allowed to use a particular client identifier.

To apply authentication in a HiveMQ extension, you implement a SimpleAuthenticator and register an AuthenticatorProvider with HiveMQ. This implementation is done in the extensionStart method of the main class. The custom provider for authenticator is registered with the SecurityRegistry.

Registering Authenticator
public void extensionStart(ExtensionStartInput extensionStartInput, ExtensionStartOutput extensionStartOutput) {
    //register the provider with the SecurityRegistry
    Services.securityRegistry().setAuthenticatorProvider(new MyAuthenticatorProvider());
}

In this example, the Authenticator Provider returns a new Authenticator for every new MQTT connection. It is also possible to share an instance of an authenticator between multiple MQTT connections, see this example.

Example Authenticator Provider
public class MyAuthenticatorProvider implements AuthenticatorProvider {

    @Override
    public Authenticator getAuthenticator(AuthenticatorProviderInput authenticatorProviderInput) {
        //return an instance of an Authenticator
        return new MyAuthenticator();
    }

}

A SimpleAuthenticator that implements username/password based authentication for MQTT clients uses the information from the input object to decide whether or not a client is allowed to connect.

In this example, only clients with the username "admin" and password "hivemq" are allowed to connect. Clients that lack the username or password are not permitted.

Example Username/Password Authenticator
public class MyAuthenticator implements SimpleAuthenticator {

    @Override
    public void onConnect(SimpleAuthInput simpleAuthInput, SimpleAuthOutput simpleAuthOutput) {
        ...
        //check if the user is "admin" and the password is "hivemq"
        if (username.equals("admin") && password.equals("hivemq")) {
            simpleAuthOutput.authenticateSuccessfully();
        } else {
            simpleAuthOutput.failAuthentication();
        }
    }
}
Full Source of Example Username/Password Authenticator
public class MyAuthenticator implements SimpleAuthenticator {

    @Override
    public void onConnect(SimpleAuthInput simpleAuthInput, SimpleAuthOutput simpleAuthOutput) {

        //get the contents of the MQTT connect packet from the input object
        ConnectPacket connect = simpleAuthInput.getConnectPacket();

        //check if the client set username and password
        if (!connect.getUserName().isPresent() || !connect.getPassword().isPresent()) {
            simpleAuthOutput.failAuthentication();
        }

        //get username and password from the connect packet
        String username = connect.getUserName().get();
        String password = Charset.forName("UTF-8").decode(connect.getPassword().get()).toString();

        //check if the user is "admin" and the password is "hivemq"
        if (username.equals("admin") && password.equals("hivemq")) {
            simpleAuthOutput.authenticateSuccessfully();
        } else {
            simpleAuthOutput.failAuthentication();
        }
    }
}

In the authenticator, the extension developer has access to all client information that can be used for authentication. This information includes all of the information from the CONNECT packet and the TLS certificate of the client. Access to all available authentication information makes it possible to implement even the most complex authentication mechanisms.

Authorization

Authorization is the mechanism that controls access based on certain identities.
Let’s continue the air travel example that we began in the authentication section. After your ticket and passport authenticate your identity, a boarding pass is used to authorizes your access to a specific plane, row, and seat.
In the context of an MQTT client, 'access' refers to the ability to publish and subscribe to certain topics.

HiveMQ extensions support two authorization mechanisms for the MQTT publish and subscribe actions:

  • Default Permissions: This mechanism provides access control on a per topic basis.

  • Authorizers: This mechanism implements authorization on a per-packet basis.

Default Permissions are the simplest and most scalable of the mechanisms. The permissions for a client are represented by a list of topics and attributes to which a client can or cannot publish or subscribe. Default permissions are independent for each client and can be set in an authenticator or in a ClientInitializer.

In this example, each client is allowed to publish and subscribe only to topics that start with the same client identifier as the client. Here, we add a few lines to the existing authenticator that add the default permissions of the client.

Default Permissions
...

//create a topic permission for all topics starting with the client identifier
final TopicPermission permission = Builders.topicPermission() (1)
        .topicFilter(simpleAuthInput.getConnectPacket().getClientId() + "/#") (2)
        .type(TopicPermission.PermissionType.ALLOW) (3)
        .build();

//add this permission to the client's default permissions
simpleAuthOutput.getDefaultPermissions().add(permission);

...
1 The extension SDK provides builders for classes that need to be constructed by an extension developer. The class Builders has static methods to access the specific builders.
2 Topic filter for this permission, the MQTT wildcards # and + can be used to create topic filters that match multiple topics.
3 A white-list approach that only allow access to specific topics is used in this example. A black-list approach is also possible.
Full Authorizer with default permissions
public class MyAuthenticator implements SimpleAuthenticator {

    @Override
    public void onConnect(SimpleAuthInput simpleAuthInput, SimpleAuthOutput simpleAuthOutput) {

        //get the contents of the MQTT connect packet from the input object
        ConnectPacket connect = simpleAuthInput.getConnectPacket();

        //check if the client set username and password
        if (!connect.getUserName().isPresent() || !connect.getPassword().isPresent()) {
            simpleAuthOutput.failAuthentication();
        }

        //get username and password from the connect packet
        String username = connect.getUserName().get();
        String password = Charset.forName("UTF-8").decode(connect.getPassword().get()).toString();

        //check if the user is "admin" and the password is "hivemq"
        if (username.equals("admin") && password.equals("hivemq")) {

            //create a topic permission for all topics starting with the client identifier
            TopicPermission permission = Builders.topicPermission()
                    .topicFilter(simpleAuthInput.getConnectPacket().getClientId() + "/#")
                    .build();

            //add this permission to the client's default permissions
            simpleAuthOutput.getDefaultPermissions().add(permission);

            simpleAuthOutput.authenticateSuccessfully();
        } else {
            simpleAuthOutput.failAuthentication();
        }

    }

}
The permissions for an MQTT client can include more detail than only the topic. For example, the QoS level or the retained-message flag. For a list of all available permission options, see the Topic Permission chapter.

The other mechanism for authorization are Authorizers. Authorizers implement authorization on a per-packet basis.

To learn more about Authorizers, see the Publish Authorizer and Subscription Authorizer chapters.

Lifecycle Events

To get notified when certain events in the lifecycle of an MQTT client occur, an extension can listen to lifecycle events. These events include a client establishing a connection, a client disconnecting, successful authentication of a client, and more.

To create a lifecycle-event listener, register a ClientLifecycleEventListenerProvider with the EventRegistry.

Register provider with EventRegistry
 Services.eventRegistry().setClientLifecycleEventListener(new MyClientLifecycleEventListenerProvider());

The provider returns a new instance of a ClientLifecycleEventListener for each client. If desired, you can share the same instance of a listener between multiple clients. For more information, see this example.

Example ClientLifecycleEventListenerProvider
public class MyClientLifecycleEventListenerProvider implements ClientLifecycleEventListenerProvider {

    @Override
    public ClientLifecycleEventListener getClientLifecycleEventListener(ClientLifecycleEventListenerProviderInput input) {
        return new MyClientLifecyleEventListener();
    }
}

In this example, a log statement is issued when a client connects, when authentication is successful, and when a client disconnects.

Example ClientLifecycleEventListener
public class MyClientLifecyleEventListener implements ClientLifecycleEventListener {

    private static final Logger log = LoggerFactory.getLogger(MyClientLifecyleEventListener.class);

    @Override
    public void onMqttConnectionStart(ConnectionStartInput connectionStartInput) {
        log.info("Client {} connects.", connectionStartInput.getConnectPacket().getClientId());
    }

    @Override
    public void onAuthenticationSuccessful(AuthenticationSuccessfulInput authenticationSuccessfulInput) {
        log.info("Client {} authenticated successfully.", authenticationSuccessfulInput.getClientInformation().getClientId());
    }

    @Override
    public void onDisconnect(DisconnectEventInput disconnectEventInput) {
        log.info("Client {} disconnected.", disconnectEventInput.getClientInformation().getClientId());
    }
}
You can listen to more detailed events by overriding other methods from the interface ClientLifecycleEventListener.


Intercept & Manipulate MQTT Messages

To manipulate the content of an MQTT message before the broker processes the message, a HiveMQ extension can intercept the message. The developer registers an Interceptor callback that HiveMQ calls each time an MQTT message is received.

In this example, a PublishInterceptor that manipulates the topic of incoming publishes is registered. An Interceptor is always registered in a ClientInitializer. The initializer is a callback that HiveMQ calls each time an MQTT client session is initialized.

Register ClientInitializer with the InitializerRegistry
Services.initializerRegistry().setClientInitializer(new MyClientInitializer());

An Interceptor is then added to the clientContext of the Initializer.

Register ClientInitializer with the InitializerRegistry
public class MyClientInitializer implements ClientInitializer {

    @Override
    public void initialize(InitializerInput initializerInput, ClientContext clientContext) {

        clientContext.addPublishInboundInterceptor(new PublishInboundInterceptor() { (1)
            @Override
            public void onInboundPublish(PublishInboundInput publishInboundInput, PublishInboundOutput publishInboundOutput) {
                //set the topic to "new/topic"
                publishInboundOutput.getPublishPacket().setTopic("new/topic"); (2)
            }
        });
    }
}
1 A new Interceptor is created and added to the context of the client.
2 The changed topic. Most of the attributes of an MQTT PUBLISH packet can be manipulated.
The Publish Interceptor can also be used to prevent onward delivery of incoming messages. For more information, see this example.

Publish MQTT Messages

You can use an extension to manipulate the messages clients send or to create and publish new messages.

The Publish Service allows you to publish messages to all subscribers of a topic or to a specific client. For more information, see this example.

Publish a MQTT message
Publish message = Builders.publish()
    .topic("the/topic")
    .qos(Qos.EXACTLY_ONCE)
    .payload(Charset.forName("UTF-8").encode("payload"))
    .build();

Services.publishService().publish(message);

The PUBLISH messages that the Publish Service sends are processed in the same way as a PUBLISH message that a client sends.


Modify Subscriptions

The Subscription Store allows extensions to manipulate the subscriptions of a client. Methods can add, remove, or get subscriptions for a client.

In this example, a subscription for the topic new/topic with QoS 1 is added for the client with the identifier client-ID.

Add a subscription for a client
TopicSubscription subscription = Builders.topicSubscription()
    .topicFilter("new/topic")
    .qos(Qos.AT_LEAST_ONCE)
    .build();

Services.subscriptionStore().addSubscription("client-ID", subscription);
When a subscription for a client is added via the extension system or the HiveMQ Control Center, the retained message for the topics that are included in the subscription are not published to the client.


Modify Retained Messages

In the extension SDK, the RetainedMessageStore can set, remove, and fetch retained messages.

For this example, a retained message is added (or replaced) for the topic retain/topic with QoS 0 and the payload content.

Add a subscription for a client
RetainedPublish publish = Builders.retainedPublish()
    .topic("retain/topic")
    .payload(Charset.forName("UTF-8").encode("content"))
    .qos(Qos.AT_MOST_ONCE)
    .build();

Services.retainedMessageStore().addOrReplace(publish);
Unlike the retained messages that clients send, retained messages that are added with the Retained Message Store are not published to active subscribers.


Cluster Discovery

The HiveMQ cluster discovery feature can utilize the custom discovery mechanisms that an extension provides. This functionality enables deep integration of a HiveMQ cluster into an existing infrastructure.

Register cluster discovery callback
Services.clusterService().addDiscoveryCallback(new MyClusterDiscovery());
Example cluster discovery
public class MyClusterDiscovery implements ClusterDiscoveryCallback {

    //this is a placeholder for an implementation of your custom external service
    MyExternalService externalService = new MyExternalService(); (1)

    @Override
    public void init(ClusterDiscoveryInput clusterDiscoveryInput, ClusterDiscoveryOutput clusterDiscoveryOutput) {
        externalService.registerNode(clusterDiscoveryInput.getOwnAddress()); (2)
        clusterDiscoveryOutput.provideCurrentNodes(externalService.getAllClusterNodes());
    }

    @Override
    public void reload(ClusterDiscoveryInput clusterDiscoveryInput, ClusterDiscoveryOutput clusterDiscoveryOutput) {
        clusterDiscoveryOutput.provideCurrentNodes(externalService.getAllClusterNodes()); (3)
    }

    @Override
    public void destroy(ClusterDiscoveryInput clusterDiscoveryInput) {
        externalService.removeNode(clusterDiscoveryInput.getOwnAddress()); (4)
    }
}
1 This class is a placeholder for a custom external service.
2 When this cluster node starts, the node registers with the external service.
3 The reload method is called regularly and a list of addresses for all cluster nodes is provided to the output object. You can configure the reload interval.
4 When this cluster node stops, the node is unregistered with the external service.
If you want to use cluster discovery from an extension, the cluster configuration of HiveMQ must be set to extension. For more information, see configuration example.


Metrics

HiveMQ extensions can provide their own metrics and access HiveMQ metrics.

Metrics in HiveMQ extensions are handled by the widely-used and well-known Dropwizard Metrics library.

Example read HiveMQ metric
Counter currentConnectionsCounter = Services.metricRegistry().counter("com.hivemq.networking.connections.current");
log.info("currently connected {}", currentConnectionsCounter.getCount());
Example add custom metric
final Counter customCounter = Services.metricRegistry().counter("demo.helloworld.started");
customCounter.inc();
Many types of metrics are available. Metric types include: Gauges, Counters, Meters, Histogramms, and Timers.

Next steps

  • Get acquainted with the APIs and Services that the extension SDK provides.

  • Look at existing extension code on github.

  • Implement your custom logic in a HiveMQ extension.