HiveMQ Enterprise Bridge Extension

The HiveMQ Enterprise Bridge Extension enables HiveMQ to bridge to one or more MQTT brokers for scalable, reliable, and bi-directional exchange of MQTT messages.

HiveMQ Enterprise Bridge Extension Architecture

Features

  • Customizable bidirectional topic mapping that allows you to configure a set of topic filters that forward matching messages to specific destination topics at another MQTT broker.

  • Fine-grained configuration of the source topic filter and destination topic with the use of all MQTT-specific variables as well as configuration-specific templating.

  • Extensive logging capabilities and useful metrics for monitoring.

Requirements

  • An MQTT bridge broker that runs HiveMQ Professional or Enterprise Edition installation, version 4.4.0 or higher.

  • A remote MQTT message broker that is compliant with the MQTT 5 specification. The remote broker can be a HiveMQ broker or a different MQTT-5 compliant broker.

Installation

  1. Place your HiveMQ Enterprise Bridge Extension license file (.elic) in the license folder of your HiveMQ installation.

  2. Move the entire hivemq-bridge-extension folder into the extensions folder of your HiveMQ installation.

  3. In the bridge-configuration.xml, configure the Enterprise Bridge Extension as desired. An example from the configuration-samples folder can be used to get started.
    For HiveMQ to recognize and start the extension, the bridge-configuration.xml must be located in the hivemq-bridge-extension folder.

  4. To use the bridge-configuration.xml to adjust the logging scope for your individual use case, copy the logback-example-adapter in the logback-example.xml file to the logback.xml of your HiveMQ installation.

└─ <HiveMQ folder>
    ├─ bin
    ├─ conf
    │   ├─ logback.xml
    ├─ data
    ├─ extensions
    │   ├─ hivemq-bridge-extension
    │   ├──── bridge-configuration.xml
    │   ├──── bridge-extension.jar
    │   ├──── hivemq-extension.xml
    │   └──── ...
    │   └─ ...
    ├─ license
    ├─ log
    └─ ...
If you want to bridge from an entire HiveMQ cluster, you must install identically configured Enterprise Bridge Extensions on every broker node in the HiveMQ cluster.

To verify the installation and configuration of the bridge extension, check your HiveMQ logs.

You can change the configuration of your HiveMQ Enterprise Bridge Extension during runtime:

  • Add a DISABLED file to the extension folder.

  • Change the configuration of the extension in the bridge-configuration.xml file.

  • Remove the DISABLED file from the extension folder.

Configuration

The minimum bridge extension configuration must contain the following items:

  • A bridge name that is unique within your local node structure.

  • Connection information for the remote broker.

  • A topic filter to forward incoming publishes from MQTT clients on the bridge broker to the remote broker.

To use the bridge extenaion, you must set up and store a configuration file named bridge-configuration.xml in the extension folder of your HiveMQ installation.
Example minimum bridge extension configuration
<hivemq-bridge-extension>
    <bridges>
        <bridge>
            <name>A Unique Bridge Name</name>
            <remote-broker>
                <connection>
                    <static>
                        <host>the.hostname.of.remote.broker</host>
                    </static>
                </connection>
            </remote-broker>
            <topics>
                <topic>
                    <filter>a/topic/filter</filter>
                </topic>
            </topics>
        </bridge>
    </bridges>
</hivemq-bridge-extension>

A basic configuration example can be found in the configuration-samples folder of the Enterprise Bridge Extension.

Bridge Extension Configuration Options

The HiveMQ Enterprise Bridge Extension provides many flexible configuration options. You can adjust the settings of each bridge to meet your needs. At least one bridge must be configured in the bridge extension.

Table 1. Bridge Extension Configuration Options
Configuration Required Description

bridge

The configuration settings for one bridge:

  • enabled: Defines whether the selected bridge is enabled.
    The default setting is true.

  • name: The unique identifier of the bridge. The name must be unique per cluster node. The name can contain the following characters: [a-z] [A-Z] [0-9] - _.
    The environment variable $ENV{HIVEMQ_BRIDGE_NAME} can be used to name the bridge.

remote-broker

Configuration of the remote broker to which bridge clients connect.

  • authentication: Optional setting that defines the type of authentication used for the MQTT connection.
    By default, no values are set:

    • username: The username required to establish the MQTT connection

    • password: The password required to establish the MQTT connection

    • token: The token that must be presetned to establish the MQTT connection

  • connection: Connection information for the MQTT listener of the remote broker:

    • host: The hostname of the MQTT listener.
      The environment variable $ENV{REMOTE_BROKER_HOST} can be used to set the hostname.

    • port: The port of the MQTT listener.
      The environment variable $ENV{REMOTE_BROKER_PORT} can be used to set the listener port. By default, the port is set on 1883.

    • websocket: Optional WebSocket configuration for the MQTT connection:

      • enabled: Defines whether the WebSocket configuration is enabled. The default value is false.

      • server-path:

  • mqtt: Optional internal settings for establishment of MQTT connections for MQTT 5 clients.

    • clean-start: Defines whether the client wants to start a new session upon reconnection (clean-start = true) or resume a previous session if present (clean-start = false). The default setting is false.

To prevent loss of messages when the remote broker is not available, best practice is to resume existing sessions with a clean-start setting of false.
  • session-expiry: The time in seconds until the bridge client sessions expire, when the connections are lost or closed. The default setting is 3600 seconds (one hour).

  • keep-alive: The time in seconds that the broker permits between when a client finishes sending one MQTT packet and starts to send the next. The broker must disconnect a client that does not send a a message or a PINGREQ packet in one and a half times the stated keep alive interval. The default setting is 60 seconds.

    • tls:Optional TLS configuration for the bridge connection:

  • enabled: Defines whether the TLS cryptographic protocol is used to provide an encrypted connection the bridge. The default setting is true.

  • keystore: When TLS is enabled, a keystore must be configured.

    • path: Location of the P12 or JKS certificates that the keystore uses to store entries.

    • password: The password to the associated keystore.

    • private-key-password: The password that protects the private key of the keystore.

  • truststore: Configuration of an optional truststore:

    • path: Location of the P12 or JKS certificates that the truststore uses to store entries.

    • password: The password to the associated truststore.

  • cipher-suites: The set of algorithms the bridge clients use to help secure the TLS network connection:

    • cipher-suite: The unique name of the cipher suite. The name indicates the algorithmic contents of the suite. Each segment in a cipher suite name stands for a different algorithm or protocol:

      • TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

      • TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256

      • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

      • TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

      • TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA

      • TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA

      • TLS_RSA_WITH_AES_128_GCM_SHA256

      • TLS_RSA_WITH_AES_128_CBC_SHA

      • TLS_RSA_WITH_AES_256_CBC_SHA

  • protocols: Optional entry that defines the TLS protocol version for the bridge clients. The following values are possible:

    • TLSv1.1

    • TLSv1.2

    • TLSv1.3

topics

Configuration of one or more topics that map MQTT messages for the bridge clients.
At least one topic must be configured.

  • topic: The configuration of one MQTT topic.

    • filter: The UTF-8 string that the broker uses to filter messages for the bridge clients. The topic filter must conform to the standard syntax of MQTT topic filtering paths.

    • mode: Optional setting that defines method of message transfer. The following options are possible:

      • PUB: MQTT PUBLISH messages are transferred to the remote broker.

      • SUB: MQTT PUBLISH messages are transferred from the remote broker.
        The default mode is PUB.

    • exclude: Optional setting that excludes specific topics from bridging. The default setting does not exclude any topics.

    • destination: Optional setting that routes the message for a specific topic to a different destination topic. The default setting uses the original topic.

Example HiveMQ Enterprise Bridge Extension configuration options
<hivemq-bridge-extension>
    <!-- A list of bridges, at least one bridge must be configured  -->
    <bridges>

        <bridge>

            <enabled>true</enabled>
            <name>My-Bridge</name>

            <remote-broker>

                <authentication>
                    <mqtt-simple-authentication>
                        <username>a-users-name</username>
                        <password>a-users-password</password>
                        <token>a-token-to-auth-with</token>
                    </mqtt-simple-authentication>
                </authentication>

                <connection>
                    <static>
                        <host>localhost</host>
                        <port>8000</port>
                    </static>
                    <websocket>
                        <enabled>true</enabled>
                        <server-path>mqtt</server-path>
                    </websocket>
                </connection>

                <mqtt>
                    <clean-start>false</clean-start>
                    <session-expiry>3600</session-expiry>
                    <keep-alive>60</keep-alive>
                </mqtt>

                <tls>

                    <enabled>true</enabled>
                    <keystore>
                        <path>path/to/keystore.jks</path>
                        <password>changeme</password>
                        <private-key-password>changemetoo</private-key-password>
                    </keystore>

                    <truststore>
                        <path>path/to/truststore.jks</path>
                        <password>changeit</password>
                    </truststore>

                    <cipher-suites>
                        <cipher-suite>TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384</cipher-suite>
                        <cipher-suite>TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256</cipher-suite>
                        <cipher-suite>TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256</cipher-suite>
                        <cipher-suite>TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384</cipher-suite>
                        <cipher-suite>TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA</cipher-suite>
                        <cipher-suite>TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA</cipher-suite>
                        <cipher-suite>TLS_RSA_WITH_AES_128_GCM_SHA256</cipher-suite>
                        <cipher-suite>TLS_RSA_WITH_AES_128_CBC_SHA</cipher-suite>
                        <cipher-suite>TLS_RSA_WITH_AES_256_CBC_SHA</cipher-suite>
                    </cipher-suites>

                    <protocols>
                        <protocol>TLSv1.2</protocol>
                    </protocols>
                </tls>
            </remote-broker>

            <topics>
                <topic>
                    <filter>+/+/machine/station</filter>
                    <mode>PUB</mode>
                    <exclude>test/+/machine/station</exclude>
                    <destination>mytest/{#}</destination>
                </topic>
            </topics>
        </bridge>
    </bridges>
</hivemq-bridge-extension>

Topic Mapping

Topic mapping filters the messaging flow from the source topics to the destination topics. Each topic can consist of one or more topic levels. Topic levels are separated by a forward slash (topic level separator).

Topic mapping functions are bidirectional:

  • The topic filter determines which messages the bridge extension processes.

  • The exclude tag of the topic filter defines a subset of topics from the topic filter that are not forwarded to the destination topic.

  • The destination tag modifies the selected topic to map to a different topic at the remote broker. This modification is executed before the message is published.

<topic>
    <filter>{incomingFilter}</filter>
    <exclude>{excludeFilter}</exclude>
    <destination>{filterDestination}</destination>
</topic>
Pipe

Modifications of destination tag based on topic filter:

  • Variables can be used inside the destination and exclude tags

  • Each topic level can be addressed by number, starting with 1

  • Static text can be added

  • A prefix and postfix can be used

Topic Filter Configuration Examples

1. Forward to a static remote topic.

Example forward to a static remote topic
<topic>
   <filter>+/status</filter>
   <destination>remote/Data</destination>
</topic>
SimpleForwarding

2. Exclude specific topics.

Example exclusion of a topic
<topic>
   <filter>+/status</filter>
   <exclude>client2/status</exclude>
   <destination>remote/Data</destination>
</topic>
  • The bridge extension consumes all topics that match the filter and forwards the messages to the destination topic.

ExcludeSimpleForwarding

3. Add prefix and postfix to the incoming topics as destination.

<topic>
   <filter>+/status</filter>
   <destination>prefix/{#}/data</destination>
</topic>
  • The bridge extension consumes all topics that match the filter and uses the full MQTT topic from the client as the destination topic at the remote broker with a prefix and a postfix.

UseTopicPlaceholder

4. Use parts of the topic filter in the destination topic.

<topic>
   <filter>+/+/+/+</filter>
   <destination>prefix/{1-3}/allData</destination>
</topic>
  • This example uses a four-level incoming filter. The bridge extension forwards all publish topics that match the pattern.

  • For the destination topic at the remote broker, only the first three parts of the local publish topic are used and a prefix and a postfix are added.

UseTopicfilterParts

5. Use prefix, postfix, and parts of the topic filter as the destination.

<topic>
   <filter>a/+/+/+</filter>
   <exclude>{1}/client2/{#}</exclude>
   <destination>prefix/{2-3}/allData</destination>
</topic>
  • This example uses a four-level incoming filter. The bridge extension forwards all publish topics that match the pattern.

  • For the destination topic at the remote broker, only the second and third parts of the local publish topic are used and a prefix (prefix) and postfix (allData) are added.

  • Messages that have the string client2 at the second level are not forwarded.

UseTopicfilterParts

6. Use prefix, postfix, and parts of the topic filter as the destination topic.

<topic>
   <filter>+/+/+/+</filter>
   <destination>prefix/{2}/NEU/{4}/allData</destination>
</topic>
  • This example uses a four-level incoming filter. The bridge extension forwards all publish topics that match the pattern.

  • For the destination topic at the remote broker, only the second and fourth level from the incoming topic are used. The third level is replaced with a constant and a prefix is added.

UseTopicfilterParts

7. Use variables and topic filter parts as the destination topic.

<topic>
    <filter>+/+/+/+</filter>
    <destination>prefix/{2}/${bridge.name}/{4}/allData</destination>
</topic>
  • This example uses a four-level incoming filter. The bridge extension forwards all publish topics that match the pattern.

  • For the destination topic at the remote broker, only the second and the fourth level of the incoming topic are used. The third level is replaced with a variable value that contains the bridge name and a prefix is added.

  • A set of 10 variables can be configured. For example, a variable can be configured to use the mqtt clientId, bridgeName, or userName.

UseTopicfilterParts

Further examples of topic level addressing:

  • {2}: Topic filter level 2

  • {2 - #}: Topic filter level 2 and all following levels

  • {#}: Whole topic filter

  • prefix/{#}/postfix: Whole topic filter inside prefix and postfix

  • {1}/A/{2}/B: Level 1 + "/A" + level 2 + "/B"

  • {#}/{1-3): Whole topic filter + level 1, 2, and 3

Logging Configuration

The HiveMQ Enterprise Bridge Extension can log key activity between the bridge and the remote broker. It is possible to log the following events:

  • Bridge client connects to a remote broker

  • Bridge client disconnects from a remote broker

  • Bridge client sends a PUBLISH message to a remote broker

  • Bridge client sends a SUBSCRIBE message to a remote broker

  • Bridge client receives a message from remote broker

The <message-log> tag defines the logging behavior for all configured bridges globally.

Table 2. Message Logging Configuration
Configuration Default Value Description

enabled

false

Defines whether the logging configuration is enabled. Possible settings are false and true.

log-level

-

Defines the type of logging information the extension provides. The following options are possible:

  • TRACE

  • DEBUG

  • INFO

  • WARN

  • ERROR

Example message logging configuration
<hivemq-bridge-extension>
    <message-log>
        <enabled>true | false</enabled>
        <log-level>DEBUG | INFO | TRACE | ERROR | WARN</log-level>
        <mqtt-packets>CONNECT, DISCONNECT, INCOMING-PUBLISH, SUBSCRIBE, OUTGOING-PUBLISH</mqtt-packets>
    </message-log>
    <bridges>
        ...
    </bridges>
</hivemq-bridge-extension>

Bridge Metrics

The Bridge Extension exposes a dashboard of several useful metrics that you can use to monitor behavior:

com.hivemq.bridge-extension.{bridgeName}.total.connect.count
com.hivemq.bridge-extension.{bridgeName}.total.connect-failed.count
com.hivemq.bridge-extension.{bridgeName}.total.disconnect.count
com.hivemq.bridge-extension.{bridgeName}.total.publish.count
com.hivemq.bridge-extension.{bridgeName}.total.publish-failed.count
com.hivemq.bridge-extension.{bridgeName}.total.subscribe.count
com.hivemq.bridge-extension.{bridgeName}.total.subscribe-failed.count

com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.connect.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.connect-failed.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.disconnect.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.publish.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.publish-failed.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.subscribe.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.subscribe-failed.count

The {bridgeName} is replaced with the configured bridge name. The {clientId} is replaced with the MQTT client ID that is used internally for bridging.

Error Handling

  • During the start of the extension, the bridge configuration is validated.

  • If parts of the bridge are incorrectly configured, the extension does not start and a DISABLED flag is set in the extension folder.

  • Setting up the connection to the remote broker is an important part of the configuration.

    • During start up, the bridge extension establishes connection to the specified remote broker. If connection is not possible for any reason, the bridge extension attempts to reconnect a predefined number times. If a connection is not established within the predefined number of retries, the bridge extension is disabled.

    • If a connection problems occur during runtime, the bridge attempts to reconnect internally an unlimited number of times.

  • The bridge extension logs errors in the HiveMQ log file.