HiveMQ Enterprise Security Extension (ESE) Start Up Guide

Installation

The HiveMQ Enterprise Security Extension requires HiveMQ version 4.0 or higher. If you currently use an earlier HiveMQ version, you must upgrade before you install ESE. For more information, see our Migration Guide.

  1. Download the HiveMQ Enterprise Security Extension from the HiveMQ Marketplace.

  2. Unzip the download file and open the hivemq-enterprise-security-extension folder.

  3. In the hivemq-enterprise-security-extension folder, locate the enterprise-security-extension.xml file and adjust the configuration for your individual setup.

  4. Move the ESE folder into the extensions folder of your HiveMQ installation.

└─ <HiveMQ folder>
    ├─ bin
    ├─ config
    ├─ data
    ├─ extensions
    │   ├─ hivemq-enterprise-security-extension
    │   └─ ...
    ├─ license
    ├─ log
    └─ ...
To secure an entire HiveMQ cluster, you must install the HiveMQ Enterprise Security Extension on every HiveMQ broker node in the cluster.
If you want to change the configuration of the ESE, you can add a DISABLED file to the ESE folder to disable the extension. To enable ESE once more, remove the DISABLED file that you added.


Configuration

The Enterprise Security Extension needs to be configured to use your particular authentication and authorization sources and meet your individual requirements. Enter your configuration preferences in the enterprise-security-extension.xml file that is located in the hivemq-enterprise-security-extension folder.

The following snippet shows the overall structure of the configuration file:

Configuration file structure
<enterprise-security-extension
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="enterprise-security-extension.xsd"
        version="1">

    <access-log>
        <!-- optional access log configuration -->
    </access-log>
    <realms>
        <!-- add one or more realm configurations -->
    </realms>
    <pipelines>
        <!-- add one or more pipeline configurations -->
    </pipelines>
</enterprise-security-extension>

The root tag in the configuration file is called <enterprise-security-extension>. This tag encloses the configurations for your access log, realms, and pipelines. For more information, refer to the individual sections.

To facilitate configuration and validation, the ESE provides an enterprise-security-extension.xsd schema file next to the configuration file.


Start Up Guide with SQL Databases

This start up guide uses the Postgres docker image. You can also download the Postgres application from the official download page and follow the installation instructions according to your operating system.
  1. To get your PostgreSQL database, open the command line and pull the official docker image for Postgres:

    docker pull postgres
  2. Start the docker container:

    docker run -d --name hivemq-ese-database -p 5432:5432 postgres
  3. Open your favourite database GUI. For example, pgAdmin.

  4. Connect the docker container.

    1. Click on Add New Server.

      Connect postgresql docker container to pgAdmin.

    2. Insert a server name that identifies your server, e.g. hivemq-ese-db.

      Insert postgresql docker server name.

    3. Insert the connection information to the server.

      Insert postgresql docker connection information.

  5. To create a new database:

    1. Use right click > create > Database…​

      Create a new database in pgAdmin.

    2. Insert the name of your database. For example, hivemq-ese-db.

    Insert the name of the new database in pgAdmin.

  6. Create your database tables with the predefined sql script:

    1. Open the Query Editor.

      Click the button with the flash to open the Query Editor.

      Open the Query Editor in pgAdmin.

    2. Open the file search assistant

      Click the button with the folder to open the file search assistant.

    Open File Search Assistant in pgAdmin.

    1. Navigate to the sql subfolder of your HiveMQ Enterprise Security Extension folder.

    2. Select the postgresql_create.sql file.

      1 Select the postgresql_create.sql file.
      2 Click the select button.

      Select the postgresql_create.sql file.

      Alternatively, you can copy and paste following sql script to the query editor:

      SQL Script to create PostgreSQL database tables
      create table users
      (
        id serial not null
          constraint users_pkey
          primary key,
        username text not null
          constraint users_username_unique unique,
        password text,
        password_iterations integer not null,
        password_salt text not null,
        algorithm text not null
      );
      
      create unique index users_id_uindex
        on users (id);
      
      create unique index users_username_uindex
        on users (username);
      
      comment on column users.password is 'Base64 encoded raw byte array';
      
      comment on column users.password_salt is 'Base64 encoded raw byte array';
      
      create table roles
      (
        id serial not null
          constraint roles_pkey
          primary key,
        name text not null
          constraint roles_name_unique unique,
        description text
      );
      
      create unique index roles_id_uindex
        on roles (id);
      
      create unique index roles_name_uindex
        on roles (name);
      
      create table user_roles
      (
        user_id integer not null
          constraint user_roles_users_id_fk
          references users,
        role_id integer not null
          constraint user_roles_roles_id_fk
          references roles,
        constraint user_roles_user_role_pk
        primary key (user_id, role_id)
      );
      
      create table permissions
      (
        id serial not null
          constraint permissions_pkey
          primary key,
        topic text not null,
        publish_allowed boolean default false not null,
        subscribe_allowed boolean default false not null,
        qos_0_allowed boolean default false not null,
        qos_1_allowed boolean default false not null,
        qos_2_allowed boolean default false not null,
        retained_msgs_allowed boolean default false not null,
        shared_sub_allowed boolean default false not null,
        shared_group text
      );
      
      create index permissions_topic_index
        on permissions (topic);
      
      comment on table permissions is 'All permissions are whitelist permissions';
      
      create table role_permissions
      (
        role integer not null
          constraint role_permissions_roles_id_fk
          references roles,
        permission integer not null
          constraint role_permissions_permissions_id_fk
          references permissions,
        constraint role_permissions_role_permission_pk
        primary key (role, permission)
      );
      
      create table user_permissions
      (
        user_id integer not null
          constraint user_permissions_users_id_fk
          references users,
        permission integer not null
          constraint user_permissions_permissions_id_fk
          references permissions,
        constraint user_permissions_user_permission_pk
        primary key (user_id, permission)
      );
    3. Execute the sql statements. In pgAdmin, you should see the created tables in the table section of your side navigation.

      Execute the sql statement in pgAdmin.

      Created Tables result in the side navigation.

  7. Download the Postgres JDBC driver jar file and move it into the drivers/jdbc subfolder of your ESE setup.

    └─ <HiveMQ folder>
        ├─ bin
        ├─ config
        ├─ data
        ├─ extensions
        │   ├─ hivemq-enterprise-security-extension
        │   │   ├─ drivers
        │   │   │   ├─ jdbc
        │   │   │   │   ├─ postgresql-<version>.jar
        │   │   │   ├─ ...
        │   │   ├─ ...
        │   └─ ...
        ├─ license
        ├─ log
        └─ ...
  8. Fill data into the prepared database.

    For your convenience, we provide the ESE Helper Tool to help you insert username and password values for your MQTT clients. Use the ESE Helper Tool to create SQL INSERT statements that you can execute directly in your database.

    For example, to create an MQTT client with the username username, the password password and the algorithm SHA-512 and 100 iterations.

    Command-line command to create an sql insert statement with the ESE Helper Tool
    java -jar hivemq-ese-helper.jar db create-insert-statement username password

    The output is similar to the following example:

    Output of the ESE Helper Tool
    insert into users (username, password, password_iterations, password_salt, algorithm) values ('username', 'yHfGSuyVwq5/hInsBO0T5Xq1Dl25l5iigYHGailLU/5RqnroFW0YKhj4Hr85hnpbqUEcaUZ9bxDOLetP/Vknhg==', 100, 'jkxQqLohaFB5cilFqprocQ==', 'SHA512');
    Because the ESE Helper Tool creates the salt that is prepended to the password randomly, the actual output varies.

    To get started quickly, execute the following sql script in the query editor. This script fills your database with users, roles, and permissions for following three clients:

    • Backendservice: Allows subscription to the topic/+/status topic filter and reception of all messages that match to that topic filter. Publication of messages is not permitted.

      Enables connection with the username = backendservice and password = backendpassword.

    • Frontendclient: Allows publication of messages to the topic/frontendclient/status topic. Subscriptions to the topic/frontendclient/status topic is not permitted.

      Enables connection with the username = frontendclient and password = clientpassword.

    • Superuser: Allows everything. This user has no restrictions.

      Enables connection with the username = superuser and password = supersecurepassword.

      This is just an example. Create more meaningful roles and permissions before you decide to deploy your HiveMQ with HiveMQ ESE to Production.
    SQL script to fill the PostgreSQL database with values
    insert into public.users
      (id, username, password, password_iterations, password_salt, algorithm)
      values
        (1, 'backendservice', 'wtUo2dri+ttHGHRpngg9uG21piWLiKSX7IaNSnU/BfN9pt+ZOLQByG/3JlPPQ7t/pl8S3tjR2+Um/DPBdAQULg==', 100, 'Nv6NU9XY7tvHdSGaKmNTOw==', 'SHA512'),
        (2, 'frontendclient', 'ZHg/rNJel1BHOYMEvc40ekCRUE5vVLcsPF6mk9GPDcdEmX3stm50MplaqjGb8Lxhy6rNFQZSQRSbOxmFZ8ps1Q==', 100, 'JhpW27QU9WfIaG6FJT5MkQ==', 'SHA512'),
        (3, 'superuser', 'nOgr9xVnkt51Lr68KS/rAKm/LqxAt8oEki7vCerRod3qDbyMFfDBGT8obnkw+AGygxCQDWdaA2sQnXXoAbVK6Q==', 100, 'wxw+3diCV4bWXQHb6LLniA==', 'SHA512');
    
    insert into public.permissions
      (id, topic, publish_allowed, subscribe_allowed, qos_0_allowed, qos_1_allowed, qos_2_allowed, retained_msgs_allowed, shared_sub_allowed, shared_group)
      values
        (1, 'topic/+/status', false, true, true, true, true, false, false, ''),
        (2, 'topic/${mqtt-clientid}/status', true, false, true, true, true, true, false, ''),
        (3, '#', true, true, true, true, true, true, true, '');
    
    insert into public.roles
      (id, name, description)
      values
        (1, 'backendservice', 'only allowed to subscribe to topics'),
        (2, 'frontendclients', 'only allowed to publish to topics'),
        (3, 'superuser', 'is allowed to do everything');
    
    insert into public.user_roles
      (user_id, role_id)
      values
        (1, 1),
        (2, 2),
        (3, 3);
    
    insert into public.role_permissions
      (role, permission)
      values
        (1, 1),
        (2, 2),
        (3, 3);
  9. You configure your HiveMQ Enterprise Security Extension with the enterprise-security-extension.xml file that is located in the ESE extension folder.

    Configuration file example for use of a PostgreSQL database
    <?xml version="1.0" encoding="UTF-8" ?>
    <enterprise-security-extension
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:noNamespaceSchemaLocation="enterprise-security-extension.xsd"
            version="1">
        <realms>
            <!-- a postgresql db-->
            <sql-realm>
                <name>postgres-backend</name> (1)
                <enabled>true</enabled>
                <configuration>
                    <db-type>POSTGRES</db-type>
                    <db-name>hivemq</db-name> (2)
                    <db-host>hostname</db-host> (3)
                    <db-port>5432</db-port>
                    <db-username>hivemq</db-username> (4)
                    <db-password>password</db-password> (5)
                </configuration>
            </sql-realm>
        </realms>
        <pipelines>
            <listener-pipeline listener="ALL"> (6)
                <!-- authenticate over a sql db -->
                <sql-authentication-manager>
                    <realm>postgres-backend</realm> (7)
                </sql-authentication-manager>
                <!-- authorize over a sql db -->
                <sql-authorization-manager>
                    <realm>postgres-backend</realm> (7)
                    <use-authorization-key>false</use-authorization-key>
                    <use-authorization-role-key>true</use-authorization-role-key>
                </sql-authorization-manager>
            </listener-pipeline>
        </pipelines>
    </enterprise-security-extension>
    1 Give the PostgreSQL realm a unique name that can be used as a reference.
    2 Replace 'hivemq' with the name of the database that you use.
    3 Replace 'hostname' with the name or the IP address of the host on which the PostgreSQL database runs.
    4 Replace 'hivemq' with your database username.
    5 Replace 'password' with your database password.
    6 Create a listener pipeline and assign it to the listeners that you want to have active on the pipeline.
    7 In the authentication manager and the authorization manager, use the unique realm name that you assigned to the PostgreSQL realm to reference the realm that is used.
  10. That completes your basic setup. Now, you can start HiveMQ with the HiveMQ Enterprise Security Extension.

To get started with ESE on AWS using AWS RDS, have a look at our detailed and highly informative Blogpost.