HiveMQ with Azure Kubernetes Service (AKS)

Introduction

With the release of the HiveMQ Kubernetes Operator, deploying, configuring and running a high-available HiveMQ Cluster to the cloud has become easy and convenient. User only need to manage a single or very few YAML file(s) and the Operator will take care of orchestrating the entire cluster.

The HiveMQ Operator is built to work with any K8s cluster, as long as it is version 1.13 or later. This includes OnPrem installation as well as managed cloud offerings. Today’s blog post focuses on the use of Microsoft’s managed Azure Kubernetes Service (AKS). This service allows us to deploy a HiveMQ Cluster to the Azure Cloud in just a few steps:

As illustrated in the image above we will use the Azure CLI to create an Azure Resource Group in which we will then deploy a 4-node Kubernetes Cluster. Next, we will use Helm to install the HiveMQ K8s Operator and use it with a customized YAML file to install a 3-node HiveMQ Cluster and to setup Prometheus and Grafana on the fourth node so that we can monitor HiveMQ. The HiveMQ K8s Operator will also define a Load Balancer that can be used to connect MQTT clients to the HiveMQ Cluster. For demonstration purposes we will be using MQTT-CLI.

Prerequisites

Azure Account

In order to use the Azure services you need an Azure Account.

(Optional) HiveMQ License

Unless you have a valid HiveMQ license, the deployed HiveMQ Cluster will only be a trial version to evaluate HiveMQ. The cluster will be restricted to 25 concurrent connections in this case.

Contact us in case you want to evaluate HiveMQ with a bigger load.

Azure CLI

The Azure command-line interface is used to create and manage Azure resources. We will use it to create an Azure Resource Group and a Kubernetes cluster in it on the Azure cloud. You can install it on macOS via Homebrew:

1

brew install az

kubectl

kubectl is the official command-line interface to interact with Kubernetes clusters. We will use it to manage the created Kubernetes cluster on Azure. You can install kubectl on all operating systems by using the previously installed Azure CLI:

1

az aks install-cli

Helm

Helm is the package manager for Kubernetes. It is used to easily define, install and update the software running on a Kubernetes cluster. We will use it to install and configure HiveMQ on the cluster. You can install Helm on macOS via Homebrew:

1

brew install helm

MQTT-CLI

MQTT-CLI is a command-line interface provided by HiveMQ to quickly simulate MQTT-Clients. We will use it to connect clients and interact with our HiveMQ Cluster. You yan install MQTT-CLI on macOS via Homebrew:

1

brew install hivemq/mqtt-cli/mqtt-cli

Setup Kubernetes Cluster With AKS

The first step to deploy a HiveMQ Cluster on AKS is to create a Kubernetes cluster on Azure by using the Azure CLI. Open a terminal and use the command:

1

az login

This will open your browser with a login page asking for your Microsoft credentials. After successfully submitting your credentials you should see a confirmation in your terminal.

Before you can create the cluster you need to choose a region for hosting. You can get a JSON of all available locations by using the following command:

1

az account list-locations

After having chosen your location (We will use ‘germanywestcentral’ in this guide), you need to create an Azure resource group, which holds all the necessary resources for your cluster:

1

az group create --name hmqResourceGroup --location germanywestcentral

Finally, create a AKS-managed Kubernetes cluster consisting of four nodes with the Azure virtual machine type Standard_A8_v2 (8 CPUs, 16 GiB RAM) within this resource group (This may take some time):

1

az aks create -g hmqResourceGroup -n HiveMQCluster --node-count 4 --node-vm-size Standard_A8_v2

In order to use kubectl to manage your previously created Kubernetes cluster you need to download the cluster’s access credentials:

1

az aks get-credentials -g hmqResourceGroup -n HiveMQCluster

Next, you should be able to verify that your four nodes are available:

1

kubectl get nodes

The command should give you an output similar to this:

1
2
3
4
5

NAME                                STATUS   ROLES   AGE     VERSION
aks-nodepool1-37471664-vmss000000   Ready    agent   5m44s   v1.18.10
aks-nodepool1-37471664-vmss000001   Ready    agent   5m50s   v1.18.10
aks-nodepool1-37471664-vmss000002   Ready    agent   5m47s   v1.18.10
aks-nodepool1-37471664-vmss000003   Ready    agent   5m47s   v1.18.10

(Optional) Add A License To Your HiveMQ Cluster

If you have a valid HiveMQ license make it accessible for the HiveMQ deployment in the next step by creating a Kubernetes ConfigMap hivemq-license from your license file:

1

kubectl create configmap hivemq-license --from-file=hivemq-license.lic

Deploy HiveMQ On The Cluster

Finally, we can use the HiveMQ K8s Operator to deploy and manage our HiveMQ Cluster on the Kubernetes nodes. For this we need to add the HiveMQ Helm repository to our Helm installation:

1

helm repo add hivemq https://hivemq.github.io/helm-charts

Next, you need to define the custom configuration YAML file for your HiveMQ Cluster. To create a 3-node HiveMQ Cluster with a load balancer exposing the MQTT port, access to the HiveMQ Control Center and a dedicated monitoring instance you can save the following configuration as a yaml file called hivemq.yaml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

hivemq:
  nodeCount: "3"

  ports:
    - name: "mqtt"
      port: 1883
      expose: true
      patch:
        - '[{"op":"add","path":"/spec/type","value":"LoadBalancer"}]'
    - name: "cc"
      port: 8080
      expose: true
      patch:
        - '[{"op":"add","path":"/spec/sessionAffinity","value":"ClientIP"}]'

 #  Uncomment the following lines if you have a valid HiveMQ license
 #  configMaps:
 #   - name: hivemq-license
 #     path: /opt/hivemq/license

monitoring:
  dedicated: true
  enabled: true

Lastly, install the HiveMQ K8s Operator which will deploy the configured HiveMQ Cluster from the previously defined YAML file:

1

helm upgrade --install -f hivemq.yaml hivemq hivemq/hivemq-operator

You can check if your cluster is running by using the following command:

1

kubectl get hivemq-clusters

As soon as the STATUS of the cluster has changed to Running you can proceed with the next steps.

Connect To Your HiveMQ Cluster

After the setup of your HiveMQ Cluster was successful you can now connect to it by using MQTT-CLI. Once more we need to use kubectl get hivemq-clusters to obtain the exposed ENDPOINT of the created load balancer on which we can then connect to our cluster:

1
2
3

$ kubectl get hivemq-clusters
NAME     SIZE   IMAGE            VERSION     STATUS    ENDPOINT       MESSAGE
hivemq   3      hivemq/hivemq4   k8s-4.4.3   Running   20.52.209.35   Live updates completed successfully

Next, we start up the shell mode of the CLI with the following command:

1

mqtt sh

In the shell-mode we can connect a new MQTT client to our cluster on the previously acquired endpoint address:

1

con -h 20.52.209.35

We now have successfully connected a new MQTT client to a HiveMQ Cluster hosted on Azure. Check out the MQTT-CLI documentation on how to further interact with HiveMQ.

Monitor HiveMQ

Control Center

The fastest way to get some insight into HiveMQ is by using the HiveMQ Control Center. We can get access to the Control Center by forwarding its port to our local machine:

1

kubectl port-forward svc/hivemq-hivemq-cc 8080:8080

This will block the console and make the Control Center available on http://localhost:8080, where you can login by using the username admin and the password hivemq:

Grafana

To gain an in-depth view into HiveMQ you can use the Grafana dashboard provided by the HiveMQ Kubernetes Operator. To access the dashboard, forward its port to your local machine:

1

kubectl port-forward svc/hivemq-grafana 8081:80

You can now access the Grafana dashboard by visiting http://localhost:8081 and logging into it by using admin as username and prom-operator as password. Then select the dashboard HiveMQ Cluster to access a pre-defined dashboard to get insights into the most important metrics of HiveMQ:

Configure Your HiveMQ Cluster

You can easily adapt the HiveMQ Cluster to your needs by making changes to the YAML file and applying those changes via Helm. To get an overview of all the possible configuration options, which you may want to override use the following Helm command:

1

helm show values hivemq/hivemq-operator

After you have modified your YAML file according to your desired configuration you just need to apply it via Helm:

1

helm upgrade --install -f hivemq.yaml hivemq hivemq/hivemq-operator

This will initiate a rolling upgrade of your cluster and updates it to the new desired state. You can track the state of your cluster by using kubectl get hivemq-clusters.

Tearing Down The Cluster

To delete the whole created cluster simply delete the Azure resource group by using the Azure CLI:

1

az aks delete -g hmqResourceGroup -n HiveMQCluster

Conclusion

This guide demonstrated how convenient it is to leverage the HiveMQ K8s Operator and a managed Kubernetes service in the cloud to quickly create a high availability HiveMQ Cluster with monitoring.
The same tools and principles can be used with other Kubernetes deployments. Let us know in the comments, if you have used the HiveMQ K8s Operator yourself.

About Till Seeberger

Till is a software engineer at HiveMQ and maintainer of the MQTT CLI open-source project.

Contact Till