This extension allows your HiveMQ cluster nodes to discover each other dynamically by exchanging their information via S3 from Amazon Web Services (AWS).
Each node will place its own ip-address and port to the given bucket and will read the files from other nodes.
The ip-address and port are taken from the external-address
and external-port
which is configured in the cluster transport
(config.xml).
If they are not set, the bind-address
and bind-port
will be used.
The extension will regularly check the configured S3 bucket for files from other HiveMQ nodes, which contains the information on how to connect to them.
Additionally every broker updates its own file on a regular basis to prevent the file from expiring.
The s3discovery.properties
can be reloaded during runtime.
extensions
folder of your HiveMQ nodes.s3discovery.properties
file for your needs.extension
.Config name | Required | Description |
---|---|---|
s3-bucket-name | x | Name of the S3 bucket to use. |
s3-bucket-region | x | The region in which this S3 bucket resides. (List of regions: AWS documentation) |
file-prefix | x | Prefix for the filename of every node’s file. |
file-expiration | x | Timeout in seconds after a file on S3 will be removed. |
update-interval | x | Interval in seconds in which the own information will be updated. (Must be smaller than file-expiration ) |
s3-endpoint | Endpoint url to use other S3 compatible storage services. | |
s3-endpoint-region | The region of the endpoint. (Optional) | |
s3-path-style-access | De-/activate path style access. Information about path style access can be found in the AWS documentation. |
Example Configuration
|
|
The extension uses AWS API operations for exchanging cluster information.
These operations work with Access Keys
, if you don’t know what access keys are or how to generate them please look at
the official AWS documentation
about credentials.
Default Authentication tries to access S3 via the default mechanisms in the following order:
Example Default Config
credentials-type:default
Uses environment variables to specify your AWS credentials the following variables need to be set:
Linux example
|
|
Example Environment Variables Config
|
|
Uses Java system properties to specify your AWS credentials the following Java system properties need to be set:
Example Java System Properties Config
|
|
You can for example extend the run.sh
or run.bat
in order to start HiveMQ with the system properties:
Example for extending run.sh
|
|
Uses the credentials file which can be created by calling ‘aws configure’ (AWS CLI).
This file is usually located at ~/.aws/credentials. The exact location can vary based on the platform.
The location of the file can be configured by setting the environment variable AWS_CREDENTIAL_PROFILE_FILE
to the location of your file.
Example Java System Properties Config
|
|
Uses the IAM Roles assigned to the EC2 instance running HiveMQ to access S3.
WARNING: This only works if HiveMQ is running on an EC2 instance and your EC2 instance has configured the right IAM Role to access S3!
Example Instance Profile Credentials Config
|
|
Uses the credentials specified in the s3discovery.properties
file.
The variables you must provide are:
credentials-access-key-id
credentials-secret-access-key
Example Instance Profile Credentials Config
|
|
Uses the credentials specified in s3discovery.properties
file to authenticate with a temporary session.
The variables you must provide are:
credentials-access-key-id
credentials-secret-access-key
credentials-session-token
Example Instance Profile Credentials Config
|
|