Deploying HiveMQ on AWS ECS vs. AWS EKS: Pros and Cons
AWS ECS or AWS EKS? Which should you choose when deploying HiveMQ? HiveMQ is a world-class, enterprise-ready IoT Data Streaming Platform, built on MQTT, offering fast, efficient, and reliable movement of data to and from connected IoT devices. HiveMQ is the most trusted, scalable, available, and performant way to run MQTT-centric systems. It is the best way to run Brokers at the core of MQTT-centric systems in Smart Manufacturing, Internet of Things (IoT) and Industrial Internet of Things (IIoT), and Industry 4.0 scenarios.
HiveMQ MQTT Broker is flexible and can run on many operating systems, cloud-based environments, and on-premises environments. These include all major Public Cloud providers—AWS, Azure, GCP, Oracle; hypervisors and virtualization environments like VMWare, Nutanix, HyperV; Linux and Windows operating systems; ARM or Intel-based hardware; and Raspberry PI and industrial computers (IPCs).
HiveMQ excels when it comes to highly available and scalable systems by virtue of its clustering capability. HiveMQ’s proprietary masterless cluster mechanism provides near-linear scale-out to meet changing demands in workload. In order to run HiveMQ in an elastic, scalable fashion, it is most common to use a container orchestration system like AWS Elastic Container System (ECS) or AWS Elastic Kubernetes (EKS). EKS and ECS help you run HiveMQ and other applications by managing the provisioning, configuration, scaling, operations, and upgrades in a standard, managed way. This lets you focus on the application rather than scripting and performing operations manually.
While EKS and ECS services both run and manage containerized applications, there are some key differences to keep in mind. This article dives into key considerations for choosing whether to deploy HiveMQ to AWS ECS or AWS EKS.
Tl;DR: Pros and Cons of AWS ECS vs. AWS EKS
| AWS ECS | AWS EKS | |
|---|---|---|
| Pros | Simpler to set up and manage, cost-effective for many use cases | Highly flexible and scalable; many customization options; portable and avoids vendor-lock to an extent; abstracts infrastructure and application concepts for flexibility and portability |
| Cons | Less flexibility compared to Kubernetes; potential vendor lock-in; not suited for moderate to high complexity or load applications; not portable outside of AWS | More complex setup and management and higher operational overhead than ECS; potentially more expensive cloud cost to operate |
Understanding AWS ECS
Amazon Elastic Container Service (ECS) is one of AWS’s services for running containerized applications. It is a managed container orchestration service, and uses concepts and principles like Tasks, Task Definitions, Services, and Clusters to define, declare, and manage one or more groups of containers. AWS ECS is very powerful while still being relatively easy to implement.
Understanding AWS EKS
Amazon Elastic Kubernetes Service (EKS) is AWS’s service for running Kubernetes Clusters in the AWS Cloud. It is a managed Kubernetes service that uses all of the concepts and features of Kubernetes, including Pods, Services, Deployments, Namespaces, and Custom Resource Definitions (CRDs). It extends standard, open-source Kubernetes with rich integration possibilities with other AWS services like AWS IAM, Secrets, Networking, and more. It lets you use all of the familiar Kubernetes features and make them work natively in AWS environments.
Pros and Cons of Deploying HiveMQ on AWS EKS
Running HiveMQ on Kubernetes is highly flexible and scalable. There is a large community for support. And, because Kubernetes is open-source, you need not be concerned with vendor lock-in. You can easily run Kubernetes on public clouds, on-premises environments, and local workstation environments. If you need to move a workload from one platform to another, the common base is Kubernetes. It abstracts away many details of local infrastructure
EKS has a more complex setup and management, as well as higher operational overhead, compared to ECS.
Comparison: AWS ECS versus AWS EKS for HiveMQ
HiveMQ on ECS versus EKS shares similarities and differences in:
Deployment complexity
Scalability and resource management.
Cloud Cost
Operational overhead
Community support and ecosystem
Security considerations
The following table summarizes key factors of ECS and EKS that should be taken into account when running stateful applications like HiveMQ Platform Broker. The material is derived from AWS source information (website and documentation).
| Factor | AWS ECS | AWS EKS |
|---|---|---|
| Orchestration Model | Offers a more AWS-native approach. Storage is directly configured in task definitions. It's often considered simpler for those not already invested in Kubernetes. | Provides the full power and complexity of Kubernetes. Stateful application management relies on Kubernetes primitives like StatefulSets, PVs, PVCs, and CSI, offering a highly flexible and standardized model widely used in the container ecosystem. |
| State Management Primitives | Lacks a direct equivalent of StatefulSets. Ordered deployments and sticky identities need more manual setup or custom solutions. | StatefulSets are a core feature designed specifically for complex stateful applications, simplifying their deployment and management. |
| Storage Abstraction and Flexibility | Provides direct integration with specific AWS storage services. | The CSI model offers a broader abstraction, allowing for a wider range of storage solutions (including third-party) to be integrated in a standard Kubernetes way. |
The choice between using AWS ECS and AWS EKS depends on your team’s experience and familiarity with Kubernetes, the complexity of your stateful application, and your desired level of control versus operational simplicity.
If you are already using Kubernetes or need its advanced features for stateful workloads (like StatefulSets), AWS EKS is a natural fit.
However, if you prefer a simpler, more AWS-integrated experience for container orchestration and your stateful needs can be met with direct AWS storage integrations, AWS ECS may be a more straightforward choice.
Bear in mind that there are still complexities and considerations associated with running stateful applications like HiveMQ on AWS ECS. You’ll need to carefully plan for stateful storage and storage performance, depending on the choices you make in ECS.
For example, Amazon Elastic File System (EFS) is an easy way to have persistent, accessible storage from containerized applications. However, EFS is not a good choice for HiveMQ persistent storage due to its lower performance as compared to Amazon Elastic Block Store (EBS). On the other hand, EBS is higher performance, but you must make sure that the volumes are appropriately mounted in the containerized application.
Decision Factors for Running HiveMQ on AWS
The following are factors to consider when planning and deciding between AWS EKS and AWS ECS. Please reach out to HiveMQ to set up a discussion. If you’re an active customer, contact your customer success representative for a conversation about HiveMQ on ECS versus EKS.
1. Business Requirements
What is the complexity and scope of your systems?
2. Team Expertise and Experience
Is there a strong experience with one technology or the other?
3. Budget Constraints
Use AWS’s Pricing Calculator to compare ECS and EKS costs; run several scenarios to get a better understanding of potential costs.
4. Scalability Needs
Are your workloads expected to be low, medium, or high? Will they fluctuate or need to scale-out, scale-in?
5. Long-Term Strategy
What is your organization’s approach to running containerized applications? Will your company stay in AWS? Do you need to remain portable? Is the company moving toward more Kubernetes-based applications?
Conclusion
HiveMQ can run on both AWS ECS and AWS EKS. AWS ECS may be a good choice for lighter workloads or where cloud costs are a driving factor. ECS is a relatively easy way to get started running containerized workloads like HiveMQ in the cloud. However, ECS comes with some additional caveats and limitations related to persistent storage, clustering, and load balancing.
On the other hand, EKS is a good choice for more complex workloads. EKS manages the lifecycle of stateful applications like HiveMQ very well. Running HiveMQ on Kubernetes comes with additional complexity. Consider using the HiveMQ Platform Operator for Kubernetes to better manage and simplify operating HiveMQ on Kubernetes.
In summary, when it comes to operating containerized HiveMQ Brokers, carefully consider your choices. We recommend seriously evaluating and using the HiveMQ Platform Operator for Kubernetes, because it can ease the complexities and simplify managing containerized HiveMQ Broker clusters. The Operator deploys and manages HiveMQ Platform clusters on Kubernetes. It supports all HiveMQ Platform versions starting with our 4.28 Long-Term Support (LTS) and later. The Operator features numerous usability, reliability, and scalability enhancements. Contact us for more information and a technical discussion.
Additional Resources
See these links for more information on running HiveMQ on ECS and EKS as well as running HiveMQ on Kubernetes using the HiveMQ Platform Operator for Kubernetes.
Bill Sommers
Bill Sommers is a Technical Account Manager at HiveMQ, where he champions customer success by bridging technical expertise with IoT innovation. With a strong background in capacity planning, Kubernetes, cloud-native integration, and microservices, Bill brings extensive experience across diverse domains, including healthcare, financial services, academia, and the public sector. At HiveMQ, he guides customers in leveraging MQTT, HiveMQ, UNS, and Sparkplug to drive digital transformation and Industry 4.0 initiatives. A skilled advocate for customer needs, he ensures seamless technical support, fosters satisfaction, and contributes to the MQTT community through technical insights and code contributions.
