Managing Clusters
Also available as:

Configure autoscaling

Autoscaling allows you to adjust cluster capacity based on Ambari metrics and alerts, as well as schedule time-based capacity adjustment. Refer to this section if you would like to configure autoscaling for Cloudbreak-managed clusters.

When creating an autoscaling policy, you define:

  • An alert that triggers a scaling policy. An alert can be based on an Ambari metric or can be time-based.
  • A scaling policy that adds or removes a set number of nodes to a selected host group when the conditions defined in the attached alert are met.

Metric-based autoscaling

Cloudbreak accesses all available Ambari metrics and allows you to define alerts based on these metrics. For example:

Alert Definition Policy Definition
ResourceManager CPU alert with CRITICAL status for 5 minutes Add 10 worker nodes
HDFS Capacity Utilization alert with WARN status for 20 minutes Set the number of worker nodes to 50
Ambari Server Alerts alert with CRITICAL status for 15 minutes Decrease the number of worker nodes by 80%

Time-based autoscaling

Time-based alerts can be defined by providing a cron expression. For example:

Alert Definition Policy Definition
Every day at 07:00 AM (GMT-8) Add 90 worker nodes
Every day at 08:00 PM (GMT-8) Remove 90 worker nodes

Cluster resizing is not supported for HDF clusters.

Overview of configuring autoscaling

  1. Enable autoscaling.
  2. Define a time-based or metric-based alert.
  3. Create a scaling policy.
  4. Review and, if needed, configure autoscaling settings.