Using Spot Instances

To help you manage your cloud resource costs, Cloudera supports Spot instances. Spot instances are Amazon EC2 instances that you can bid on. Unlike On-Demand Amazon EC2 instances, Spot instances only run as long as the price you bid exceeds the current Spot price. This allows you to add capacity to your workload at a low price.

Spot instances run just like On-Demand instances, except that they are not provisioned until the instance price falls below your bid. They also terminate automatically when the instance price exceeds or equals your bid price.

For more information about using Spot instances, see the Amazon EC2 documentation. For help with bidding on Spot instances, see the Spot Bid Advisor.

Planning for Spot Instances

It is normal for Spot instances on a cluster to disappear over time. However, Cloudera Manager does not see that these instances are terminated. If you use Cloudera Manager to restart a cluster that contains a Spot instance group, and the Spot instances have terminated, the restart fails. If you are modifying any group in the cluster that has lost Spot instances, do not select the Restart checkbox.

If your bid price is so low that you do not obtain an instance when the group is created, you will have 0 instances in your group. If this happens, you can:
  • Delete the entire group.
  • Add more instances to the group.
  • Delete unprovisioned instances from the group (only as part of adding more instances to the group).
  • Retry (repair) existing instances.
You cannot do the following:
  • Change the bid price, due to AWS restrictions for spot instance.
  • Delete all instances without adding more, due to the minimum instance count requirement.

The bid price for Spot instances is set in an instance template. This template is associated with a group. Although you can modify the group, you cannot change the bid price. Therefore, if you set the bid price too low for successful provisioning, you must delete the group where that price is set and create a new group with the higher bid price. You must also delete the current group and create a new one if you want to drop the bid price.

Specifying Spot Instances

To specify Spot instances, create a new instance template and use this template for your group. In the Advanced Options section of the Create New Instance Template wizard, check the Use Spot Instances checkbox and enter a value in the Spot bid field.


Spot Blocks: Specifying a Duration for Spot Instances

The AWS Spot block feature enables you to specify a fixed duration ranging from one to six hours for Spot instances. Spot instances with a predefined duration use a fixed hourly price that remains in effect for the Spot instance while it runs. The price for instances with a Spot block will not be as low as that for ordinary Spot instances, but will be lower than that of on-demand instances, and Spot block instances are guaranteed to run for the specified duration, after which they are terminated.

To configure an instance template for Spot block instances, check the Use Spot Instances checkbox, enter a value in the Spot bid field, and choose a value from one to six hours (in intervals of 60 minutes) in the Spot Block Duration dropdown.


See Specifying a Duration for Your Spot Instances in the AWS documentation for more information about Spot blocks.

Best Practices for Using Spot Instances

  • Use a Spot instance worker group in conjunction with an On-Demand worker group. This ensures that the cluster can redo computational tasks run on Spot instances that could be terminated before the tasks are finished.
  • Use Spot instances only in contexts where the loss of the instance can be tolerated, as in a worker group. Do not use Spot instances for master nodes or for data storage.
  • Use a minimum count of 0 for Spot instance groups. If you use a number above 0, the cluster will likely enter a failed state. If the cluster fails, contact Cloudera support for help.

Additional best practices for using spot instances can be found in Spot Instance Interruptions in the AWS documentation.