Pausing a Cluster in AWS

If all data for a cluster is stored on EBS volumes, you can pause the cluster and stop your AWS EC2 instances during periods when the cluster will not be used. The cluster will not be available while paused and can’t be used to ingest or process data, but you won't be billed by Amazon for the stopped EC2 instances. Provisioned EBS storage volumes will continue to accrue charges.

Shutting Down and Starting Up the Cluster

In the shutdown and startup procedures below, some steps are performed in the AWS console and some are performed in Cloudera Manager:
  • For AWS actions, use one of the following interfaces:
    • AWS console
    • AWS CLI
    • AWS API
  • For cluster actions, use one of the following interfaces:
    • The Cloudera Manager web UI
    • The Cloudera API start and stop commands

Shutdown procedure

To pause the cluster, take the following steps:
  1. Navigate to the Cloudera Manager web UI.
  2. Stop the cluster.
    1. On the Home > Status tab, click to the right of the cluster name and select Stop.
    2. Click Stop in the confirmation screen. The Command Details window shows the progress of stopping services.

      When All services successfully stopped appears, the task is complete and you can close the Command Details window.

  3. Stop the Cloudera Management Service.
    1. On the Home > Status tab, click to the right of the service name and select Stop.
    2. Click Stop in the next screen to confirm. When you see a Finished status, the service has stopped.
  4. In AWS, stop all cluster EC2 instances, including the Cloudera Manager host .

Startup procedure

To restart the cluster after a pause, the steps are reversed:
  1. In AWS, start all cluster EC2 instances.
  2. Navigate to the Cloudera Manager UI.
  3. Start the Cloudera Management Service.
    1. On the Home > Status tab, click to the right of the service name and select Start.
    2. Click Start in the next screen to confirm. When you see a Finished status, the service has started.
  4. Start the cluster.
    1. On the Home > Status tab, click to the right of the cluster name and select Start.
    2. Click Start that appears in the next screen to confirm. The Command Details window shows the progress of starting services.

      When All services successfully started appears, the task is complete and you can close the Command Details window.

More information

For more information about stopping the Cloudera Management Service, see Stopping the Cloudera Management Service in the Cloudera Enterprise documentation.

For more information about restarting the Cloudera Management Service, see Restarting the Cloudera Management Service in the Cloudera Enterprise documentation.

For more information about starting and stopping a cluster in Cloudera Manager, see Starting, Stopping, Refreshing, and Restarting a Cluster in the Cloudera Enterprise documentation.

For more information about stopping and starting EC2 instances, see Stop and Start Your Instance in the AWS documentation.

Considerations after Restart

Since the cluster was completely stopped before stopping the EC2 instances, the cluster should be healthy upon restart and ready for use. You should be aware of the following about the restarted cluster:
  • After starting the EC2 instances, Cloudera Manager and its agents will be running but the cluster will be stopped. There will be gaps in Cloudera Manager’s time-based metrics and charts.
  • EC2 instances retain their internal IP address and hostname for their lifetime, so no reconfiguration of CDH is required after restart. The public IP and DNS hostnames, however, will be different. Elastic IPs can be configured to remain associated with a stopped instance at additional cost, but it isn’t necessary to maintain proper cluster operation.