Pausing Cloudera Director Instances

The Cloudera Director server can be stopped at any time to reduce costs, as can clusters running on Microsoft Azure. These actions are described in the sections below.

Pausing the Cloudera Director Server

Although you can stop and start the Cloudera Director server at any time, you should wait for running workflows to complete.

To start or stop the server, enter the following command:

$ sudo service cloudera-director-server [start | stop]

Pausing a Cluster in Azure

There are times when it makes sense to shut down a cluster and stop your Azure virtual machines. The cluster will not be immediately available while stopped and it cannot be used to ingest or process data, but you won't be billed by Microsoft for virtual machines that are stopped and deallocated. Provisioned premium storage disks, and space used by standard storage disks will continue to accrue charges.

Pausing a cluster requires using premium or standard storage disks for all storage, both on management and worker nodes. Data stored on ephemeral disks will be lost after a cluster pause.

Pausing a cluster also requires that the private IP address for all Azure Virtual Machines be set to static.

Notes on Pausing a Cluster

Shutdown procedure

To pause the cluster, take the following steps:
  1. Change all virtual machine’s private IP address from dynamic to static by following the instructions in the Important note paragraph above in Notes on Pausing a Cluster.
  2. Stop the cluster
  3. Stop Cloudera Management Service
  4. Stop all cluster virtual machines, including the Cloudera Manager host

Startup procedure

To restart the cluster after a pause, the steps are reversed:
  1. Start all cluster virtual machines
  2. Start Cloudera Management Service
  3. Start the cluster
Since the cluster was completely stopped before stopping the virtual machines, the cluster should be healthy upon restart and ready for use.

Considerations

After starting the virtual machines, Cloudera Manager and its agents will be running but the cluster will be stopped. There will be gaps in Cloudera Manager’s time-based metrics and charts.

Dynamic public IP addresses will be different after virtual machine shutdown. Static public IP addresses can be configured to remain associated with a stopped virtual machine at additional cost, but it isn’t necessary to maintain proper cluster operation.