Cloudera Docs

Suspending and resuming Cloudera Data Engineering Services using the CLI - Technical preview

This document provides a step-by-step guide on how to suspend and resume Cloudera Data Engineering Services using the CLI, enabling efficient management of resources and cost optimization.

In AWS
Suspending the Service scales down all compute nodes used by the Cloudera Data Engineering Service and its Virtual Clusters (VCs) to zero.
In Azure
The feature stops the AKS cluster during suspension and restarts it when the Service is resumed.

To suspend and resume your Cloudera Data Engineering Services, you need to:

  • Request the DE_ENABLE_SUSPEND_RESUME entitlement
  • Have DE Admin user privileges at the environment level
  1. Suspend your Cloudera Data Engineering Service.
    1. Enter maintenance mode.

      The maintenance mode stops all running jobs and pauses all scheduled jobs.

      cdp de suspend-resume-service --cluster-id <value> --step "PREPARE"

      Once the maintenance step completes, you can suspend the Service.

    2. Suspend the Cloudera Data Engineering Service.

      Depending on the cloud provider, this operation scales down the cluster to zero (AWS) or shuts down the cluster (Azure).

      cdp de suspend-resume-service --cluster-id <value> --step "SUSPEND"
    3. Check the suspend-resume status of the Virtual Cluster. 
      cdp de get-suspend-resume-status --cluster-id <value>
  2. Resume the Cloudera Data Engineering Service.
    1. Resume your suspended Cloudera Data Engineering Service.
      This action scales up the compute nodes. 
      cdp de suspend-resume-service --cluster-id <value> --step "RESUME"

      Once the Service and Virtual Clusters are all running, you can exit maintenance mode.

    2. Check the suspend-resume status of the Virtual Cluster. 
      cdp de get-suspend-resume-status --cluster-id <value>
    3. Exit maintenance mode.
      This action restores the Cloudera Data Engineering cluster to an operational state, and resumes all previously paused scheduled jobs.
      cdp de suspend-resume-service --cluster-id <value> --step "QUIT"