Suspending and resuming Cloudera Data Engineering virtual clusters using a Python-based CLI tool - [Technical preview]

You have the option to leverage the suspend and resume Python-based CLI tool to automate suspending and resuming your Cloudera Data Engineering Virtual Clusters (VCs) using the CDP CLI. This tool provides robust error handling, automatic retries, and support for multiple VCs.

After specifying the cluster ID, the VC IDs, the operation type, and the profile, the suspend-resume-vc.py script suspends or resumes the VCs specified by the VC IDs.

To suspend and resume your Cloudera Data Engineering VCs, you need the following:

  • Request the DE_ENABLE_VC_SUSPEND_RESUME entitlement
  • Have DEAdmin user privileges at the environment level
  • Python 3.6 or higher
  • Install the CDP CLI Beta. For more information, see Installing Beta CDP CLI.
  • Configure CDP CLI profiles with appropriate credentials:
    1. Configure a profile (if not already done).
      cdp configure --profile [***MY-PROFILE***]
    2. Verify the configuration.
      cdp de list-services --profile [***MY-PROFILE***]
  1. Clone or download the suspend-resume-vc.py script from GitHub.
  2. Optional: Ensure that the script is executable.
    chmod +x suspend-resume-vc.py
  3. Using the CDP CLI Beta, run the suspend and resume script:
    python suspend-resume-vc.py \
      --cluster-id [***CLUSTER-ID***] \
      --vc-ids [***VC-ID-1***] [***VC-ID-2***] [***VC-ID-...***] \
      --operation [***SUSPEND | RESUME***] \
      --profile [***CDP-PROFILE-NAME***]
    

    Specify the following parameters:

    --cluster-id
    The ID of the Cloudera Data Engineering service.

    To obtain the cluster ID:

    1. On the Cloudera Data Engineering UI, go to Administration in the left-navigation menu.
    2. Select the respective service, then click on the Service Details icon, and copy the CLUSTER ID.
    --vc-ids
    One or more VC IDs separated with a space.

    To obtain the VC ID:

    1. On the Cloudera Data Engineering UI, go to Administration in the left-navigation menu.
    2. Select the respective service and then the respective VC.
    3. Click on the Virtual Cluster Details icon, and copy the VC-ID.
    --operation
    The operation type, which can be suspend or resume.
    --profile
    The CDP CLI profile name.

    Example for suspending multiple VCs:

    python suspend-resume-vc.py \
      --cluster-id cluster-j5fjn9tf \
      --vc-ids dex-app-aaa dex-app-bbb dex-app-ccc \
      --operation suspend \
      --profile default

    Example for resuming multiple VCs:

    python suspend-resume-vc.py \
      --cluster-id cluster-j5fjn9tf \
      --vc-ids dex-app-aaa dex-app-bbb dex-app-ccc \
      --operation resume \
      --profile default
    The tool provides detailed logging with timestamps.
  4. If the suspend or resume operation fails, try once again before contacting Cloudera Support.