Cloudera Docs

Suspending and resuming Cloudera Data Engineering virtual clusters using the API [Technical Preview]

Learn about suspending and resuming Cloudera Data Engineering Virtual Clusters (VCs) using the Cloudera Data Engineering API.

To suspend or resume Cloudera Data Engineering VCs, you need to:

  • Have DEAdmin role
  • Request the DE_ENABLE_VC_SUSPEND_RESUME entitlement
  • Get the base URL
    • You can obtain the BASE-URL by copying the console URL up to the first forward slash (/).

      Example for the console URL: https://console.us-west-1.cdp.cloudera.com/dex/home

      An example for the BASE-URL, which changes according to the region: https://console.us-west-1.cdp.cloudera.com

    • Alternatively:
      1. In the Cloudera Management Console, click the Data Engineering tile.
      2. In the left navigation menu, click Administration.
      3. Right click on the Administration page and select Inspect from the list of options.
      4. Click the Console tab.
      5. Type window.location.origin and press Enter.
  • Get the cdp-session-token (CST)
    1. In the Cloudera Management Console, click the Data Engineering tile.
    2. In the left navigation menu, click Administration.
    3. Right click and select Inspect.
    4. If you use Google Chrome, click the Application tab, or alternatively, if you use Firefox, click the Storage tab.
    5. Click Cookies and select the URL of the console.
    6. Select cdp-session-token.
    7. Double click the displayed cookie value, then right click and select Copy.
    8. Open a terminal screen.
    9. Paste the cookie value in the following command and run the command to get the cdp-session-token (CST). 
      export CST=[***COOKIE-VALUE***]

The virtual cluster-level suspend and resume feature is implemented as a convenient, user-friendly, one-step procedure. Learn about the procedures that are performed when you suspend and resume a virtual cluster.

When you suspend a VC
  1. When a VC is suspended, the VC is transitioned to maintenance mode, where:
    • The VC is locked
    • Running jobs and sessions are killed
    • Job schedules are paused
  2. The suspend operation is started by scaling down all the infrastructure deployments of the given VC.

    Depending on the distribution of the pods, the associated node can be released.

When you resume a VC
  1. During the resume operation, first the VC is resumed.
  2. The VC exits maintenance mode, which involves:
    • Unpausing all the job schedules
    • Unlocking the VC
  1. Suspend a Cloudera Data Engineering VC. 
    curl -H "Cookie: cdp-session-token=${CST}" \  '[***BASE-URL***]/dex/api/v1/cluster/[***CLUSTER-ID***]/instance/<[***INSTANCE-ID***]/suspend>' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -X 'POST' -d ''
  2. Resume the suspended Cloudera Data Engineering VC. 
    curl -H "Cookie: cdp-session-token=${CST}" \  '[***BASE-URL***]/dex/api/v1/cluster/[***CLUSTER-ID***]/instance/<[***INSTANCE-ID***]/resume>' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -X 'POST' -d ''