Suspending and resuming Cloudera Data Engineering Services using the API - Technical preview

This document provides a step-by-step guide on how to suspend and resume Cloudera Data Engineering Services using the API, enabling efficient management of resources and cost optimization.

In AWS: Suspending the Service scales down all compute nodes used by the Cloudera Data Engineering service and its Virtual Clusters (VCs) to zero.

In Azure: The feature stops the AKS cluster during suspension and restarts it when the Service is resumed.

To suspend and resume your Cloudera Data Engineering Services, you need to:

Request the DE_ENABLE_SUSPEND_RESUME entitlement
Have DE Admin user privileges at the environment level

Get the base URL

You can obtain the BASE-URL by copying the console URL up to the first forward slash (/).

An example for the BASE-URL, which changes according to the region: https://console.us-west-1.cdp.cloudera.com

Alternatively:

In the Cloudera Management Console click the Data Engineering tile and navigate to the Cloudera Data Engineering Administration page.
Right click and select Inspect.
Click the Console tab.
Type window.location.origin and press enter in the Console tab.
Use the output as the value for BASE-URL.

Get the cdp-session-token (CST)

In the Cloudera Management Console click the Data Engineering tile and navigate to the Cloudera Data Engineering Administration page.
Right click and select Inspect.
If you use Google Chrome, click the Application tab, or alternatively, if you use Firefox, click the Storage tab.
Click Cookies and select the URL of the console.
Select cdp-session-token.
Double click the displayed cookie value, then right click and select Copy.
Open a terminal screen.
Paste the cookie value in the following command and run the command to get the CST.
```
export CST=[***COOKIE-VALUE***]
```

Suspend your Cloudera Data Engineering Service.
1. Enter maintenance mode.
  The maintenance mode stops all running jobs and pauses all scheduled jobs.
```
curl -H "Cookie: cdp-session-token=${CST}" \  '[***BASE-URL***]/dex/api/v1/cluster/<cluster-id>/maintenance/enter' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -X 'POST' -d '{
  "message": "Reason to suspend",
  "operation": "Suspend"
}'
```
  Once the maintenance step completes, you can suspend the service.
2. Suspend the Cloudera Data Engineering Service.
  Depending on the cloud provider, this operation scales down the cluster to zero (AWS) or shuts down the cluster (Azure).
```
curl -H "Cookie: cdp-session-token=${CST}" \
  '[***BASE-URL***]/dex/api/v1/cluster/<cluster-id>/suspend' \
  -H 'accept: application/json' \
  -X 'POST' -d ''
```
Resume your suspended Cloudera Data Engineering Service.
This action scales up the compute nodes.
```
curl -H "Cookie: cdp-session-token=${CST}" \
  '[***BASE-URL***]/dex/api/v1/cluster/<cluster-id>/resume' \
  -H 'accept: application/json' \
  -X 'POST' -d ''
```
Once the Service and Virtual Clusters are all running, you can exit maintenance mode.
Exit maintenance mode.
This action restores the Cloudera Data Engineering cluster to an operational state, and resumes all previously paused scheduled jobs.
```
curl -H "Cookie: cdp-session-token=${CST}" \  '[***BASE-URL***]/dex/api/v1/cluster/<cluster-id>/maintenance/quit' \
  -H 'accept: application/json' \
  -X 'POST' -d ''
```