Migrating Cloudera Data Engineering Services deployed on deprecated Redhat UBI images to Security-Hardened images
An optional restore path is available to migrate Virtual Cluster (VC) backups created with deprecated Redhat UBI images to Security-Hardened images. This capability allows administrators to restore backups after the support for insecure Redhat UBI images was removed in Cloudera Data Engineering 1.26.2.
The force_secure_spark_os parameter was added to the
restore service API, allowing insecure images to be force-migrated to Security-Hardened images
when set to true. By default, this parameter is set to
false.
Steps
- Add the following payload when calling the restore service
API:
cdpcurl -X 'POST' \ 'CDP_ENDPOINT_URL/dex/api/v1/cluster/backup/restore' \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d {"backup_id": [***BACKUP-ID***], "environment_crn": "[***ENVIRONMENT-CRN***]", "cluster_name": "[***CLUSTER-NAME***]", "force_secure_spark_os": true}The payload uses the following variables:
BACKUP-ID- The ID of the backup that you are restoring from. For example:"backup_id": 4ENVIRONMENT-CRN- The Customer Resource Number (CRN) of the Cloudera environment with which a restored Cloudera Data Engineering Service is associated. Currently, you can restore the Cloudera Data Engineering service only to the same Cloudera environment to which the backed-up service is associated. For example:"environment_crn": "crn:cdp:environments:us-west-1:9d74eee4-1cad-45d7-b645-7ccf9edbb73d:environment:2db7823a-c51a-4a0e-b73a-dc5b2a0418d5"CLUSTER-NAME- The name of the restored Cloudera Data Engineering Service. For example:"cluster_name": "test-insecure3"
Step result: The affected VCs are restored with Security-Hardened images.
Step troubleshooting
- Check the job restore logs at
${[***BASE-URL***]}/dex/api/v1/cluster/${[***CLUSTER-ID***]}/initialization-logsto find out what failed to be restored.The command uses the following variables:
BASE-URLGet the base URL:
- You can obtain the
BASE-URLby copying the console URL up to the first forward slash (/).Example for the console URL:
https://console.us-west-1.cdp.cloudera.com/dex/homeAn example for the BASE-URL, which changes according to the region: https://console.us-west-1.cdp.cloudera.com
- Alternatively:
- In the Cloudera Management Console, click the Data Engineering tile.
- In the left navigation menu, click Administration.
- Right click on the Administration page and select Inspect from the list of options.
- Click the Console tab.
- Type
window.location.originand press Enter.
- You can obtain the
CLUSTER-IDTo obtain the
CLUSTER-ID, log in to the Cloudera Data Engineering UI and navigate to Administration in the left-navigation menu. Select the respective Service, then click on the Service Details icon, and copy the CLUSTER ID.
- Resolve the failed Jobs case-by-case.
