Restoring Cloudera Data Engineering jobs from backup

You can restore Cloudera Data Engineering (CDE) jobs and associated resources from a backup using the API. Backups are zip files that can be used to restore jobs and their associated resources. Currently, backup and restore operations are supported using the API only.

You must have a valid backup file to restore from. For instructions on backing up CDE jobs, see Backing up Cloudera Data Engineering jobs.

Request an access token and save it as an environment variable to use in API calls. For instructions, see Getting a Cloudera Data Engineering API access token.

  1. Determine the API URL for the virtual cluster containing the job you want to back up:
    1. Navigate to the Cloudera Data Engineering Overview page.
    2. In the Environments column, select the environment containing the virtual cluster you want to interact with using the API.
    3. In the Virtual Clusters column on the right, click the Cluster Details icon for the virtual cluster containing the job to back up.
    4. Copy the URL under JOBS API URL, and set it as an environment variable.
      For example:
      export CDE_JOBS_API="https://pmjkrgn5.cde-czlmkz4y.na-01.xvp2-7p8o.cloudera.site/dex/api/v1"
  2. Restore jobs from the backup file by uploading the backup file to the /admin/import endpoint.
    You can choose how to handle duplicate job names using the duplicatehandling=<policy> parameter. Options are:
    error
    Return an error if there are duplicate job names, and abort the restore operation.
    rename
    If a job name in the backup conflicts with an existing job, keep the existing job and rename the restored job by appending a numerical identifier to the job name.
    keep-original
    If a job name in the backup conflicts with an existing job, keep the existing job and do not restore the backed up job.
    For example, to restore a backup named cde-etl-jobs-backup.zip using the rename duplicate handling policy:
    curl -k \
    -H "Authorization: Bearer ${CDE_TOKEN}" \
    -X POST "${CDE_JOBS_API}/admin/import \
    -H "accept: application/json" \
    -H "Content-Type: multipart/form-data" \
    -F "file=@/path/to/cde-etl-jobs-backup.zip;type=application/zip" \
    -F duplicatehandling=rename
    | jq