Getting a Cloudera Data Engineering API access token

• Create a requirements.txt file specifying the Python package and version dependencies required by your Cloudera Data Engineering job.

Steps

1. Generate Cloudera Access Keys in User Management Console.
2. Generate DE workload auth token using Cloudera IAM API.

   The IAM API endpoint <CDP_ENDPOINT>/api/v1/iam/generateWorkloadAuthToken is called to generate a Cloudera Access Token. A Cloudera API call requires a request signature to be passed in the "x-altus-auth" header, along with a corresponding timestamp in the "x-altus-date" header. The cdpcurl library constructs the headers automatically. However, if you would rather use a different HTTP client, such as ordinary curl, you can use the cdpv1sign script within cdpcurl to generate these required headers.

   The request body contains workload-name as DE and is authenticated using the Cloudera Access Key. This request must also be signed as per the specification is available here either manually or use the cdpv1sign library to generate these necessary headers through automation script.

   curl -X POST '<CDP_ENDPOINT>/api/v1/iam/generateWorkloadAuthToken' \
   -H "Content-Type: application/json" \
   -H "x-altus-date: Tue, 15 Mar 2022 07:22:57 GMT" \
   -H "x-altus-auth: <signature-string-as-per-the-specification>" \
   -i --insecure \
   -d '{
      "workloadName": "DE"
   }'

3. The response will include the Cloudera Access Token in the token field and expiry time in the expireAt.

    {
       "token": "<token-string>",
       "expireAt": "2021-05-03T15:34:03.727000+00:00"
   }

4. Export the Cloudera token to CDP_TOKEN from above output.

   export CDP_TOKEN=<token-string>

5. Once you have a Cloudera access token CDP_TOKEN from the previous step, you can manually exchange it for a Cloudera Data Engineering access token.

   curl https://service.cde-czlmkz4y.na-01.xvp2-7p8o.cloudera.site/gateway/authtkn/knoxtoken/api/v1/token \
   -XPOST \
   -H "Accept: application/json" \
   -H "Authorization: Bearer ${CDP_TOKEN}" \
   -i --insecure

6. In the output, the access_token value is the JWT. For convenience, copy it and set it as an environment variable:

    export CDE_TOKEN=<access_token>

   Alternatively, you can set the token in a single step using jq to extract the token:.

   export CDE_TOKEN=$(curl <auth_endpoint> -XPOST \
   -H "Accept: application/json" \
   -H "Authorization: Bearer ${CDP_TOKEN}" \
   -i --insecure | jq -r '.access_token')