Cloudera Docs

cdpcurl

This documentation provides information about cdpcurl.

Building cdpcurl

You can access cdpcurl and review the license here.

  1. Download cdpcurl:
    wget https://github.com/cloudera/cdpcurl/archive/master.zip
unzip master.zip
cd cdpcurl-master
  2. If desired, create a virtual environment:

    For a typical virtual environment:

    
$ virtualenv cdpcurlenv
$ . cdpcurlenv/bin/activate

    For a python virtual environment:

    
$ pyenv virtualenv cdpcurlenv
$ pyenv activate cdpcurlenv
  3. In this directory, run the following command:
    $ pip install .

Generating an API access key

A Cloudera user account (a user or a machine user) must have API access credentials to access Cloudera services through the CDP CLI or API.

When you use this method to generate an access key and then manually configure the access key in the ~/.cdp/credentials, the access credentials are permanent until they are removed from the ~/.cdp/credentials file. A login command is not required if access credentials are stored in the ~/.cdp/credentials file. If you prefer that the API access key is shorter-lived, refer to the topic Logging into the CDP CLI/SDK, which describes a method of logging into the CLI/SDK through any SAML-compliant identity provider.

Required roles: Users who have the IAMUser role can generate an API access key from their own account page. As a Cloudera administrator or PowerUser, you can generate an API access key for all user accounts.

Generate your own access key

Steps

  1. Sign in to the Cloudera console.

  2. Click on your user name in the bottom left corner and then select Profile.
  3. On the user profile page that appears, click Generate Access Key.

  4. Cloudera creates the key and displays the information on the screen.

    Copy the access key and private key to a text file and send it to the Cloudera user who requires it. The private key is a very long string of characters. Make sure that you copy the full string. You can optionally download the credentials file containing the access key information.

  5. Click OK to exit the access key window.

Once you've generated the access key, you can configure CDP CLI, SDK, or other utilities that require it.

Generate an access key for another user or machine user (admin only)

Steps

  1. Sign in to the Cloudera console.

  2. From the Cloudera home page, click Cloudera Management Console.
  3. On the side navigation panel, click Users.
  4. On the Users page, click the name of the user or machine user account for which you want to generate an access key.
  5. On the user account page, go to the Access Keys section and click Generate Access Key.

  6. Cloudera creates the key and displays the information on the screen.

    Copy the access key and private key to a text file and send it to the Cloudera user who requires it. The private key is a very long string of characters. Make sure that you copy the full string. You can optionally download the credentials file containing the access key information.

  7. Click OK to exit the access key window.

Once you've generated the access key, you can configure CDP CLI, SDK, or other utilities that require it.

Passing the API access keys to cdpcurl

After you generate the API access key and private key, you must pass the keys to cdpcurl.

You have two options for passing the API access key and private key to cdpcurl:

  • Recommended method: Create a profile in $HOME/.cdp/credentials that contains the keys, and then use the --profile option in cdpcurl calls. For example:
    [myuserprofile]
cdp_access_key_id = 6744f22e-c46a-406d-ad28-987584f45351
cdp_private_key = abcdefgh...................................=
  • Alternatively, when you make the API call, pass the keys to cdpcurl using the --access-key and --private-key options.

Making API calls with cdpcurl

To use cdpcurl to make API calls, first form the correct URI using the API reference documentation, and then construct the request.

To form the required URI, start by determining the hostname based on the service being called:

  • iam:
    • iamapi.us-west-1.altus.cloudera.com if your Cloudera account runs in Cloudera Control Plane region us-west-1
    • api.<control-plane-region>.cdp.cloudera.com if your Cloudera account runs in any other Cloudera Control Plane region
  • All other services: api.<control-plane-region>.cdp.cloudera.com
Then form the URI. The correct URI is an HTTPS URL at the chosen host, with a path indicated for your desired endpoint in the API documentation. For example, the URI for a listEnvironments call would look something like this:
https://api.us-west-1.cdp.cloudera.com/api/v1/environments2/listEnvironments

Most Cloudera API calls are POST requests, so be sure to use the [-X REQUEST] and [-d DATA] arguments in the command. These are briefly documented in the awscurl documentation, but [-X REQUEST] specifies the type of request (most often POST in Cloudera) and [-d DATA] specifies the request content (in the POST example below, an empty JSON request). If the -d option value begins with "@", then the remainder of the value is the path to a file containing the content; otherwise, the value is the content itself.

You can then run the request. For example:
$ cdpcurl --profile sandbox -X POST -d '{}' https://api.us-west-1.cdp.cloudera.com/api/v1/environments2/listEnvironments

For a complete list of cdpcurl options, run cdpcurl --help.

cdpcurl example

After you install cdpcurl and generate the API access key, you are ready to make an API call.

If you have not created a profile in $HOME/.cdp/credentials that contains the API access key ID and private key pair, make sure that you have the API access key ID and private key that you generated.
The example below walks you through the steps required to make an API call that lists all environments in the CDP Control Plane.
  1. In the CDP Control Plane Public API Documentation, find the endpoint for the list environments request and form the correct URI.
    The correct URI is an HTTPS URL that contains the hostname for the desired service and the path to the desired endpoint. In this "list environments" example, the correct URI for a CDP account running in the us-west-1 Control Plane region would be: 
    https://api.us-west-1.cdp.cloudera.com/api/v1/environments2/listEnvironments
  2. Form the cdpcurl command.

    The CDP API uses the HTTPS protocol to make API calls, and most CDP API calls are POST requests, so be sure to use the [-X REQUEST] and [-d DATA] arguments in the command. For more details, refer to the awscurl documentation and the topic Making API calls with cdpcurl. The --profile argument below points to the profile you likely created in $HOME/.cdp/credentials, that contains the API access key and private key pair. If you did not create this profile, you will have to pass the --access_key and --private_key arguments as well.

    In this example, the command would be:

    $ cdpcurl --profile sandbox -X POST -d '{}' https://api.us-west-1.cdp.cloudera.com/api/v1/environments2/listEnvironments
  3. Run this command in a terminal window to make the API call, and then receive the response. If the call is successful, the response is the EnvironmentSummary array.