Cloudera Docs

Cloudera Data Engineering CLI authentication

The Cloudera Data Engineering (CDE) CLI tool supports both interactive and transparent authentication. For interactive authentication, if you have configured the CLI with your workload username, you are prompted for a password. For transparent authentication, the CDE CLI supports a password file, Cloudera Data Platform (CDP) access keys, and CDP credentials file.

The CDE CLI provides the following mechanisms for authentication:

  • CDP access key stored in a credentials file
  • CDP access key specified by CLI flag or environment variable
  • Interactive prompt for workload password
  • Workload password specified by CLI flag or environment variable

In all cases, the CLI uses the provided credentials to obtain an authentication token for the specified user, and caches it locally in a file on the machine where the CLI is running. You can disable caching of tokens entirely by using the --auth-no-cache CLI flag or the CDE_AUTH_NO_CACHE environment variable.

The cache file location is automatically determined based on the default system user cache:

  • Linux: $HOME/.cache/cloudera/cde or $XDG_CACHE_HOME/cloudera/cde/
  • macOS: $HOME/Library/Caches/cloudera/cde/
  • Windows: %LocalAppData%\cloudera\cde\

If you want to use a custom location, specify it with the --auth-cache-file flag or the CDE_AUTH_CACHE_FILE environment variable. You can use the special string $USERCACHE, which is expanded according to the default system user cache (as listed above, without the /cloudera/cde/ suffix).

CDP credentials file

When you generate a CDP access key, you can download it to a credentials file:

The access key is only displayed and available for download when you first generate it. After you close the dialog, there is no way to recover the key.

Save or copy the credentials file to $HOME/.cdp/credentials on the machine where you are running the CDE CLI. Credentials stored in this file are automatically discovered by both the CDE and CDP CLIs. If a credentials file is found, authentication occurs transparently using the discovered CDP access key.

The CDE CLI automatically looks for a CDP access key in the following locations in the order given:
  1. ./credentials
  2. $HOME/.cde/credentials
  3. /etc/cde/credentials
  4. $HOME/.cdp/credentials

You can override this by using the --credentials-file </path/to/credentials> CLI flag to specify a different file location.

You can also skip credential discovery by using the --skip-credentials-file flag.

CDP access key

If you do not want to use the credentials file, you can specify the access key using environment variables or command line flags as follows:

Table 1. CDP access key environment variables and CLI flags
Parameter Environment variable CLI flag
Access key ID CDE_ACCESS_KEY_ID=<access_key_id> --access-key-id <access_key_id>
Access key secret CDE_ACCESS_KEY_SECRET=<access_key_secret> --access-key-secret string <access_key_secret>

Along with the above flags, CDE CLI expects CDP endpoint URL to be configured. CDP Endpoint URL is same as the CDP private cloud console URL. You can configure this using environment variables or command line flags as follows:

Table 2. CDP endpoint environment variables and CLI flags
Parameter Environment variable CLI flag
CDP Endpoint CDE_CDP_ENDPOINT=<cdp_endpoint> --cdp-endpoint <cdp_endpoint>

Workload password prompt

When the CLI requires a new token for a virtual cluster, you are prompted for the password for the workload user, identified by the --user CLI flag or the CDE_USER environment variable.

The workload password, for both human and machine users, can be set using the CDP User Management console. For more information, see Managing user access and authorization.

Workload password file

If you do not want to be prompted for your workload password, you can provide a password file. A password file is a file containing your workload password, and nothing else.

You can specify the password file by using an environment variable or a command line flag as follows:

Environment variable
CDE_AUTH_PASS_FILE=</path/to/password/file>
Command line flag
--auth-pass-file </path/to/password/file>