Cloudera Docs

Configuring cloud credentials using HashiCorp Vault

Configure Knox IDBroker with HashiCorp Vault to securely manage AWS credentials. This process requires retrieving short-lived tokens, enhancing security, and eliminating long-term credential storage.

Knox IDBroker can be configured with HashiCorp Vault to enhance AWS credentials management. HashiCorp Vault enables Knox to retrieve short-lived AWS credentials from HashiCorp Vault, improving security and eliminating the need for long-term credential storage.

  • HashiCorp Vault must be installed and properly configured.
  • An AWS account must be available with the required IAM roles.
  • Network connectivity must be established between Knox and the HashiCorp Vault server.
  1. Go to Cloudera Manager > Knox > Configuration.
  2. Search for the Save Alias Command Input - IDBroker property.
  3. Enter the following parameter: 
    [***TOPOLOGY NAME***].vaultTokenAlias=[***VAULT TOKEN***]
  4. Click Save Changes(CTRL+S).
  5. Select the Save Alias - IDBroker action from the Actions drop-down list.
  6. Search for the Knox IDBroker Advanced Configuration Snippet (Safety Valve) for conf/cdp-resources.xml advanced configuration snippet.
  7. Click the + icon and add the following entries:
    • Name= [***TOPOLOGY NAME***]
    • Value=
      providerConfigRef=cab-providers\#IDBROKER:cloud.policy.config.provider=default\#IDBROKER:cloud.client.provider=AWS\#IDBROKER:hashicorp.vault.enabled=true\#IDBROKER:hashicorp.vault.address=[***VAULT ADDRESS***]\#IDBROKER:hashicorp.vault.path=aws/sts\#IDBROKER:hashicorp.vault.server.cert.path=[***VAULT CERTIFICATE PATH***]
      The value string includes the following parameters:
      • hashicorp.vault.enabled=true – Enables the vault integration for that specific topology.
      • hashicorp.vault.address=[***VAULT ADDRESS***] – Replace [***VAULT ADDRESS***] with the actual address of the vault.
      • hashicorp.vault.path=aws/sts –Specifies the path for the STS credentials.
      • (Optional) hashicorp.vault.server.cert.path=[***VAULT CERTIFICATE PATH***] – Adds the path for the vault certificate. Required in case of self-signed certificates. Replace [***VAULT CERTIFICATE PATH***] with the path to the certificate.
  8. Click Save Changes(CTRL+S).
  9. Search for the Knox IDBroker AWS User Mapping property.
  10. Enter the following parameter: 
    rangerraz=arn:aws:iam::<ARN>:role/S3RazRole
  11. Refresh the Knox instance configuration by clicking the Stale Configuration: Refresh needed indicator and wait until the refresh process completes.
  12. Verify that the vault integration was successful by running the following command in the Command Line Interface (CLI): 
    kinit rangerraz
hdfs dfs -ls s3a://[***RESOURCE***]