Cloudera Docs

Using Configuration Properties to Authenticate

To configure authentication with S3, explicitly declare the credentials in a configuration file such as core-site.xml.

The credential need to be added to the HDFS service in Cloudera Manager to correctly propagate them to the relevant services. You need to complete the following steps to configure the credentials in Cloudera Manager:
  1. Go to your cluster in Cloudera Manager.
  2. Select HDFS from the list of services.
  3. Select Configuration.
  4. Search for Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml in the search bar.
  5. Add the following properties and values to the Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml field:
    <property>
<name>fs.s3a.access.key</name>
<value>[***ACCESS KEY***]</value>
</property>

<property> 
<name>fs.s3a.secret.key</name>
<value>[***SECRET KEY***]</value>
</property>
    If using AWS session credentials for authentication, the secret key must be that of the session, and the fs.s3a.session.token option set to your session token.
    <property>
<name>fs.s3a.session.token</name>
<value>[***SESSION TOKEN***]</value> 
</property>
  6. Click Save changes.
  7. Restart the required services.

This configuration can be added for a specific bucket. For more information, see Using Per-Bucket Credentials to Authenticate.

To protect these credentials, we recommend that you use the credential provider framework to securely store and access your credentials.

To validate that you can successfully authenticate with S3, try referencing S3 in a URL.