Using Per-Bucket Credentials to Authenticate

S3A supports per-bucket configuration, which can be used to declare different authentication credentials and authentication mechanisms for different buckets. For example, a bucket s3a://nightly/ used for nightly data can be configured with a session key.

<property> 
          <name>fs.s3a.bucket.nightly.access.key</name> 
          <value>AKAACCESSKEY-2</value> 
          </property> 
          
          <property> 
          <name>fs.s3a.bucket.nightly.secret.key</name> 
          <value>SESSIONSECRETKEY</value> 
          </property>

Similarly, you can set a session token for a specific bucket:

<property> 
          <name>fs.s3a.bucket.nightly.session.token</name> 
          <value>SESSION-TOKEN</value> 
          </property>

This technique is useful for working with external sources of data, or when copying data between buckets belonging to different accounts.