S3A supports per-bucket configuration, which can be used to declare different authentication credentials and authentication mechanisms for different buckets.

For example, a bucket s3a://nightly/ used for nightly data can be configured with a session key:

<property>
  <name>fs.s3a.bucket.nightly.access.key</name>
  <value>AKAACCESSKEY-2</value>
</property>

<property>
  <name>fs.s3a.bucket.nightly.secret.key</name>
  <value>SESSIONSECRETKEY</value>
</property>

Similarly, you can set a session token for a specific bucket:

<property>
  <name>fs.s3a.bucket.nightly.session.token</name>
  <value>SESSION-TOKEN</value>
</property>

This technique is useful for working with external sources of data, or when copying data between buckets belonging to different accounts.

Related Links

Customizing Per-Bucket Secrets Held in Credential Files