Using Per-Bucket Credentials to Authenticate
S3A supports per-bucket configuration, which can be used to declare different authentication credentials and authentication mechanisms for different buckets.
For example, a bucket s3a://nightly/
used for nightly data can be
configured with a session key:
<property> <name>fs.s3a.bucket.nightly.access.key</name> <value>AKAACCESSKEY-2</value> </property> <property> <name>fs.s3a.bucket.nightly.secret.key</name> <value>SESSIONSECRETKEY</value> </property>
Similarly, you can set a session token for a specific bucket:
<property> <name>fs.s3a.bucket.nightly.session.token</name> <value>SESSION-TOKEN</value> </property>
This technique is useful for working with external sources of data, or when copying data between buckets belonging to different accounts.
Also see Customizing Per-Bucket Secrets Held in Credential Files.