The S3A Committers and Third-Party Object Stores
The S3A committers will work with any object store which implements the AWS S3 protocols.
The directory committer requires a consistent cluster filesystem to propagate commit information from the worker processes to the job committer. This is normally done in HDFS.
If the third-party object store is consistent, then it may also be used as the cluster
fs.s3a.committer.staging.tmp.path to a chosen path in the