When working correctly, the only sign the new committers are in use is that it should be faster to use S3 as a destination of work.

There is an straightforward way to determine if a new committer was used: examine the _SUCCESS file created in the destination directory of a query. With the original file committer, this is a zero-byte file. The new S3A committers all write a JSON file describing the committer used, the files created and various diagnostics information.

Listing this file is enough to show whether an S3A committer was used:

hadoop fs -ls s3a://guarded-bucket/datasets/orc/_SUCCESS

If this file is of size 0. then no S3A committer was used. If the file length is greater than zero, then an S3A committer was used.

To see more details about the job commit operation, the file's contents can be printed.

hadoop fs -cat s3a://guarded-bucket/datasets/orc/_SUCCESS