Disabling S3Guard and Destroying a S3Guard Database

You can disable S3Guard and destroy a S3Guard database.

  1. Delete the per-bucket fs.s3a.metadatastore.impl parameter or set it back to the default org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore.
  2. If you are not using a shared table, in DynamoDB console on AWS, delete the DynamoDB table to avoid incurring unnecessary costs.

    The command hadoop s3guard destroy can be used on the command line to destroy a table.

    > hadoop s3guard destroy s3a://guarded-table/
                2019-07-31 20:48:18,448 [main] INFO  s3guard.S3GuardTool (S3GuardTool.java:initMetadataStore(321)) - Metadata store DynamoDBMetadataStore{region=eu-west-2, tableName=s3guard-metadata, tableArn=arn:aws:dynamodb:eu-west-2:980678866538:table/s3guard-metadata} is initialized.
                2019-07-31 20:48:18,450 [main] INFO  s3guard.DynamoDBMetadataStore (DynamoDBMetadataStore.java:destroy(1416)) - Deleting DynamoDB table s3guard-metadata in region eu-west-2
                Metadata store is deleted

    Destroying a table does not destroy the data in S3; it merely removes the summary data used by S3Guard to provide consistent listings.We can verify this by listing the bucket:

    > hadoop fs -ls -R  s3a://guarded-table/
                -rw-rw-rw-   1 alice alice          0 2019-07-31 20:45 s3a://guarded-table/example