Disabling S3Guard and Destroying a S3Guard Database
If you decide to disable S3Guard:
- Delete the per-bucket fs.s3a.metadatastore.impl parameter or set it back to the default "org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore".
- If you are not using a shared table, in DynamoDB console on AWS, delete the DynamoDB table to avoid incurring unnecessary costs.
The command hadoop s3guard destroy can be used on the command line to destroy a table.
> hadoop s3guard destroy s3a://guarded-table/ 2019-07-31 20:48:18,448 [main] INFO s3guard.S3GuardTool (S3GuardTool.java:initMetadataStore(321)) - Metadata store DynamoDBMetadataStore{region=eu-west-2, tableName=s3guard-metadata, tableArn=arn:aws:dynamodb:eu-west-2:980678866538:table/s3guard-metadata} is initialized. 2019-07-31 20:48:18,450 [main] INFO s3guard.DynamoDBMetadataStore (DynamoDBMetadataStore.java:destroy(1416)) - Deleting DynamoDB table s3guard-metadata in region eu-west-2 Metadata store is deleted
Destroying a table does not destroy the data in S3; it merely removes the summary data used by S3Guard to provide consistent listings.We can verify this by listing the bucket:
> hadoop fs -ls -R s3a://guarded-table/ -rw-rw-rw- 1 alice alice 0 2019-07-31 20:45 s3a://guarded-table/example