Using the S3Guard CLI
The S3Guard CLI offers other maintenance commands. for information on how to use them, refer toApache documentation.
One useful command to use is
hadoop s3guard prune -tombstone, which
removes "tombstone markers" from the table. These are entries created when files are
deleted, so as to identify recently deleted files which should be omitted from listings.
After the S3 Store has become consistent with these deletions, the markers are no longer
deleted.> hadoop s3guard prune -tombstone -days 1 s3a://hwdev-steve-ireland-new/
2019-08-20 16:55:40,790 [main] INFO s3guard.S3GuardTool - Metadata store DynamoDBMetadataStore{region=eu-west-1, tableName=example1, tableArn=arn:aws:dynamodb:eu-west-1:98067886600:table/example1} is initialized.
2019-08-20 16:55:40,810 [main] INFO s3guard.DynamoDBMetadataStore - Starting: Pruning DynamoDB Store
2019-08-20 16:55:40,846 [main] INFO s3guard.DynamoDBMetadataStore - Pruning DynamoDB Store: duration 0:00.036s
2019-08-20 16:55:40,846 [main] INFO s3guard.DynamoDBMetadataStore - Finished pruning 0 items in batches of 25
