Verifying that S3Guard is Enabled on a Bucket
When S3Guard is working, apart from some messages in the logs, there is no obvious clue
that it is enabled. To verify that a bucket does have S3Guard enabled, use the command-line
command hadoop s3guard bucket-info
. This will print bucket information, and can
be used to explicitly check that a bucket has s3guard enabled.
hadoop s3guard bucket-info s3a://example1/ Filesystem s3a://example1 Location: eu-west-1 Filesystem s3a://example1 is using S3Guard with store DynamoDBMetadataStore{region=eu-west-1, tableName=example1, tableArn=arn:aws:dynamodb:eu-west-1:98067886600:table/example1} Authoritative Metadata Store: fs.s3a.metadatastore.authoritative=false Authoritative Path: fs.s3a.authoritative.path= Metadata Store Diagnostics: ARN=arn:aws:dynamodb:eu-west-1:98067886600:table/example1 billing-mode=per-request description=S3Guard metadata store in DynamoDB name=example1 persist.authoritative.bit=true read-capacity=0 region=eu-west-1 retryPolicy=ExponentialBackoffRetry(maxRetries=9, sleepTime=250 MILLISECONDS) size=14431 status=ACTIVE table={AttributeDefinitions: [{AttributeName: child,AttributeType: S}, {AttributeName: parent,AttributeType: S}],TableName: example1,KeySchema: [{AttributeName: parent,KeyType: HASH}, {AttributeName: child,KeyType: RANGE}],TableStatus: ACTIVE,CreationDateTime: Wed Jun 05 13:23:18 BST 2019,ProvisionedThroughput: {NumberOfDecreasesToday: 0,ReadCapacityUnits: 0,WriteCapacityUnits: 0},TableSizeBytes: 14431,ItemCount: 96,TableArn: arn:aws:dynamodb:eu-west-1:98067886600:table/example1,TableId: b4b1b660-9cbd-0000f10623,BillingModeSummary: {BillingMode: PAY_PER_REQUEST,LastUpdateToPayPerRequestDateTime: Wed Jun 05 13:32:07 BST 2019},} write-capacity=0 The "magic" committer is supported S3A Client Signing Algorithm: fs.s3a.signing-algorithm=(unset) Endpoint: fs.s3a.endpoint=s3-eu-west-1.amazonaws.com Encryption: fs.s3a.server-side-encryption-algorithm=none Input seek policy: fs.s3a.experimental.input.fadvise=normal Change Detection Source: fs.s3a.change.detection.source=etag Change Detection Mode: fs.s3a.change.detection.mode=server Delegation token support is disabled
In this example the bucket is shown to be guarded by a table whose billing mode is "per-request".
.