Choosing a DynamoDB Table and IO Capacity
Amazon's DynamoDB service bills its customers for allocated writes per second ("Write Capacity"), reads per second ("Read Capacity") and storage, which is billed by the gigabyte.
S3Guard uses DynamoDB read capacity when it is listing directories or querying file status; write capacity when it is updating the database after querying S3 or when a client using S3Guard creates, updates or deletes objects in S3. The more allocated capacity, the more throughput clients using S3Guard can get when interacting with the S3Guard database, but the higher the monthly costs —even when the database is not being used.
S3Guard only stores filenames and other metadata in its tables: for each directory only the name is stored; for a file the size and timestamp are preserved; storage costs should be low. It is the amount of provisioned IO capacity which can be the main expense for using S3Guard.
There are three strategies to manage IO capacity:
Dedicate a single table to a bucket; choose read and write capacity based on expected use. This isolates the table for both capacity and security. When a bucket is not being used, you are still billed for the allocated capacity.
Share a single table with all buckets in a single AWS region; choose read and write capacity based on expected average use across all buckets. This balances capacity across buckets. However, it gives all clients the ability to this the contents of all buckets in the table.
Either of the two previous strategies, but using DynamoDB Auto Scaling to manage the capacity. This can keep costs down when idle. yet allow it to scale up to handle higher load.
If using allocated capacity, we would advise using a shared table; when using auto scaling, the main disadvantage of a single table is lost, and because of the security isolation, it is the policy we recommend.
If using autoscaling, be aware that when the scaling up/down takes place, the table may briefly switch into the UPDATING state. Existing connections to the database should continue, but trying to start new connections will fail with an error message that the table "did not transition into ACTIVE state.". This should be a transient failure; after DynamoDB has marked the table as active, new work can be executed against it.