HBase Object Store Semantics overview

You can use Amazon S3 as a storage layer for HBase in a scenario where HFiles are written to S3, but WALs are written to HDFS.

The HBase Object Store Semantics(HBOSS) adapter bridges the gap between HBase, that assumes some file system operations are atomic, and object-store implementation of S3A filesystem, which does not provide atomic semantics to these operations natively.

HBOSS depends on S3Guard for accessing S3A buckets, so ensure given cluster and target AWS account fulfill all S3Guard requirements. The S3Guard feature guarantees a consistent view of data stored in Amazon S3 by storing additional metadata in a table residing in an Amazon DynamoDB instance. To enable S3Guard, you have to set up an Amazon DynamoDB database from Amazon Web Services. Amazon charges an hourly rate for this service based on the capacity you provision. See Amazon DynamoDB Pricing.

HBOSS relies on Zookeeper locks for emulating atomic access to file system tree nodes. It can be deployed to use the same Zookeeper service that is already accessed by other HBase features.