1. Introduction to HBase High Availability

Architecturally, HBase has had a strong consistency guarantee from the start. All reads and writes are routed through a single Region Server, which guarantees that all writes happen in order, and all reads access the most recently committed data.

However, because of this "single homing" of reads to a single location, if the server becomes unavailable, the regions of the table that are hosted in the Region Server become unavailable for some time until they are recovered. There are three phases in the region recovery process: detection, assignment, and recovery. Of these, the detection phase is usually the longest, currently 20 to 30 seconds, depending on the Zookeeper session timeout setting when the Region Server fails but the Zookeeper session is running. After that, data is recovered from the Write Ahead Log and assigns the region to a different server. Until the recovery is complete, during this time clients cannot read data from that region.

For some use cases, the data may be read-only or stale data is acceptable. However, for use cases where latency is not acceptable, HBase can be used. To achieve high availability for reads, HBase provides a feature called region replication. When region replication is used each region of a table can be replicated and opened in different Region Servers. By default, the region replication is set to 1, so only a single region replica is deployed and there will not be any changes from the original model. If region replication is set to 2 or more, then the master assigns replicas of the regions of the table. The Load Balancer ensures that the region replicas are not co-hosted in the same Region Servers and in the same rack if possible.

All of the replicas for a single region have a unique replica ID, starting with 0. The region replica with replica ID = 0 is called the "primary region." The others are called “secondary region replicas,” or "secondaries." Only the primary region can accept writes from the client, and the primary always contains the latest changes. Since all writes must go through the primary region, writes are not highly available because they might be blocked for some time if the region becomes unavailable.

For example, in the following image Region Server 1 is responsible for responding to queries and scans for keys 10 through 40. If Region Server 1 crashes, the region holding keys 10-40 is unavailable for a short time until the region recovers.

HA provides a way to access keys 10-40 even if Region Server 1 is not available, by hosting replicas of the region and assigning the region replicas to other Region Servers as backups. In the following image, Region Server 2 hosts secondary region replicas for keys 10-20, and Region Server 3 hosts the secondary region replica for keys 20-40. Region Server 2 also hosts the secondary region replica for keys 80-100. There are no separate Region Server processes for secondary replicas. Rather, Region Servers can serve regions in primary or secondary mode. When Region Server 2 services queries and scans for keys 10-20, it acts in secondary mode.

[Note]Note

Regions acting in secondary mode are also known as Secondary Region Replicas. However, there is no separate Region Server process. A region in secondary mode can read but cannot write data. In addition, the data it returns may be stale, as described in the following section.

Timeline and Strong Data Consistency

HBase guarantees timeline consistency for all data served from Region Servers in secondary mode. This means that all HBase clients see the same data in the same order, but that data may be slightly stale. Only the primary Region Server is guaranteed to have the latest data. Timeline consistency simplifies the programming logic for complex HBase queries and provides lower latency than quorum-based consistency.

In contrast, strong data consistency means that the latest data is always served. However, strong data consistency can greatly increase latency in the case of a Region Server failure, because only the primary Region Server is guaranteed to have the latest data. The HBase API allows application developers to specify the data consistency that is required for a query.

[Note]Note

The HBase API contains a method called Result.isStale(), which specifies whether data returned in secondary mode can be stale, which means that the data has not been updated with the latest write operation to the primary Region Server.