HBase Replication Internals

The state of HBase replication is contained in the base node /hbase/replication. The peers znode contains a list of all peer replication clusters with their respective statuses. The rs znode contains a list of WAL logs to be replicated.

Replication State in ZooKeeper

HBase replication maintains its state in ZooKeeper. By default, the state is contained in the base node /hbase/replication. This node contains two child nodes, the Peers znode and the RS znode.

The Peers Znode

The peers znode is stored in /hbase/replication/peers by default. It consists of a list of all peer replication clusters along with the status of each of them. The value of each peer is its cluster key, which is provided in the HBase shell. The cluster key contains a list of ZooKeeper nodes in the cluster quorum, the client port for the ZooKeeper quorum, and the base znode for HBase in HDFS on that cluster.

The RS Znode

The rs znode contains a list of WAL logs which need to be replicated. This list is divided into a set of queues organized by Region Server and the peer cluster that the RegionServer is shipping the logs to. The rs znode has one child znode for each RegionServer in the cluster. The child znode name is the RegionServer hostname, client port, and start code. This list includes both live and dead RegionServers.