Overview

Know the basics of HBase cache-aware load balancer.

In the stochastic load balancer, the region assignment is done based on multiple factors. However, this region assignment does not consider the amount of data already cached on a region server. During the region assignment, if the region is assigned to another region server, the data for this region that is already cached on the old region server has to be thrown away and it has to be prefetched afresh on the newly assigned region server.

HBase uses an ephemeral cache to cache the blocks by reading them from the slow storage and storing them in the bucket cache. This cache is warmed up every time a region server is started. Depending on the data size and the configured cache size, the cache warm-up activity can take anywhere from a few minutes to a few hours. Doing this every time the region server starts can be an expensive process. To eliminate this, HBase implemented the cache persistence feature where the region servers periodically persist the blocks cached in the bucket cache. This persisted information is then used to resurrect the cache in the event of a region server restart because of a normal restart or crash.

HBase implements the cache-aware load balancer which is aimed at enhancing the capability of HBase to enable the balancer to consider the cache allocation of each region on region servers when calculating a new assignment plan. This balancer uses the region or region server cache allocation information reported by region servers to calculate the percentage of HFiles cached for each region on the hosting server, and then use that as another factor when deciding on an optimal new assignment plan.

The master node captures the prefetch information from all the region servers. It uses this information to decide the region assignments while ensuring a minimal impact on the warm-up cache. A region is assigned to the region server where it has a better cache ratio as compared to the region server where it is currently hosted.