If you have less than 20 GB of RAM available for use by HBase, consider tailoring the default on-heap BlockCache implementation (LruBlockCache) for your cluster.
If you have more than 20 GB of RAM available, consider adding off-heap BlockCache (BucketCache).
The first few steps are the same for both options:
Specify the maximum amount of on-heap RAM to allocate to the HBase RegionServer on each node. The default is 1 GB, which is too small for production.
To alter the default allocation, set the "RegionServers maximum Java heap size" value (Ambari), or set the
HBASE_HEAPSIZE
environment variable inhbase-env.sh
(manual installation). Specify the value in megabytes. The HBase startup script uses$HBASE_HEAPSIZE
to override the default maximum JVM heap size (-Xmx
).The following example sets the maximum on-heap memory allocation to 20 GB in
hbase-env.sh
:export HBASE_HEAPSIZE=20480
Determine (or estimate) the proportions of reads and writes in your workload, and use these proportions to specify on-heap memory for BlockCache and MemStore. The sum of the two allocations must be less than or equal to 0.8. The following table describes the two properties.
Property
Default Value
Description
hfile.block.cache.size
0.4
Proportion of maximum JVM heap size (Java
-Xmx
setting) to allocate to BlockCache. A value of 0.4 allocates 40% of the maximum heap size.hbase.regionserver.global.memstore.upperLimit
0.4
Proportion of maximum JVM heap size (Java
-Xmx
setting) to allocate to MemStore. A value of 0.4 allocates 40% of the maximum heap size.Use the following guidelines to determine the two proportions:
The default configuration for each property is 0.4, which configures BlockCache for a mixed workload with roughly equal proportions of random reads and writes.
If your workload is read-heavy and you do not plan to configure off-heap cache -- your amount of available RAM is less than 20 GB -- increase
hfile.block.cache.size
and decreasehbase.regionserver.global.memstore.upperLimit
so that the values reflect your workload proportions. This will optimize read performance.If your workload is write-heavy, decrease
hfile.block.cache.size
and increasehbase.regionserver.global.memstore.upperLimit
proportionally.As noted earlier, the sum of
hfile.block.cache.size
andhbase.regionserver.global.memstore.upperLimit
must be less than or equal to 0.8 (80%) of the maximum Java heap size specified byHBASE_HEAPSIZE
(-Xmx
). If you allocate more than 0.8 across both caches, the HBase RegionServer process will return an error and will not start.Do not set
hfile.block.cache.size
to zero. At a minimum, specify a proportion that allocates enough space for HFile index blocks. To review index block sizes, use the RegionServer Web GUI for each server.
Edit the corresponding values in your
hbase-site.xml
file(s). Here are the default definitions:<property> <name>hfile.block.cache.size</name> <value>0.4</value> <description>Percentage of maximum heap (-Xmx setting) to allocate to block cache used by HFile/StoreFile. Default of 0.4 allocates 40%. </description> </property> <property> <name>hbase.regionserver.global.memstore.upperLimit</name> <value>0.4</value> <description>Maximum size of all memstores in a region server before new updates are blocked and flushes are forced. Defaults to 40% of heap. </description> </property>
If you have less than 20 GB of RAM for use by HBase, you are done with the configuration process. Restart (or rolling restart) your cluster. Check log files for error messages. If you have more than 20 GB of RAM for use by HBase, consider configuring the variables and properties in the next subsection.