Using Apache HBase to store and access data
Also available as:
PDF
loading table of contents...

Configure the size and number of WAL files

HBase uses the Write Ahead Log, or WAL, to recover MemStore data not yet flushed to disk if a RegionServer crashes. Administrators should configure these WAL files to be slightly smaller than the HDFS block size.

By default, an HDFS block is 64 MB and a WAL is approximately 60 MB. You should ensure that enough WAL files are allocated to contain the total capacity of the MemStores. Use the following formula to determine the number of WAL files needed:

(regionserver_heap_size * memstore fraction) / (default_WAL_size)

For example, assume that your environment has the following HBase cluster configuration:
  1. 16 GB RegionServer heap
  2. 0.4 MemStore fraction
  3. 60 MB default WAL size

The formula for this configuration is as follows:

(16384 MB * 0.4) / 60 MB = approximately 109 WAL files

Use the following properties in the hbase-site.xml configuration file to configure the size and number of WAL files:

Configuration Property

Description

Default

hbase.regionserver.maxlogs

Sets the maximum number of WAL files

32

hbase.regionserver.logroll.multiplier

Multiplier of HDFS block size

0.95

hbase.regionserver.hlog.blocksize

Optional override of HDFS block size

Value assigned to actual HDFS block size

Note
Note

If recovery from failure takes longer than expected, try reducing the number of WAL files to improve performance.