Off-heap BucketCache
If the BucketCache is enabled, it stores data blocks, leaving the on-heap cache free
for storing indexes and Bloom filters. The physical location of the BucketCache
storage can be either in memory
(off-heap) or in a file stored in a fast disk.
- Off-heap: This is the default configuration.
-
File-based: You can use the file-based storage mode
to store the
BucketCache
on an SSD or FusionIO device,
You can
configure a column family to keep its data blocks in the L1 cache instead of the
BucketCache, using the
HColumnDescriptor.cacheDataInL1(true)
method or by using the following syntax
in HBase
Shell:hbase> alter 'myTable', CONFIGURATION => {CACHE_DATA_IN_L1 => 'true'}}
This table summaries the important configuration properties for the
BucketCache
. To configure the BucketCache
. The table
is followed by three diagrams that show the impacts of different blockcache settings.Property | Default | Description |
---|---|---|
hbase.bucketcache.combinedcache.enabled |
true |
When BucketCache is enabled, use it as a L2 cache for
LruBlockCache . If set to true, indexes and Bloom filters
are kept in the LruBlockCache and the data blocks are kept
in the BucketCache . |
hbase.bucketcache.ioengine |
none (BucketCache is disabled by default) |
Where to store the contents of the BucketCache . Its
value can be offheap ,
file:PATH ,
mmap:PATH or
pmem:PATH where PATH is the path
to the file that host the file-based cache. |
hfile.block.cache.size |
0.4 | A float between 0.0 and 1.0. This factor multiplied by the Java heap size is the size of the L1 cache. In other words, the percentage of the Java heap to use for the L1 cache. |
hbase.bucketcache.size |
not set | When using BucketCache , this is a float that represents
one of two different values, depending on whether it is a
floating-point decimal less than 1.0 or an integer greater than 1.0.
|
hbase.bucketcache.bucket.sizes |
4, 8, 16, 32, 40, 48, 56, 64, 96, 128, 192, 256, 384,
512 KB |
A comma-separated list of sizes for buckets for the
BucketCache if you prefer to use multiple sizes. The
sizes should be multiples of the default blocksize, ordered from smallest to
largest. The sizes you use will depend on your data patterns. This
parameter is experimental. |
-XX:MaxDirectMemorySize |
MaxDirectMemorySize = BucketCache + 1 |
A JVM option to configure the maximum amount of direct memory available
for the JVM. It is automatically calculated and configured based on the
following formula: MaxDirectMemorySize = BucketCache size + 1
GB for other features using direct memory, such as DFSClient.
For example, if the BucketCache size is 8 GB, it will be
-XX:MaxDirectMemorySize=9G . |