Prerequisites to configure Off-Heap Memory (BucketCache)
Note | |
---|---|
Before configuring off-heap memory, complete the tasks in the previous "Configuring BlockCache" section. |
To prepare for BucketCache configuration, compare the figure and table below before proceeding to the "Configuring BucketCache" steps.
In the following table:
-
The first column refers to the elements in the figure.
-
The second column describes each element and, if applicable, its associated variable or property name.
-
The third column contains values and formulas.
-
The fourth column computes values based on the following sample configuration parameters:
-
128 GB for the RegionServer process (there is additional memory available for other HDP processes)
-
A workload of 75% reads, 25% writes
-
HBASE_HEAPSIZE
= 20 GB (20480 MB)
-
Note | |
---|---|
Most of the following values are specified in megabytes; three are proportions. |
Item |
Description |
Value or Formula |
Example |
A |
Total physical memory for RegionServer operations: on-heap plus off-heap ("direct") memory (MB) |
(hardware dependent) |
131072 |
B |
The This value was set when the BlockCache was configured. |
Recommendation: 20480 |
20480 |
C |
The -XX: MaxDirectMemorySize option: Amount of off-heap
("direct") memory to allocate to HBase (MB) |
A - B |
131072 - 20480 = 110592 |
Dp |
The This value was set when the BlockCache was configured. |
(proportion of reads) * 0.8 |
0.75 * 0.8 = 0.6 |
Dm |
Maximum amount of JVM heap to allocate to BlockCache (MB) |
B * Dp |
20480 * 0.6 = 12288 |
Ep |
The |
0.8 - Dp |
0.8 - 0.6 = 0.2 |
F |
Amount of off-heap memory to reserve for other uses (DFSClient; MB) |
Recommendation: 1024 to 2048 |
2048 |
G |
Amount of off-heap memory to allocate to BucketCache (MB) |
C - F |
110592 - 2048 = 108544 |
The |
G |
108544 |