Recommended configurations for the Balancer
The HDFS Balancer can run in either Background or Fast modes. Depending on the mode in which you want the Balancer to run, you can set various properties to recommended values.
Background and Fast Modes
-
HDFS Balancer runs as a background process. The cluster serves other jobs and applications at the same time.
- Fast Mode
-
HDFS Balancer runs at maximum (fast) speed.
| Property | Default | Background Mode | Fast Mode |
|---|---|---|---|
|
dfs.datanode.balance.- max.concurrent.moves |
5 | 4 x (# of disks) | 4 x (# of disks) |
|
dfs.datanode.balance.- max.bandwidthPerSec |
1048576 (1 MB) | use default | 10737418240 (10 GB) |
| Property | Default | Background Mode | Fast Mode |
|---|---|---|---|
|
dfs.datanode.balance.- max.concurrent.moves |
5 | # of disks | 4 x (# of disks) |
|
dfs.balancer.- moverThreads |
1000 | use default | 20,000 |
|
dfs.balancer.- max-size-to-move |
10737418240 (10 GB) | 1073741824 (1GB) | 107374182400 (100 GB) |
|
dfs.balancer.- getBlocks.min-block-size |
10485760 (10 MB) | use default | 104857600 (100 MB) |