Advanced erasure coding configuration
You can customize the behavior of EC through a combination of the
ec subcommand and the Cloudera Manager Admin
This procedure provides information about configuring certain properties for EC with the Cloudera Manager Admin Console.For information about the
hdfs ecsee Erasure coding CLI command.
- Select Clusters and choose the HDFS cluster you want to configure.
- Navigate to the Configuration tab and select the Erasure Coding category.
- Configure the advanced EC properties.
- DataNode Striped Read Timeout: The timeout for striped reads during background data reconstruction in milliseconds.
- DataNode Striped Read Threads: The number of threads that a DataNode can use during background data reconstruction.
- Erasure Coding Reconstruction Weight: The relative weight of
resources used by EC for data recovery. The number of blocks that must be read is
based on the EC policy used. For example, RS-6-3-1024k requires six blocks to be read.
Replication only requires one block to be read. Higher values result in fewer
reconstruction tasks being able to run concurrently. The number of blocks required to
be read to recover data is multiplied by this weight to determine the total weight of
the recovery task. The total weight of the recovery task counts against the limit set
- Fallback Erasure Coding Policy: The fallback Erasure Coding
policy that HDFS uses if no policy is specified when you run the