Replicating Directories with Thousands of Files and Subdirectories

Replicating data includes a directory with several hundred thousand files or subdirectories.

1. On the destination Cloudera Manager instance, go to the HDFS service page.
2. Click the Configuration tab.
3. Select Scope > HDFS service name (Service-Wide) and CategoryAdvanced.
4. Locate the HDFS Replication Advanced Configuration Snippet property.
5. Increase the heap size by adding a key-value pair, for instance, HADOOP_CLIENT_OPTS=-Xmx1g. In this example, 1g sets the heap size to 1 GB. This value should be adjusted depending on the number of files and directories being replicated.
6. Enter a Reason for change, and then click Save Changes to commit the changes.