Cloudera Docs

Monitoring the performance of HDFS replication policies

You can monitor the progress of an HDFS replication policy using the performance data that you can download as a CSV file from the Cloudera Manager Admin console.

The performance report contains information about the files being replicated, the average throughput, and other details that can help diagnose performance issues during HDFS replications. You can view this performance data for running HDFS replication jobs and for completed jobs. The performance data is collected every two minutes. Therefore, no data is available during the initial execution of a replication job because not enough samples are available to estimate throughput and other reported data.

To view the performance data for a running HDFS replication policy, perform the following steps:

  1. Go to the Cloudera Manager > Replication > Replication Policies page.
  2. Locate and select the replication policy. Click Actions > Show History.
  3. Click Download CSV for the HDFS Replication Report field, and choose one of the following options to download the following performance reports:
    • Performance file contains a summary report about the performance of the replication job which includes the last performance sample for each mapper working on the replication job.
    • Full Performance file contains the complete performance report about the job which includes all the samples taken for all mappers during the full run of the replication job.
  4. Open the file in a spreadsheet program such as Microsoft Excel.
    The following columns appear in the CSV file:
    • Timestamp when the performance data was collected.
    • Host where the YARN or MapReduce job was running.
    • Number of Bytes Copied for the file currently being copied.
    • Time Elapsed (ms) for the copy operation of the file currently being copied.
    • Number of Files Copied.
    • Avg Throughput (KB/s) since the start of the file currently being copied in kilobytes per second.
    • File size of the Last File (bytes).
    • Time taken to copy Last File Time (ms).
    • Last file throughput (KB/s) that is being copied in kilobytes per second.
  5. Download the following CSV reports to view more information about the replication job:
    • Listing report contains the list of files and directories copied during the replication job.
    • Status report contains the full status report of the files where the replication status is shown as:
      • ERROR occurred during replication, therefore the file was not copied.
      • DELETED for deleted files.
      • SKIPPED for up-to-date files that were not replicated.
    • Error Status Only report contains the status report of all copied files with errors. The file lists the status, path, and message for the copied files with errors.
    • Deleted Status Only report contains the status report of all deleted files. The file lists the status, path, and message for the databases and tables that were deleted.
    • Skipped Status Only report contains the status report of all skipped files. The file lists the status, path, and message for the databases and tables that were skipped.
Note the following limitations and known issues about the replication reports:
  • If you click the CSV download too soon after the replication job starts, Cloudera Manager returns an empty file or a CSV file that has columns headers only and a message to try later when performance data has actually been collected.
  • If you employ a proxy user with the form user@domain, performance data is not available through the links.
  • If the replication job only replicates small files that can be transferred in less than a few minutes, no performance statistics are collected.
  • If you specify the Dynamic Replication Strategy during replication policy creation, statistics regarding the last file transferred by a MapReduce job hide previous transfers performed by that MapReduce job.
  • Only the last trace per MapReduce job is reported in the CSV file.