Cloudera Docs

Managing Ozone replication policies

After you create an Ozone replication policy, you can perform and monitor various tasks related to the replication policy. You can view the job progress and replication logs. You can edit the advanced options to optimize a job run. You can suspend a job and also activate a suspended job.

  1. On the Replication Policies page, you can view the following details about the replication policy:
    1. Shows a row of information for each replication policy, and the following columns for each replication policy:
      • Internally generated ID for the replication policy. Click the column label to sort the replication policies.
      • Replication policy Name that you provide during replication policy creation.
      • Replication policy Type.
      • Source cluster in the replication policy.
      • Destination cluster in the replication policy.
      • Average Throughput per mapper/file for all the files written.
      • Replication job Progress.
      • Timestamp when the replication job Completed.
      • Replication policy job’s Next Run.
    2. Provides the following options under the Actions menu:
      • Show History opens the Replication History page for the replication policy.
      • Edit Configuration allows you to change the replication policy options as required.
      • Dry Run simulates a run of the replication job where no files or tables are copied.

        After the dry run completes, select Actions > Show History to view the potential error messages and the number and size of files or tables that would be copied in an actual replication appears on the Replication History page.

      • Run Now initiates a replication job.
      • Collect Diagnostic Data opens the Send Diagnostic Data dialog box where you can:
        • Collect Diagnostic Data for the last 10 runs of the replication policy, and Download it as a ZIP file to your machine.
        • Select Send Diagnostic Data to Cloudera (optionally, add a Cloudera support ticket number and comments) and click Collect Diagnostic Data to automatically send the bundle to Cloudera Support for further assistance.
      • Disable an active replication policy.

        You can Enable it later, as necessary.

      • Delete the replication policy permanently. Deleting a replication policy does not delete copied files or tables.
  2. On the Replication History page, you can view the following details about a replication policy job:
    1. Shows the replication policy Name; replication policy Type; Source cluster name; Destination cluster name; and Next Run of the replication policy.
    2. Shows a row of information for each replication policy job run, and the following columns for each replication policy:
      Column Description
      Duration Time taken for the replication job to complete.
      Outcome Status of the replication job as In progress, Successful, or Failed.
      Files Expected Number of files expected to be copied and its file size based on the parameters of the replication policy.
      Files Copied Number of files copied and its file size for the replication job.
      Files Failed Number of files that failed to be copied and its file size for the replication job.
      Files Deleted Number of files that were deleted and its file size for the replication job
      Files Skipped Number of files skipped and its file size for the replication job. The replication process skips files that already exist in the destination and have not changed.
    3. Expand a job to view the following information:
      • Replication job Started At timestamp.
      • Duration to complete the job.
      • Command Details appear in a new tab after you click View.

        The All Recent Commands page shows the job Status; Context (click to view the service status page); Started At timestamp; Duration to complete the job run; and Download the job run command summary as a JSON file to your local machine.

        The page can also Show All Steps; Show Only Failed Steps; or Show only Running Steps of the replication policy job run commands with stdout and stderr output. Click Full Log file to view the logs in a new browser tab.

        For more information, see Viewing Running and Recent Commands.

      • Click MapReduce Job ID to view more details about the job on the YARN service page.
      • Download CSV files of the following Ozone Replication Reports to track the replication jobs and to troubleshoot issues:
        Report Description
        Listing Lists all the files and directories copied during the replication job.
        Status Shows the following status for each file as:
        • an Error occurred and the file was not copied.
        • a Deleted file.
        • an up-to-date file for which the replication was Skipped.
        Error Status Only Status report of all the copied files with errors. Each file shows the status, path, and message for the copied files with errors.
        Skipped Status Only Status report of all skipped files. Each file lists the status, path, and message for the databases and tables that were skipped.
        Deleted Status Only Status report of all deleted files. Each file lists the status, path, and message for the databases and tables that were deleted.
        Performance Summary report about the performance of the running replication job which includes the last performance sample for each mapper that is working on the replication job.
        Full Performance Performance report of the job which includes the samples taken for all mappers during the replication job.
      • Run As Username is the username specified during replication policy creation to run the replication job.
      • Run on Peer as Username is the username specified during replication policy creation.
      • Message shows the total number of files copied to target cluster and the number of files that remain unchanged on the source cluster.