Cloudera Docs

Manage and monitor Iceberg replication policies

After you create an Iceberg replication policy, you can perform and monitor various tasks related to the replication policy. You can view the job progress and replication logs. You can edit the advanced options to optimize a job run. You can suspend a job and also activate a suspended job.

Replication policy details on the Replication Policies page

On the Replication Policies page, you can view the following details about the replication policy:

  • Shows a row of information for each replication policy, and the following columns for each replication policy:
    • Internally generated ID for the replication policy. Click the column label to sort the replication policies.
    • Replication policy Name that you provide during replication policy creation.
    • Replication policy Type.
    • Source cluster in the replication policy.
    • Destination cluster in the replication policy.
    • Average Throughput per mapper/file for all the files written.
    • Replication job Progress.
    • Timestamp when the replication job Completed.
    • Replication policy job’s Next Run.
  • Provides the following options under the Actions menu:
    • Show History opens the Replication History page for the replication policy.
    • Edit Configuration enables you to change the replication policy options as required.
    • Run Now initiates a replication job.
    • Disable an active replication policy.

      You can Enable it later, as necessary.

  • Delete the replication policy permanently. Deleting a replication policy does not delete copied files or tables.

Replication History page

Click Actions > Show History for a replication policy on the Replication Policies page to view the Replication History page.

On the Replication History page, you can view the following run details about a replication policy job:

  • Shows the replication policy Name; replication policy Type; Source cluster name; Destination cluster name; and Next Run of the replication policy.
  • Shows a row of information for each replication policy job run, and the following columns for each replication policy:
    Column Description
    Start Time Shows the timestamp when the replication job started.
    Origin TimeStamp (UTC) Shows the timestamp when the export step started on the source cluster.
    Duration Shows the time taken to complete the replication job.
    Outcome Shows the replication job status as Running, Successful, or Failed.
    Number of tables processed Shows the number of tables processed by the replication policy job.
    Number of files copied Shows the number of successfully copied files.
    Number of files deleted Shows the number of files deleted by the replication policy job.
    Number of manifests transformed Shows the number of copied manifest files (with incorrect path replacements) that were corrected to point to the correct data files on the target cluster.
  • Expand a job to view the following information on the All Recent Commands window:
    • Status of the replication job.
    • Iceberg Replication in the Context field opens the Clusters > Iceberg Replication window where more details about the replication policy job appears.
    • Replication job Started At timestamp.
    • Duration to complete the job.
    • Download the results to your machine.
    • Expand to Show All Steps, Show Only Failed Steps, or Show Only Running Steps for the commands used by Iceberg replication policy.
    • Show Command Timing shows the timeline for the commands used by the Iceberg replication policy.