Cloudera Docs

Troubleshooting snapshot policies in Replication Manager

You must be aware of the issues related to the snapshot policies that you create between on-premises clusters in Replication Manager and how to resolve those issues.

Error when snapshot policy is edited or deleted

Errors might appear when you edit or delete a snapshot policy that contains % . ; / \ or any character that is not ASCII printable which includes the ASCII characters less than 32 and the ASCII characters that are greater than or equal to 127.

To resolve this issue, use the update command to replace the unsupported character in the policy name with an underscore, in the SNAPSHOT_POLICIES table.

To update the snapshot policy name in the SNAPSHOT_POLICIES table, perform the following steps:

  1. Take a backup of the Cloudera Manager database.
  2. Run the update SNAPSHOT_POLICIES set NAME = replace(NAME,CHAR([***ENTER CHARACTER NUMBER***]),'_'); command to replace the unsupported character in the snapshot policy name with an underscore.

Error when target directory is out-of-sync

The 
target has been modified since snapshot [***SNAPSHOT NAME**]
error might appear when Cloudera Manager HA (planned or unplanned failover) is initiated.

This error appears when the target directory is out-of-sync with the source directory during a planned or unplanned failover. The directories might go out-of-sync if the files or folders in the target directory were modified or the source directory changed and a snapshot was created to make the synchronization point. The modifications on the target cluster include creating or dropping databases, tables, and partitions in Hive.

To mitigate this issue, you can run the replication policy which bootstraps the target. However, this action might modify the target cluster and might result in data loss. In addition, the time and resources consumed to complete the bootstrap replication job is high.

However, as a safety measure if you created a snapshot (for example, sync_v1) for the target directory before you created the HDFS or Hive replication policy, then you can run the following steps to initiate the incremental replication from the source directory to target directory, which optimizes the replication job by using less time and resources:

  1. Take a snapshot of the target cluster by using the hdfs dfs -createSnapshot [***TARGET DIRECTORY***] `NEW SNAPSHOT NAME***] command. Ensure that you use an alternate snapshot naming pattern to distinguish this snapshot from the rest of the snapshots.

    For example, hdfs dfs -createSnapshot tar_directory mod_v1

  2. Revert the target directory to the snapshot you created initially using the hadoop distcp -rdiff [***INITIAL SNAPSHOT***] [***NEW SNAPSHOT NAME***] -update [***SOURCE DIRECTORY***] [***TARGET DIRECTORY***] command.

    For example, hadoop distcp -rdiff mod_v1 sync_v1 -update src_directory tar_directory

  3. Run the replication policy to sync the source and target directories.