Cloudera Docs

CDP CLI options for Replication Manager

You can use the CDP CLI commands to create, activate, and delete HDFS, Hive, and HBase replication policies.

CDP CLI options

You can use the following CDP CLI options to perform tasks in Replication Manager:

CDP CLI option Description
abort-policy* Aborts all running instances of the specified replication policy.

Provide the CRN of the cluster where the replication policy is created and stored.
activate-hbase-policy Resumes the specified suspended HBase replication policy.

If the target Cloudera Manager API version is higher than 50, the command resumes all the HBase replication policies between the same source and destination cluster. However, if the target Cloudera Manager API version is lower than 45, the command resumes only the specified HBase replication policy. This command is not available for target Cloudera Manager API versions between 45 and 50.
activate-policy Resumes the specified suspended HDFS or Hive replication policy. Provide source CRN for HDFS replication policies and target CRN for Hive replication policies.
activate-snapshot-policy* Resumes a suspended snapshot policy run.
collect-diagnostic-bundle Triggers the diagnostic bundle collection for the specified replication policy in the target Cloudera Manager.

Use the commandId in the command output to download the diagnostic bundle.
continue-hbase-setup Continues the first-time setup on the target cluster.
create-abfs-access-key-credential* Creates an ABFS access key-based account in Replication Manager.
create-abfs-client-key-credential* Creates an Azure active directory service principal account in source Cloudera Manager.
create-abfs-credential Adds a ABFS cloud credentials to use in Replication Manager.
create-aws-access-key-credential* Creates an AWS access key cloud credential.
create-aws-credential Adds AWS cloud credentials to use in Replication Manager.
create-aws-iam-credential* Creates an AWS IAM-role based cloud credential.
create-aws-idbroker-credential* Creates an AWS IDBroker cloud credential.
create-gcs-private-key-credential* Creates a GCS private key cloud credential.
create-hbase-policy Creates an HBase replication policy with the given name on the specified cluster.
create-policy Creates an HDFS or Hive replication policy based on the provided parameters. Provide source CRN for HDFS replication policies and target CRN for Hive replication policies.
create-snapshot-policy* Creates a HDFS or HBase snapshot policy depending on the details you provide in the snapshot definition.
delete-credential Deletes the registered credentials from Replication Manager.
delete-hbase-policy Deletes the specified HBase replication policy permanently.

Enter --force if a normal delete fails. For example, when the source cluster is unreachable. Otherwise, enter --no-force.
delete-policy Deletes the specified HDFS or Hive replication policy. Provide source CRN for HDFS replication policies and target CRN for Hive replication policies.
delete-snapshot-policy* Deletes the specified snapshot policy.
download-diagnostic-bundle Generates a bundleFile, a binary string in base64 encoded format, as a ZIP file. Run a script to save the response as a file to your machine. For example, cat response.json | jq -r ‘.bundleFile’ | base64 -D > bundle.zip.

The file is generated only if the bundleStatus in the get-command-status command shows DOWNLOADABLE WITH CLI.
get-cluster-config Retrieves the configuration of a specified cluster.
get-command-status Returns the current status of the collect-diagnostic-bundle command. You can also view the status of any Cloudera Manager command using the relevant input command ID.
get-credentials Returns the cloud credentials that are available on the specified cluster. If you provide the credential name and credential ID, the ID is considered by the CDP CLI command.
get-hbase-time-series* Returns the time series data for an HBase replication peer depending on the provided parameters.
You can enter one of the following HBase replication peer metric:
  • age_of_last_shipped_operation
  • log_edits_filtered_rate
  • log_edits_read_rate
  • log_queue_size
  • log_read_in_bytes_rate
  • shipped_batches_rate
  • shipped_ops_rate
  • shipped_size_in_kb_rate
  • age_of_last_shipped_operation_25th_percentile
  • age_of_last_shipped_operation_75th_percentile
  • age_of_last_shipped_operation_90th_percentile
  • age_of_last_shipped_operation_98th_percentile
  • age_of_last_shipped_operation_99th_percentile
  • age_of_last_shipped_operation_999th_percentile
  • age_of_last_shipped_operation_max
  • age_of_last_shipped_operation_mean
  • age_of_last_shipped_operation_median
  • age_of_last_shipped_operation_min
  • age_of_last_shipped_operation_rate
  • shipped_hfiles_rate
  • size_of_hfile_refs_queue
You can enter one of the following aggregate rollup level for time series data:
  • RAW
  • TEN_MINUTELY
  • HOURLY
  • SIX_HOURLY
  • DAILY
  • WEEKLY
get-snapshot-policy* Retrieves the details about the specified snapshot policy.
list-all-credentials Provides a detailed list of cloud credentials across all clusters that are available for Replication Manager.
list-cluster-service-statuses Provides the current status of the services on all the clusters that are available for Replication Manager.
list-clusters Provides a detailed list of all the clusters that are available for Replication Manager.
list-paired-hbase-clusters Lists the paired clusters to use for HBase replication policies. The first-time setup is complete for paired clusters.
list-policies Lists the available replication policies across all the clusters. Enter the source cluster CRN to view the HDFS replication policies, and target cluster CRN to view all the Hive and HBase replication policies.
list-policy-jobs* Returns the list of jobs triggered by the replication policy. This command is a paginated operation, therefore multiple API calls might be required to retrieve the entire dataset of results. You can use one of the following methods to retrieve the results:
  • Use --no-paginate argument to disable pagination.
  • Use --max-items [***TOTAL NUMBER OF ITEMS TO RETURN***] --starting-token [***TOKEN TO START PAGINATING***] --page-size [***PAGE SIZE***] to return a specific number of jobs at a time.
list-snapshot-policies* Returns a list of all the snapshot policies across all available clusters.
list-snapshot-policy-jobs* Retrieves the snapshot history of the specified snapshot policy.
repair-hbase-policy* Runs the last failed command for a failed HBase replication policy.
rerun-policy* Runs the specified HDFS or Hive replication policy.
restore-snapshot* Restores the specified snapshot.
retry-failed-hbase-first-time-setup* Runs the first time setup configuration between the clusters in the specified HBase replication policy if the first time setup fails for the replication policy.
You can use the following arguments as required:
  • --machine-user [***ENTER USERNAME AND PASSWORD OF MACHINE USER***]

    Syntax: user=[***STRING***], password=[***STRING***], createUser=[***ENTER 'true' TO CREATE A NEW MACHINE USER, OR 'false' TO RETURN AN ERROR IF THE USER DOES NOT EXIST***]

  • --source-restart-type [***ENTER RESTART OR ROLLING_RESTART TO SPECIFY THE RESTART TYPE***]
  • --target-restart-type [***ENTER RESTART OR ROLLING_RESTART TO SPECIFY THE RESTART TYPE***]
retry-failed-hbase-snapshots* Reruns only the failed initial snapshots in the replication policy if the policy failed to replicate existing data in some or all the tables
suspend-hbase-policy Pauses an active HBase replication policy.
suspend-policy Stops all the replication tasks defined for the HDFS or Hive replication policy. Provide source CRN for HDFS replication policies and target CRN for Hive replication policies.
suspend-snapshot-policy* Suspends the specified snapshot policy.
update-abfs-access-key-credential* Updates an existing ABFS access key-based account in Replication Manager.
update-abfs-client-key-credential* Updates an existing Azure active directory service principal account in source Cloudera Manager.
update-aws-access-key-credential* Updates an existing AWS access key cloud credential.
update-aws-iam-credential* Updates an existing AWS IAM-role based cloud credential
update-aws-idbroker-credential* Updates an existing AWS IDBroker cloud credential.
update-gcs-private-key-credential* Updates and existing GCS private key cloud credential.
update-hbase-policy Modifies an existing HBase replication policy. You can change the name and description of the replication policy. You can also delete one or more tables in the replication policy.
update-policy* Modifies the HDFS or Hive replication policy.

To modify the replication policy, make appropriate changes in the JSON file, save the file, and use it in the update-policy command. Use the rerun-policy command to run the policy.
update-snapshot-policy* Modifies an existing snapshot policy depending on the arguments you choose.
verify-hbase-cluster-pair Verifies whether the specified pair of clusters are paired, not paired, or wrongly paired.

The wrongly paired status indicates that one or both of the specified clusters are paired to other clusters.
*The option is a technical preview feature and is not ready for production deployment. The components are provided ‘as is’ without warranty or support. Further, Cloudera assumes no liability for the use of preview components, which should be used by customers at their own risk. For more information, contact your Cloudera account team.

CDP CLI options to create a replication policy

The following parameters are available in the replicationmanager create-policy CDP CLI option:

Parameter Description
--cluster-crn Enter the cluster CRN. To determine the cluster CRN, run the list-clusters command.
Provide the following CRN when you create a replication policy:
  • Source cluster CRN for HDFS replication policy.
  • Target cluster CRN for Hive replication policy.
  • Target cluster CRN for HBase replication policy.
--policy-name Enter a unique name for the replication policy.
--policy-definition Enter the policy definition in the $(cat [***POLICY DEFINITION FILE NAME***]) format, and then enter the cluster CRN and policy name as command arguments.
--cli-input-json Enter the policy definition JSON file path using the cat command to read the data from the file to create, and run the replication policy.
--generate-cli-skeleton Shows a policy definition template in JSON format. You can copy the output of this command to a file, add the required parameters, and save it as a JSON file. You can use the filename while creating a replication policy.