Integrating Cloudera Replication Manager

Install, flag, and register Cloudera Replication Manager so Cloudera Migration Assistant can delegate Base-to-Base replication policies for HDFS, Hive, HBase, and HDFS-to-Ozone scenarios.

CMA uses Cloudera Replication Manager (Cloudera Replication Manager) as the replication engine for Cloudera Base on premises to Cloudera Base on premises migrations.

Cloudera Replication Manager is a Cloudera service that creates and manages data replication policies between clusters. It supports the replication of the following data:

HDFS data, including HDFS to Ozone
Hive metadata and data
HBase tables

CMA delegates all data replication operations to Cloudera Replication Manager through its REST API.

Cloudera Replication Manager is required for Cloudera Base on premises to Cloudera Base on premises migrations that involve data replication.

Table 1. Cloudera Replication Manager requirement by migration type
Migration type	Cloudera Replication Manager required?
HDFS replication (Base to Base)	Yes
HDFS to Ozone migration	Yes
Hive replication (Base to Base)	Yes
HBase replication (Base to Base)	Yes
Scanning clusters	No

Cloudera recommends deploying Cloudera Replication Manager (DMX) 1.2 or higher versions on the target cluster, on the same node as Cloudera Manager. It is installed as a separate Cloudera Manager parcel alongside the CMA parcel.

Target Cluster Node (Cloudera Manager host)
        ├── CMA  (parcel)
        └── Cloudera Replication Manager (parcel)

The Cloudera Manager version must be 7.13.2 or higher versions on the target cluster because of a requirement for HDFS to Ozone migrations.
You must deploy Cloudera Cloudera Replication Manager on the same node as Cloudera Manager on the target cluster.
The Cloudera Replication Manager REST API port 9011 must be accessible from the CMA Agent on the target cluster.
You must deploy the CMA Agent and connect to the CMA Master on the target cluster.
You must configure the Cloudera Manager administrator credentials for both source and target clusters in CMA.

The following network requirements must be met:

Table 2. Network requirements
From	To	Port	Protocol	Purpose
CMA Agent (target)	Cloudera Replication Manager	9011	REST (HTTP)	Replication policy management
Cloudera Replication Manager	Source Cloudera Manager	7180/7183	HTTP/HTTPS	Source cluster access
Cloudera Replication Manager	Target Cloudera Manager	7180/7183	HTTP/HTTPS	Target cluster access

Enable the HDFS to Ozone flag on the Cloudera Manager Server host.

For HDFS to Ozone migrations, Cloudera Manager requires the CMF_FF_API_H2O_REPLICATION feature flag to be enabled. Without this flag, the Cloudera Replication Manager API endpoints for HDFS to Ozone replication are not available.
note
This flag is only required for HDFS to Ozone migrations. It is not required for HDFS-to-HDFS, Hive, or HBase replication.
1. Append the following line to /etc/default/cloudera-scm-server on the Cloudera Manager Server host:
```
export CMF_FF_API_H2O_REPLICATION=true
```
2. Restart the Cloudera Manager Server.
```
sudo systemctl restart cloudera-scm-server
```
3. Wait for Cloudera Manager to become available again on port 7180 before proceeding with the CMA setup.
Register the Cloudera Replication Manager instance in the Cloudera Migration Assistant UI to allow CMA Agents to locate and communicate with the replication service.
1. In the CMA Web UI, go to Settings.
2. Go to the Cloudera Replication Manager App Access section.
3. Configure your connection fields using one of the following validation methods:
  - If CMA has already detected a Cloudera Replication Manager service on one of the registered clusters, select it from the Suggestions drop-down list to auto-fill the form.
  - Manually enter your target replication parameters in the following fields:
    - Cloudera Replication Manager App URL — The full URL of the Cloudera Replication Manager REST API, including protocol, host, and port, for example, https://rm-host.example.com:9011.
    - Cloudera Replication Manager App username — The username for authenticating with the Cloudera Replication Manager API.
    - Cloudera Replication Manager App password — The password for authenticating with the Cloudera Replication Manager API.
4. Click Save and Push to Agents.

CMA saves the Cloudera Replication Manager credentials and pushes them to all connected CMA Agents. The Agents then use these credentials to register the managed clusters with Cloudera Replication Manager and to manage replication policies during migration execution.

The integration subsequently automates the following background operations without manual user intervention:

Automated cluster registration

When a migration plan initializes, the CMA Master sends an internal RM_CLUSTER_REGISTRATION command to the target CMA Agent. The Agent automatically calls the Cloudera Replication Manager REST API endpoint (/cm/registerClusters) to match the source and target cluster pairs, and writes the credentials to the /store/credential endpoint. If a cluster pair is already registered, the agent skips this step automatically.
Migration execution interactivity

During active migration runtime, the target CMA Agent leverages the connection to create replication policies, manage HDFS and Ozone snapshots for efficient incremental delta transfers, poll policy statuses for real-time UI tracking, and clean up expired policies upon completion.

All Cloudera Replication Manager interactions are performed by the CMA Agent on the target cluster and reported back to CMA Master for tracking and display in the UI.

Table 3. Troubleshooting the integration
Issue	Possible cause	Resolution
Replication policy creation fails	Clusters not registered in Cloudera Replication Manager	Check CMA Agent logs for registration errors, then verify if Cloudera Replication Manager is running on port 9011.
Registration fails with credential error	Incorrect CM credentials in CMA	Update Cloudera Manager credentials in CMA for the affected cluster.
Cloudera Replication Manager not reachable from Agent	Network/firewall issue	Ensure that port 9011 is open between the Agent and Cloudera Replication Manager on the target cluster.
HDFS to Ozone replication not available	CM version too old	Upgrade Cloudera Manager on the target cluster to 7.13.2 or higher versions.