Use Cloudera Replication Manager to migrate to Cloudera
on cloud
Cloudera Replication Manager is a service to copy and migrate data from CDH
5.13+ and above clusters (HDFS, Hive, and HBase data) and Cloudera Base on premises 7.1.4 and above clusters (HDFS , Hive
external tables, and HBase data) to Cloudera
on cloud clusters. The supported on cloud services include Amazon S3 or Microsoft Azure ADLS
Gen2 (ABFS). Replication Manager from HDP clusters to Cloudera
on cloud Azure is a beta feature and is not available for
general use.
About Replication Manager Cloudera on cloud . You can create replication policies in Replication Manager to copy and migrate data from CDH (version 5.13 and higher) clusters (HDFS, Hive, and HBase data) and Cloudera Base on premises (version 7.1.1 and higher) clusters (HDFS, Hive external tables, and HBase data) to Cloudera on cloud clusters. You can also replicate HDFS data from cloud storage to classic clusters (CDH or Cloudera Base on premises clusters), and Hive external tables to Data Hubs. The supported on cloud services include Amazon S3 and Microsoft Azure ADLS Gen2 (ABFS). Replicating Hive managed tables using Replication Manager from HDP clusters to Cloudera on cloud is a beta feature and is not available for general use.Fine-grained permission to access Cloudera Replication Manager Cloudera Replication Manager in a Cloudera on cloud environment so that you can govern the access to critical replication functionalities. Accessing Replication Manager UI Cloudera > Select Replication Manager .How replication policies work Cloudera Replication Manager , you create replication policies to establish the rules you want applied to your replication jobs. The policy rules you set can include which cluster is the source and which is the destination, what data is replicated, what day and time the replication job occurs, the frequency of job runs, and bandwidth restrictions.Using HDFS replication policies Cloudera Replication Manager to replicate HDFS data. The HDFS replication policies can replicate HDFS data and metadata from classic clusters (CDH, Cloudera Base on premises , and HDP) to Cloudera on cloud storage buckets such as S3 and ABFS, and from cloud storage to classic clusters (CDH or Cloudera Base on premises clusters). To use an on-premises cluster (CDH or Cloudera Base on premises ) in the replication policy, you must register it as a classic cluster in the Cloudera Management Console . To use the cloud storage for data replication, you must register the cloud credentials in Replication Manager so that the Replication Manager service can access the cloud storage. You must also verify cluster access and configure minimum ports for replication before you create HDFS replication policies.Using Hive replication policies Cloudera Replication Manager , you must configure the required Ranger policy in Ranger, register the on-premises cluster (CDH or Cloudera Base on premises ) as a classic cluster in Cloudera Management Console , register cloud account credentials in the Replication Manager service, verify cluster access, and configure minimum ports for replication. The replication load happens on the source on-premises cluster. You can replicate data on-premises to the cloud with a single cluster if the Metastore is running on the cloud.Using HBase replication policies Cloudera Replication Manager , you must register the on-premises cluster (CDH or Cloudera Base on premises ) as a classic cluster in Cloudera Management Console , register cloud account credentials in the Replication Manager service, verify cluster access, and configure minimum ports for replication. Troubleshooting replication policies in Cloudera Replication Manager Cloudera Replication Manager .Appendix Cloudera Replication Manager , and register the on-premises clusters (CDH or Cloudera Base on premises ) as classic clusters in the Management Console. 