What's new
This section lists major features and updates for Replication Manager service.
Mar 07, 2023
This release of the Replication Manager service introduces the following new features.
HBase replication policy enhancements
The following enhancements are available for HBase replication policies:
- During HBase replication policy creation, you can choose one of the following
cloud credential options:
- Credentials are available in source cluster HDFS
service configuration setting- You can choose this
option when you want to use a Google Cloud account. You can use this
option for S3 and ADLS accounts as well.To add the advanced configuration settings for Google Cloud, Amazon S3, and ADLS accounts, perform the following steps:
-
Go to the
tab. -
Locate the HDFS Client Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml property.
-
Add the following key-value pairs to register a Google account to use in Replication Manager:
- fs.gs.impl=com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem
- fs.gs.project.id=[***enter the project ID***]
- fs.gs.system.bucket=[***enter the bucket name***]
- fs.gs.working.dir=/
- fs.gs.auth.service.account.enable=true
- fs.gs.auth.service.account.email=[***enter the service principal email ID***]
- fs.gs.auth.service.account.keyfile=/[***Enter the local path of the p12 file***]
- Add the following key-value pairs to register an S3 account
to use in Replication Manager:
- fs.s3a.access.key=[***enter the session access key***]
- fs.s3a.secret.key=[***enter the session secret key***]
- Add the following key-value pairs to register an ADLS
account to use in Replication Manager:
- fs.azure.account.oauth2.client.id=[***enter the ABFS storage client ID***]
- fs.azure.account.oauth2.client.secret=[***enter the ABFS storage client secret key***]
-
Save and restart the HDFS service for the changes to take effect.
-
- Credentials from External Account - You can choose this option for S3 and ADLS storage options.
For more information, see Creating an HBase replication policy.
- Credentials are available in source cluster HDFS
service configuration setting- You can choose this
option when you want to use a Google Cloud account. You can use this
option for S3 and ADLS accounts as well.
- You can Force Delete an HBase replication policy after Replication Manager fails to delete the replication policy (using the option on the Replication Policies page for the replication policy). For more information, see Managing HBase replication policy.
Jan 11, 2023
This release of the Replication Manager service introduces the following new features.
HBase replication policy enhancements
The following enhancements are available for HBase replication policies:
-
HBase data replication using HBase replication policies is supported when SFT setting is automatically set on SFT-enabled clusters.
- You must choose the
For more information, see Creating HBase replication policy.
option in the HBase policy creation wizard to acknowledge that
the first-time setup between the selected source and destination clusters should
be initiated after the existing pairing of the source and/or target cluster gets
cleared. - You can also use the following options to manage an HBase replication policy:
- The action suspends all the active HBase replication policies between the source and target clusters selected in the replication policy.
- The action resumes data replication for all the HBase replication policies between the source and target clusters selected in the replication policy.
- The action retries the first-time setup between the source and target clusters selected in the replication policy. This option is available only if the first-time setup configuration fails.
- The action reruns the failed initial snapshots (and only the failed ones) in the replication policy if the Replication Manager failed to replicate the existing data of some tables.
- The action opens the latest replication policy job and shows the last 15 steps of the log for the policy job run. The steps and substeps appear in a tree view. The failed steps are expanded by default.
For more information, see Managing HBase replication policy.
-
You can now monitor the first-time setup configuration steps and its progress on the
tab.
Hive replication policy enhancements
The following enhancements are available for Hive replication policies:
- You can use Hive replication policies to replicate Hive external tables and
metadata from CDH 5.10 and higher clusters and CDP Private Cloud Base 7.1.1 and
higher clusters to Data Hubs using Cloudera Manager 7.9.0 or higher.
For more information, see Creating Hive replication policy.
- The
For more information, see Managing Hive replication policy.
action opens the latest replication policy job and shows the last
15 steps of the log for the policy job run. The steps and substeps appear in a
tree view. The failed steps are expanded by default.
May 16, 2022
This release of the Replication Manager service introduces the following new features.
Updates to HBase replication policy
- If you are using Cloudera Manager version 7.6.0 or higher, you can use
the following options while creating an HBase replication policy:
- Rolling HBase Service Restart on Source - This option appears if you select COD or Data Hub as the source cluster. Select this option to enable automatic rolling restart of HBase service on the source cluster after the HBase replication policy first-time setup steps are complete. Otherwise, Cloudera Manager performs a full restart of the service.
- Rolling HBase Service Restart on Destination - Select this option to enable automatic rolling restart of HBase service on the target cluster as a rolling restart after the HBase replication policy first-time setup steps are complete. Otherwise, Cloudera Manager performs a full restart of the service.
- Validate Policy - Select to notify Replication Manager to verify the policy details after the policy creation is complete.
For more information, see Creating HBase replication policy.
- A warning message appears when you choose a cluster that is part of an
existing cluster pairing. If there are HBase replication policies for the
existing pairing, policy creation is not allowed to continue. If there are no
policies, the existing pairing is removed and policy creation continues.
For more information, see Using HBase replication policy.
Support for Replication Manager in ap-1 and eu-1 regional Control Planes
Replication Manager is now supported in the ap-1 (Australia) and eu-1 (Germany) regional Control Planes. For the list of all supported services in the supported Control Plane regions, see CDP Control Plane regions.