What's new

Replicate HBase data in all the tables in a database

You can replicate HBase data (existing tables and future tables in a database) using the Select Source > Replicate Databases option during the HBase replication policy creation process. You can use this option only if the minimum target Cloudera Manager version is 7.11.0 and the minimum source cluster versions are CDH 5.16.2 (after you upgrade the source cluster Cloudera Manager), CDP Private Cloud Base 7.1.6, or COD 7.2.17.

For more information, see HBase replication policy and Creating HBase replication policies.

Provide a unique replication policy name

You must provide a unique name to the replication policy during the replication policy creation process if the Cloudera Manager API version is 51 or higher.

For other Cloudera Manager API versions, you can continue to use an existing replication policy with an empty name. However, if you edit the replication policy and provide a name for the replication policy, ensure that the name conforms to the validation rules.

For more information, see Managing HDFS replication policy, Managing Hive replication policy, or Managing HBase replication policy.

Load replication policies and their job history on the Replication Policies page

You can choose one of the following options to Load policies faster by delaying to load their job history:

• Delay loading job history when history is too long
• Always load job history
• Never load job history

By default, the replication policies are loaded only partially on the Replication Policies page, therefore the page might display incomplete statistics about a job status and replication policies with failed jobs might take a longer time to load. You can change the behavior depending on your requirements.

For more information, see Replication Policies page

HBase replication policy enhancements

You can view the HBase RegionServer metrics for a specific replication peer in Replication Manager. You can use these metrics to monitor HBase replication jobs and to find and diagnose issues with the HBase replication peer.

For more information, see Graphical representation of HBase RegionServer replication peer metrics

CDP CLI commands for Replication Manager

You can collect and download the diagnostic bundle, add ABFS and AWS cloud credentials to use in Replication Manager and delete credentials using CDP CLI commands.

For more information, see Adding cloud credentials in Replication Manager using CDP CLI.

HBase replication policy enhancements

The following enhancements are available for HBase replication policies:

• During HBase replication policy creation, you can choose one of the following cloud credential options:
  • Credentials are available in source cluster HDFS service configuration setting- You can choose this option when you want to use a Google Cloud account. You can use this option for S3 and ADLS accounts as well.
    To add the advanced configuration settings for Google Cloud, Amazon S3, and ADLS accounts, perform the following steps:

    1. Go to the source Cloudera Manager > Clusters > HDFS service > Configuration tab.

    2. Locate the HDFS Client Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml property.

    3. Add the following key-value pairs to register a Google account to use in Replication Manager:

       • fs.gs.impl=com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem
       • fs.gs.project.id=[***enter the project ID***]
       • fs.gs.system.bucket=[***enter the bucket name***]
       • fs.gs.working.dir=/
       • fs.gs.auth.service.account.enable=true
       • fs.gs.auth.service.account.email=[***enter the service principal email ID***]
       • fs.gs.auth.service.account.keyfile=/[***Enter the local path of the p12 file***]
    4. Add the following key-value pairs to register an S3 account to use in Replication Manager:
       • fs.s3a.access.key=[***enter the session access key***]
       • fs.s3a.secret.key=[***enter the session secret key***]
    5. Add the following key-value pairs to register an ADLS account to use in Replication Manager:
       • fs.azure.account.oauth2.client.id=[***enter the ABFS storage client ID***]
       • fs.azure.account.oauth2.client.secret=[***enter the ABFS storage client secret key***]

    6. Save and restart the HDFS service for the changes to take effect.

  • Credentials from External Account - You can choose this option for S3 and ADLS storage options.

  For more information, see Creating an HBase replication policy.

• You can Force Delete an HBase replication policy after Replication Manager fails to delete the replication policy (using the Actions > Delete option on the Replication Policies page for the replication policy). For more information, see Managing HBase replication policy.