What's new

HBase replication policy enhancements

The following enhancements are available for HBase replication policies:

• During HBase replication policy creation, you can choose one of the following cloud credential options:
  • Credentials are available in source cluster HDFS service configuration setting- You can choose this option when you want to use a Google Cloud account. You can use this option for S3 and ADLS accounts as well.
    To add the advanced configuration settings for Google Cloud, Amazon S3, and ADLS accounts, perform the following steps:

    1. Go to the source Cloudera Manager > Clusters > HDFS service > Configuration tab.

    2. Locate the HDFS Client Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml property.

    3. Add the following key-value pairs to register a Google account to use in Replication Manager:

       • fs.gs.impl=com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem
       • fs.gs.project.id=[***enter the project ID***]
       • fs.gs.system.bucket=[***enter the bucket name***]
       • fs.gs.working.dir=/
       • fs.gs.auth.service.account.enable=true
       • fs.gs.auth.service.account.email=[***enter the service principal email ID***]
       • fs.gs.auth.service.account.keyfile=/[***Enter the local path of the p12 file***]
    4. Add the following key-value pairs to register an S3 account to use in Replication Manager:
       • fs.s3a.access.key=[***enter the session access key***]
       • fs.s3a.secret.key=[***enter the session secret key***]
    5. Add the following key-value pairs to register an ADLS account to use in Replication Manager:
       • fs.azure.account.oauth2.client.id=[***enter the ABFS storage client ID***]
       • fs.azure.account.oauth2.client.secret=[***enter the ABFS storage client secret key***]

    6. Save and restart the HDFS service for the changes to take effect.

  • Credentials from External Account - You can choose this option for S3 and ADLS storage options.

  For more information, see Creating an HBase replication policy.

• You can Force Delete an HBase replication policy after Replication Manager fails to delete the replication policy (using the Actions > Delete option on the Replication Policies page for the replication policy). For more information, see Managing HBase replication policy.

HBase replication policy enhancements

The following enhancements are available for HBase replication policies:

• HBase data replication using HBase replication policies is supported when SFT setting is automatically set on SFT-enabled clusters.

• You must choose the Select Destination > I want to force the setup of this HBase replication policy option in the HBase policy creation wizard to acknowledge that the first-time setup between the selected source and destination clusters should be initiated after the existing pairing of the source and/or target cluster gets cleared.

  For more information, see Creating HBase replication policy.

• You can also use the following options to manage an HBase replication policy:
  • The Replication Policies > HBase replication policy > Actions > Suspend action suspends all the active HBase replication policies between the source and target clusters selected in the replication policy.
  • The Replication Policies > HBase replication policy > Actions > Activate action resumes data replication for all the HBase replication policies between the source and target clusters selected in the replication policy.
  • The Replication Policies > HBase replication policy > Actions > Retry Create action retries the first-time setup between the source and target clusters selected in the replication policy. This option is available only if the first-time setup configuration fails.
  • The Replication Policies > HBase replication policy > Actions > Retry Failed Snapshots action reruns the failed initial snapshots (and only the failed ones) in the replication policy if the Replication Manager failed to replicate the existing data of some tables.
  • The Replication Policies > HBase replication policy > Actions > View Command Details action opens the latest replication policy job and shows the last 15 steps of the log for the policy job run. The steps and substeps appear in a tree view. The failed steps are expanded by default.

  For more information, see Managing HBase replication policy.

• You can now monitor the first-time setup configuration steps and its progress on the Replication Policies > HBase replication policy > Job History tab.

Hive replication policy enhancements

The following enhancements are available for Hive replication policies:

• You can use Hive replication policies to replicate Hive external tables and metadata from CDH 5.10 and higher clusters and CDP Private Cloud Base 7.1.1 and higher clusters to Data Hubs using Cloudera Manager 7.9.0 or higher.

  For more information, see Creating Hive replication policy.

• The Replication Policies > Hive replication policy > Actions > View Command Details action opens the latest replication policy job and shows the last 15 steps of the log for the policy job run. The steps and substeps appear in a tree view. The failed steps are expanded by default.

  For more information, see Managing Hive replication policy.

Updates to HBase replication policy

• If you are using Cloudera Manager version 7.6.0 or higher, you can use the following options while creating an HBase replication policy:
  • Rolling HBase Service Restart on Source - This option appears if you select COD or Data Hub as the source cluster. Select this option to enable automatic rolling restart of HBase service on the source cluster after the HBase replication policy first-time setup steps are complete. Otherwise, Cloudera Manager performs a full restart of the service.
  • Rolling HBase Service Restart on Destination - Select this option to enable automatic rolling restart of HBase service on the target cluster as a rolling restart after the HBase replication policy first-time setup steps are complete. Otherwise, Cloudera Manager performs a full restart of the service.
  • Validate Policy - Select to notify Replication Manager to verify the policy details after the policy creation is complete.

  For more information, see Creating HBase replication policy.

• A warning message appears when you choose a cluster that is part of an existing cluster pairing. If there are HBase replication policies for the existing pairing, policy creation is not allowed to continue. If there are no policies, the existing pairing is removed and policy creation continues.

  For more information, see Using HBase replication policy.

Support for Replication Manager in ap-1 and eu-1 regional Control Planes

Replication Manager is now supported in the ap-1 (Australia) and eu-1 (Germany) regional Control Planes. For the list of all supported services in the supported Control Plane regions, see CDP Control Plane regions.