Preparing to create an HBase replication policy

1. Complete Step 1 of Migrating HBase data from CDH/HDP to COD to install the HBase replication plugin parcel in the CDH source clusters.

   This step is applicable for CDH versions 7.2.x that are lower than 7.2.2, versions 7.1.x that are lower than 7.1.5, and for versions lower than 7.x.

2. Create the /user/hbase folder for the hbase user in HDFS in the source cluster using the following commands:

   sudo -u hdfs hdfs dfs -mkdir /user/hbase
   sudo -u hdfs hdfs dfs -chown hbase:hbase /user/hbase

   These commands allow the HBase replication policy to replicate the existing data in the source cluster.

   This step is applicable for Cloudera Manager versions 7.4.3 or lower; Cloudera Manager 7.4.4 only if the API version is lower than v45. The endpoint http://[***CM_HOST***]:[***CM_PORT***]/api/version shows the API version of the Cloudera Manager.

1. Go to the source Cloudera Manager > YARN service > Configuration tab.
2. Ensure one of the following conditions is met:
   • The Allowed System Users property must have hbase. Otherwise, add hbase to the existing property value.

     This property lists the users permitted (or allowed) to run containers. Note that the users with IDs lower than the Minimum User ID property might be permitted (or allowed) to run containers.

   • The Minimum User ID property is set to a value that is lower than the HBase user's ID.

     To check the HBase user's ID, SSH into a cluster node and run the id hbase command.

     The following sample snippet shows the HBase user's ID when you run the id hbase command:

     # id hbase
     uid=39993(hbase) gid=39993(hbase) groups=39993(hbase)

1. Go to the source Cloudera Manager > Clusters > HDFS service > Configuration tab.

2. Locate the HDFS Client Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml property.

3. Add the following key-value pairs to register a Google account to use in Replication Manager:

   • fs.gs.impl=com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem
   • fs.gs.project.id=[***ENTER THE PROJECT ID***]
   • fs.gs.system.bucket=[***ENTER THE BUCKET NAME***]
   • fs.gs.working.dir=/
   • fs.gs.auth.service.account.enable=true
   • fs.gs.auth.service.account.email=[***ENTER THE SERVICE PRINCIPAL EMAIL ID***]
   • fs.gs.auth.service.account.keyfile=/[***ENTER THE LOCAL PATH OF THE P12 FILE***]
   You can add these key-value pairs to register a Google account to use in Replication Manager from Cloudera on cloud 7.2.16.1 and higher versions.
4. Add the following key-value pairs to register an S3 account to use in Replication Manager:
   • fs.s3a.access.key=[***ENTER THE SESSION ACCESS KEY***]
   • fs.s3a.secret.key=[***ENTER THE SESSION SECRET KEY***]
5. Add the following key-value pairs to register an ADLS account to use in Replication Manager:
   • fs.azure.account.oauth2.client.id=[***ENTER THE ABFS STORAGE CLIENT ID***]
   • fs.azure.account.oauth2.client.secret=[***ENTER THE ABFS STORAGE CLIENT SECRET KEY***]

6. Save and restart the HDFS service for the changes to take effect.

To ensure the YARN application succeeds, the hbase user must be allowed to impersonate the exportSnapshotUser in HDFS by configuring the required properties and values in the HDFS core_site_safety_valve of the source cluster.

For example, if you want to allow the impersonation from any host and if the exportSnapshotUser is in the repl user group, you can set the following key-value pairs in source Cloudera Manager > Clusters > CORE SETTINGS > Configuration > Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml:

• hadoop.proxyuser.hbase.groups = repl
• hadoop.proxyuser.hbase.hosts = *

The Export snapshot user field appears in the Create HBase replication policy wizard when the following conditions are true:

• The target cluster version is 7.2.18.200 and higher.
• The source cluster version is 7.1.9 SP1 using Cloudera Manager 7.11.3 CHF7 and higher.
• You choose the Select Source > Perform Initial Snapshot option in the Create HBase replication policy wizard.

When you use IDBroker credentials to replicate from Cloudera Base on premises clusters, you must map the Kerberos username to AWS in the [***USERNAME***]=[***ARN***] format in the source Cloudera Manager > Clusters > Knox service > Instances > Configuration > Knox IDBroker AWS User Mapping property, and allow the hbase user to impersonate this user in IDBroker.