Cloudera Docs

Configuring properties for OBS bucket replication using Ozone replication policies

Before you replicate OBS buckets, you must configure additional properties that assist Ozone replication policies in CDP Private Cloud Base Replication Manager to replicate data in OBS buckets.

  1. Add the key-value pairs in the following table to the Ozone Client Advanced Configuration Snippet (Safety Valve) property in the ozone-site.xml file in the source cluster. Starting from CDP Private Cloud Base 7.1.9 SP1 using Cloudera Manager 7.11.3 CHF7, add the following key-value pairs to the ozone_replication_core_site_safety_valve property in the source cluster:
    Property Description
    fs.s3a.endpoint Enter the same value as in Ozone S3 gateway web UI as the source cluster.
    hadoop.tmp.dir Enter the temporary directory on the source cluster to buffer file uploads.

    Ensure that the user running the Ozone replication policy jobs has write access to the Hadoop temporary folder.

    fs.s3a.secret.key See Step 3 to get the required value.
    fs.s3a.access.key See Step 3 to get the required value.
    fs.s3a.impl Enter org.apache.hadoop.fs.s3a.S3AFileSystem.
    ozone.om.snapshot.load.native.lib

    (Available from 7.1.9 CHF3 onwards)

    		 Enter false.

    Incremental Ozone replication policy run uses snapshot-diff operation. This parameter ensures that the replication policy run is not affected if the snapshot-diff operation goes down during the replication policy run.

  2. Add the key-value pairs in the following table to the Ozone Client Advanced Configuration Snippet (Safety Valve) property in the ozone-site.xml file in the target cluster. Starting from CDP Private Cloud Base 7.1.9 SP1 using Cloudera Manager 7.11.3 CHF7, add the following key-value pairs to the ozone_replication_core_site_safety_valve property in the target cluster:
    Property Description
    fs.s3a.endpoint Enter the same value as in Ozone S3 gateway web UI as the target cluster.
    fs.s3a.secret.key See Step 3 to get the required value.
    fs.s3a.access.key See Step 3 to get the required value.
    fs.s3a.change.detection.version.required Enter false.
    fs.s3a.change.detection.mode Enter none.
    fs.s3a.path.style.access Enter true.
    fs.s3a.impl Enter org.apache.hadoop.fs.s3a.S3AFileSystem.
    hadoop.tmp.dir Enter the temporary directory on the target cluster to buffer file uploads.

    Ensure that the user running the Ozone replication policy jobs has write access to the Hadoop temporary folder.

    ozone.om.snapshot.load.native.lib

    (Available from 7.1.9 CHF3 onwards)

    		 Enter false.

    Incremental Ozone replication policy run uses snapshot-diff operation. This parameter ensures that the replication policy run is not affected if the snapshot-diff operation goes down during the replication policy run.

  3. If Kerberos is enabled on the source and target cluster, run the ozone s3 getsecret --om-service-id=serviceId command to get the secret and access key. Otherwise, enter any arbitrary value for the secret and access key.

    You can store the keys in a credstore such as JCEKS for non Auto-TLS clusters. After you store the keys, perform the following steps:

    1. Configure the credstore file path for the hadoop.security.credential.provider.path property in the ozone-site.xml file. For more information, see Using DistCp with Amazon S3.
    2. Add the HADOOP_CREDSTORE_PASSWORD parameter to the YARN Service Environment Advanced Configuration Snippet (Safety Valve) property for the YARN service in source Cloudera Manager.
  4. The /s3v volumes store S3 buckets. By default, you can access the buckets in /s3v volumes using S3 interface. To access other buckets through the S3 interface, you must create a “symbolic linked” bucket. You can use the ‘symbolic linked’ bucket in Ozone replication policies.
    Configure the required OBS buckets as S3-compatible buckets using the following commands before you use it in Ozone replication policies:
    1. ozone sh volume create /s3v
    2. ozone sh volume create /[***VOLUME NAME***]
    3. ozone sh bucket create /[***VOLUME NAME***]/[***BUCKET NAME***]
    4. ozone sh bucket link /[***VOLUME NAME***]/[***BUCKET NAME***] /s3v/[***SYMBOLIC LINKED BUCKET NAME***]
  5. Import the S3G CA certificate from the cluster to the local JDK path using the following commands:
    1. Run the keytool -importkeystore -destkeystore [***JDK CACERTS_LOCATION***] -srckeystore [***CM-AUTO-GLOBAL_TRUSTSTORE.JKS LOCATION***] -srcalias [***CM_ALIAS_ON_SRC_CM***] command on all the hosts of the source Cloudera Manager.
      For example, keytool -importkeystore -destkeystore /usr/java/default/lib/security/cacerts -srckeystore /var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks -srcalias cmrootca-0
    2. Run the following commands on all the hosts of the target Cloudera Manager:

      1. keytool -importkeystore -destkeystore [***JDK CACERTS LOCATION***] -srckeystore [***CM-AUTO-GLOBAL_TRUSTSTORE.JKS LOCATION***] -srcalias [***CM ALIAS ON SRC CM***]

      2. keytool -importkeystore -destkeystore [***JDK_CACERTS_LOCATION***] -srckeystore [***CM-AUTO-GLOBAL_TRUSTSTORE.JKS LOCATION***] -srcalias [***CM_ALIAS_ON_DEST_CM***]

      For example, 
      keytool -importkeystore -destkeystore /usr/java/default/lib/security/cacerts 
-srckeystore /var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks -srcalias cmrootca-0
keytool -importkeystore -destkeystore /usr/java/default/lib/security/cacerts 
-srckeystore /var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks -srcalias cmrootca-1