Cloudera Docs

Configuring properties for OBS bucket replication using Ozone replication policies

Before you replicate OBS buckets, you must configure additional properties that assist Ozone replication policies to replicate data in OBS buckets.

  1. Add the key-value pairs in the following table to the Ozone Client Advanced Configuration Snippet (Safety Valve) property in the ozone-site.xml file in the source cluster:
    Property Description
    fs.s3a.endpoint Enter the same value as in Ozone S3 gateway web UI as the source cluster.
    hadoop.tmp.dir Enter the temporary directory on the target cluster to buffer file uploads.
    fs.s3a.secret.key See Step 3 to get the required value.
    fs.s3a.access.key See Step 3 to get the required value.
  2. Add the key-value pairs in the following table to the Ozone Client Advanced Configuration Snippet (Safety Valve) property in the ozone-site.xml file in the target cluster:
    Property Description
    fs.s3a.endpoint Enter the same value as in Ozone S3 gateway web UI as the target cluster.
    fs.s3a.secret.key See Step 3 to get the required value.
    fs.s3a.access.key See Step 3 to get the required value.
    fs.s3a.change.detection.version.required Enter false.
    fs.s3a.change.detection.mode Enter none.
    fs.s3a.path.style.access Enter true.
  3. If Kerberos is enabled on the source and target cluster, run the ozone s3 getsecret --om-service-id=serviceId command to get the secret and access key. Otherwise, enter any arbitrary value for the secret and access key.
    You can store the keys in a credstore such as JCEKS for non Auto-TLS clusters. After you store the keys, perform the following steps:
    1. Configure the credstore file path for the hadoop.security.credential.provider.path property in the ozone-site.xml file. For more information, see Using DistCp with Amazon S3.
    2. Add the HADOOP_CREDSTORE_PASSWORD property to the YARN Service Environment Advanced Configuration Snippet (Safety Valve) for the YARN service in source Cloudera Manager.