Cloudera Docs

HDFS replication policy fails due to export HTTPS_PROXY environment variable

HDFS replication policies fail when the export HTTPS_PROXY environment variable is set to access AWS through proxy servers. How to resolve this issue?

Remedy

To resolve this issue, perform the following steps:
  1. Open the core-site.xml file on the source cluster.
  2. Enter the following properties in the file: 
    <property>
  <name>fs.s3a.proxy.host</name>
  <description>Hostname of the (optional) proxy server for S3 connections.</description>
</property>

<property>
  <name>fs.s3a.proxy.port</name>
  <description>Proxy server port. If this property is not set
    but fs.s3a.proxy.host is, port 80 or 443 is assumed (consistent with
    the value of fs.s3a.connection.ssl.enabled).</description>
</property>
  3. Save and close the file.
  4. Restart the source Cloudera Manager.
  5. Run the failed HDFS replication policies in Replication Manager.
    Replication Manager completes the replication successfully.