HDFS replication policy fails due to export HTTPS_PROXY environment variable
HDFS replication policies fail when the export HTTPS_PROXY environment variable is set to access AWS through proxy servers. How to resolve this issue?
To resolve this issue, perform the following steps:
- Open the core-site.xml file on the source cluster.
Enter the following properties in the file:
<property> <name>fs.s3a.proxy.host</name> <description>Hostname of the (optional) proxy server for S3 connections.</description> </property> <property> <name>fs.s3a.proxy.port</name> <description>Proxy server port. If this property is not set but fs.s3a.proxy.host is, port 80 or 443 is assumed (consistent with the value of fs.s3a.connection.ssl.enabled).</description> </property>
- Save and close the file.
- Restart the source Cloudera Manager.
Run the failed HDFS replication policies in Replication Manager.
Replication Manager completes the replication successfully.