Enable SSL for WebHDFS, MapReduce, Tez, and YARN
This section explains how to set up SSL for WebHDFS, YARN and MapReduce. Before you begin, make sure that the SSL certificate is properly configured, including the keystore and truststore that will be used by WebHDFS, MapReduce, and YARN.
- One-way SSL: SSL client validates the server identity only.
- Mutual authentication (2WAY SSL): The server and clients validate each others' identities. 2WAY SSL can cause performance delays and is difficult to set up and maintain.
-
Set the following property values (or add the properties if required)
in core-site.xml:
hadoop.ssl.require.client.cert=false hadoop.ssl.hostname.verifier=DEFAULT hadoop.ssl.keystores.factory.class=org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory hadoop.ssl.server.conf=ssl-server.xml hadoop.ssl.client.conf=ssl-client.xml
NoteSpecify the hadoop.ssl.server.conf and hadoop.ssl.client.conf values as the relative or absolute path to Hadoop SSL Keystore Factory configuration files. If you specify only the file name, put the files in the same directory as the core-site.xml.
-
Set the following properties (or add the properties if required)
in hdfs-site.xml:
dfs.http.policy=<Policy> dfs.client.https.need-auth=true (optional for mutual client/server certificate validation) dfs.datanode.https.address=<hostname>:50475 dfs.namenode.https-address=<hostname>:50470
Where
<Policy>
is either:HTTP_ONLY
: service is provided only on HTTPHTTPS_ONLY
: service is provided only on HTTPSHTTP_AND_HTTPS
: service is provided both on HTTP and HTTPS
-
Set the following properties in mapred-site.xml:
mapreduce.jobhistory.http.policy=HTTPS_ONLY mapreduce.jobhistory.webapp.https.address=<JHS>:<JHS_HTTPS_PORT> mapreduce.ssl.enabled=true mapreduce.shuffle.ssl.enabled=true
-
Set the following properties in yarn-site.xml:
yarn.http.policy=HTTPS_ONLY yarn.log.server.url=https://<JHS>:<JHS_HTTPS_PORT>/jobhistory/logs yarn.resourcemanager.webapp.https.address=<RM>:<RM_HTTPS_PORT> yarn.nodemanager.webapp.https.address=0.0.0.0:<NM_HTTPS_PORT>
-
Create an
ssl-server.xml
file for the Hadoop SSL Keystore Factory:-
Copy the example SSL Server configuration file and modify the settings for
your environment:
cp /etc/hadoop/conf/ssl-server.xml.example /etc/hadoop/conf/ssl-server.xml
. -
Configure the server SSL properties:
Configuration Properties in ssl-server.xml Property Default Value Description ssl.server.keystore.type
JKS The type of the keystore, JKS = Java Keystore, the de-facto standard in Java ssl.server.keystore.location
None The location of the keystore file ssl.server.keystore.password
None The password to open the keystore file ssl.server.truststore.type
JKS The type of the trust store ssl.server.truststore.location
None The location of the truststore file ssl server.truststore.password
None The password to open the truststore For example:<property> <name>ssl.server.truststore.location</name> <value>/etc/security/serverKeys/truststore.jks</value> <description>Truststore to be used by NN and DN. Must be specified.</description> </property> <property> <name>ssl.server.truststore.password</name> <value>changeit</value> <description>Optional. Default value is "". </description> </property> <property> <name>ssl.server.truststore.type</name> <value>jks</value> <description>Optional. The keystore file format, default value is "jks".</description> </property> <property> <name>ssl.server.truststore.reload.interval</name> <value>10000</value> <description>Truststore reload check interval, in milliseconds. Default value is 10000 (10 seconds).</description> </property> <property> <name>ssl.server.keystore.location</name> <value>/etc/security/serverKeys/keystore.jks</value> <description>Keystore to be used by NN and DN. Must be specified.</description> </property> <property> <name>ssl.server.keystore.password</name> <value>changeit</value> <description>Must be specified.</description> </property> <property> <name>ssl.server.keystore.keypassword</name> <value>changeit</value> <description>Must be specified.</description> </property> <property> <name>ssl.server.keystore.type</name> <value>jks</value> <description>Optional. The keystore file format, default value is "jks".</description> </property>
-
Copy the example SSL Server configuration file and modify the settings for
your environment:
-
Create an
ssl-client.xml
file for the Hadoop SSL Keystore Factory:-
Copy the client truststore example file:
cp /etc/hadoop/conf/ssl-server.xml.example /etc/hadoop/conf/ssl-server.xml
-
Configure the client trust store values:
ssl.client.truststore.location=/etc/security/clientKeys/all.jks ssl.client.truststore.password=clientTrustStorePassword ssl.client.truststore.type=jks
-
Copy the client truststore example file:
-
Set the following properties in the
tez-site.xml
file:tez.runtime.shuffle.ssl.enable=true tez.runtime.shuffle.keep-alive.enabled=true
- Copy the configuration files (core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml, ssl-server.xml, tez-site.xml and ssl-client.xml), including the ssl-server and ssl-client store files if the Hadoop SSL Keystore Factory uses it's own keystore and truststore files, to all nodes in the cluster.
- Restart services on all nodes in the cluster.