Enable SSL for WebHDFS, MapReduce Shuffle, Tez, and YARN
This section explains how to set up SSL for WebHDFS, YARN and MapReduce. Before you begin, make sure that the SSL certificate is properly configured, including the keystore and truststore that will be used by WebHDFS, MapReduce, and YARN.
HDP supports the following SSL modes:
One-way SSL: SSL client validates the server identity only.
Mutual authentication (2WAY SSL): The server and clients validate each others' identities. 2WAY SSL can cause performance delays and is difficult to set up and maintain.
Note | |
---|---|
In order to access SSL enabled HDP Services through the Knox Gateway, additional configuration on the Knox Gateway is required, see Configure SSL for Knox. |
To enable one-way SSL set the following properties and restart all services:
Set the following property values (or add the properties if required) in
core-site.xml
:hadoop.ssl.require.client.cert=false
hadoop.ssl.hostname.verifier=DEFAULT
hadoop.ssl.keystores.factory.class=org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory
hadoop.ssl.server.conf=ssl-server.xml
hadoop.ssl.client.conf=ssl-client.xml
Note Specify the hadoop.ssl.server.conf and hadoop.ssl.client.conf values as the relative or absolute path to Hadoop SSL Keystore Factory configuration files. If you specify only the file name, put the files in the same directory as the
core-site.xml
.Set the following properties (or add the properties if required) in
hdfs-site.xml
:dfs.http.policy=<Policy>
dfs.client.https.need-auth=true
(optional for mutual client/server certificate validation)dfs.datanode.https.address=<hostname>:50475
dfs.namenode.https-address=<hostname>:50470
where
<Policy>
is either:HTTP_ONLY
: service is provided only on HTTPHTTPS_ONLY
: service is provided only on HTTPSHTTP_AND_HTTPS
: service is provided both on HTTP and HTTPS
Set the following properties in
mapred-site.xml
:mapreduce.jobhistory.http.policy=HTTPS_ONLY mapreduce.jobhistory.webapp.https.address=<JHS>:<JHS_HTTPS_PORT>
Set the following properties in
yarn-site.xml
:yarn.http.policy=HTTPS_ONLY yarn.log.server.url=https://<JHS>:<JHS_HTTPS_PORT>/jobhistory/logs yarn.resourcemanager.webapp.https.address=<RM>:<RM_HTTPS_PORT> yarn.nodemanager.webapp.https.address=0.0.0.0:<NM_HTTPS_PORT>
Create an
ssl-server.xml
file for the Hadoop SSL Keystore Factory:Copy the example SSL Server configuration file and modify the settings for your environment:
cp /etc/hadoop/conf/ssl-server.xml.example /etc/hadoop/conf/ssl-server.xml
Configure the server SSL properties:
Table 4.3. Configuration Properties in ssl-server.xml Property Default Value Description ssl.server.keystore.type
JKS The type of the keystore, JKS = Java Keystore, the de-facto standard in Java ssl.server.keystore.location
None The location of the keystore file ssl.server.keystore.password
None The password to open the keystore file ssl.server.truststore.type
JKS The type of the trust store ssl.server.truststore.location
None The location of the truststore file ssl server.truststore.password
None The password to open the truststore For example:
<property> <name>ssl.server.truststore.location</name> <value>/etc/security/serverKeys/truststore.jks</value> <description>Truststore to be used by NN and DN. Must be specified.</description> </property> <property> <name>ssl.server.truststore.password</name> <value>changeit</value> <description>Optional. Default value is "". </description> </property> <property> <name>ssl.server.truststore.type</name> <value>jks</value> <description>Optional. The keystore file format, default value is "jks".</description> </property> <property> <name>ssl.server.truststore.reload.interval</name> <value>10000</value> <description>Truststore reload check interval, in milliseconds. Default value is 10000 (10 seconds).</description> </property> <property> <name>ssl.server.keystore.location</name> <value>/etc/security/serverKeys/keystore.jks</value> <description>Keystore to be used by NN and DN. Must be specified.</description> </property> <property> <name>ssl.server.keystore.password</name> <value>changeit</value> <description>Must be specified.</description> </property> <property> <name>ssl.server.keystore.keypassword</name> <value>changeit</value> <description>Must be specified.</description> </property> <property> <name>ssl.server.keystore.type</name> <value>jks</value> <description>Optional. The keystore file format, default value is "jks".</description> </property>
Create an
ssl-client.xml
file for the Hadoop SSL Keystore Factory:Copy the client truststore example file:
cp /etc/hadoop/conf/ssl-server.xml.example /etc/hadoop/conf/ssl-server.xml
Configure the client trust store values:
ssl.client.truststore.location=/etc/security/clientKeys/all.jks ssl.client.truststore.password=clientTrustStorePassword ssl.client.truststore.type=jks
Set the following properties in the
tez-site.xml
file:tez.runtime.shuffle.ssl.enable=true tez.runtime.shuffle.keep-alive.enabled=true
Copy the configuration files (
core-site.xml
,hdfs-site.xml
,mapred-site.xml
,yarn-site.xml
,ssl-server.xml
,tez-site.xml
andssl-client.xml
), including the ssl-server and ssl-client store files if the Hadoop SSL Keystore Factory uses it's own keystore and truststore files, to all nodes in the cluster.Restart services on all nodes in the cluster.