Enable SSL for WebHDFS, MapReduce Shuffle, Tez, and YARN

This section explains how to set up SSL for WebHDFS, YARN and MapReduce. Before you begin, make sure that the SSL certificate is properly configured, including the keystore and truststore that will be used by WebHDFS, MapReduce, and YARN.

HDP supports the following SSL modes:

One-way SSL: SSL client validates the server identity only.
Mutual authentication (2WAY SSL): The server and clients validate each others' identities. 2WAY SSL can cause performance delays and is difficult to set up and maintain.

	Note
	In order to access SSL enabled HDP Services through the Knox Gateway, additional configuration on the Knox Gateway is required, see Configure SSL for Knox.

To enable one-way SSL set the following properties and restart all services:

Set the following property values (or add the properties if required) in core-site.xml:

hadoop.ssl.require.client.cert=false

hadoop.ssl.hostname.verifier=DEFAULT

hadoop.ssl.keystores.factory.class=org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory

hadoop.ssl.server.conf=ssl-server.xml

hadoop.ssl.client.conf=ssl-client.xml

	Note
	Specify the hadoop.ssl.server.conf and hadoop.ssl.client.conf values as the relative or absolute path to Hadoop SSL Keystore Factory configuration files. If you specify only the file name, put the files in the same directory as the `core-site.xml`.

Set the following properties (or add the properties if required) in hdfs-site.xml:
- dfs.http.policy=<Policy>
- dfs.client.https.need-auth=true (optional for mutual client/server certificate validation)
- dfs.datanode.https.address=<hostname>:50475
- dfs.namenode.https-address=<hostname>:50470
  where <Policy> is either:
  - HTTP_ONLY: service is provided only on HTTP
  - HTTPS_ONLY: service is provided only on HTTPS
  - HTTP_AND_HTTPS: service is provided both on HTTP and HTTPS

Set the following properties in mapred-site.xml:

mapreduce.jobhistory.http.policy=HTTPS_ONLY
mapreduce.jobhistory.webapp.https.address=<JHS>:<JHS_HTTPS_PORT>
mapreduce.ssl.enabled=true
mapreduce.shuffle.ssl.enabled=true

Set the following properties in yarn-site.xml:

yarn.http.policy=HTTPS_ONLY
yarn.log.server.url=https://<JHS>:<JHS_HTTPS_PORT>/jobhistory/logs
yarn.resourcemanager.webapp.https.address=<RM>:<RM_HTTPS_PORT> 
yarn.nodemanager.webapp.https.address=0.0.0.0:<NM_HTTPS_PORT>

Create an ssl-server.xml file for the Hadoop SSL Keystore Factory:

Copy the example SSL Server configuration file and modify the settings for your environment:
```
cp /etc/hadoop/conf/ssl-server.xml.example /etc/hadoop/conf/ssl-server.xml
```

Configure the server SSL properties:

Table 4.3. Configuration Properties in ssl-server.xml
Property	Default Value	Description
`ssl.server.keystore.type`	JKS	The type of the keystore, JKS = Java Keystore, the de-facto standard in Java
`ssl.server.keystore.location`	None	The location of the keystore file
`ssl.server.keystore.password`	None	The password to open the keystore file
`ssl.server.truststore.type`	JKS	The type of the trust store
`ssl.server.truststore.location`	None	The location of the truststore file
`ssl server.truststore.password`	None	The password to open the truststore

For example:

<property>
  <name>ssl.server.truststore.location</name>
  <value>/etc/security/serverKeys/truststore.jks</value>
  <description>Truststore to be used by NN and DN. Must be specified.</description>
</property>

<property>
  <name>ssl.server.truststore.password</name>
  <value>changeit</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.server.truststore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".</description>
</property>

<property>
  <name>ssl.server.truststore.reload.interval</name>
  <value>10000</value>
  <description>Truststore reload check interval, in milliseconds.
  Default value is 10000 (10 seconds).</description>
</property>

<property>
  <name>ssl.server.keystore.location</name>
  <value>/etc/security/serverKeys/keystore.jks</value>
  <description>Keystore to be used by NN and DN. Must be specified.</description>
</property>

<property>
  <name>ssl.server.keystore.password</name>
  <value>changeit</value>
  <description>Must be specified.</description>
</property>

<property>
  <name>ssl.server.keystore.keypassword</name>
  <value>changeit</value>
  <description>Must be specified.</description>
</property>

<property>
  <name>ssl.server.keystore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".</description>
</property>

Create an ssl-client.xml file for the Hadoop SSL Keystore Factory:

Copy the client truststore example file:
```
cp /etc/hadoop/conf/ssl-server.xml.example /etc/hadoop/conf/ssl-server.xml
```

Configure the client trust store values:

ssl.client.truststore.location=/etc/security/clientKeys/all.jks
ssl.client.truststore.password=clientTrustStorePassword
ssl.client.truststore.type=jks

Set the following properties in the tez-site.xml file:

tez.runtime.shuffle.ssl.enable=true
tez.runtime.shuffle.keep-alive.enabled=true

Copy the configuration files (core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml, ssl-server.xml, tez-site.xml and ssl-client.xml), including the ssl-server and ssl-client store files if the Hadoop SSL Keystore Factory uses it's own keystore and truststore files, to all nodes in the cluster.
Restart services on all nodes in the cluster.

​Enable SSL for WebHDFS, MapReduce Shuffle, Tez, and YARN

Enable SSL for WebHDFS, MapReduce Shuffle, Tez, and YARN