7. Set up WebHDFS/YARN with SSL (HTTPS)

This section explains how to set up HTTPS encryption for the Web interfaces.

7.1. Install an SSL Certificate

You can use either a certificate from a Certificate Authority or a Self-Signed Certificate. Using a self-signed certificate requires some additional configuration on each host. Follow the instructions in the appropriate section to install a certificate.

7.1.1. Use a Self-Signed Certificate

To set up SSL for Hadoop HDFS operations:

Create HTTPS certificates and keystore/truststore files.

For each host in the cluster, create a directory for storing the keystore and truststore. For example, SERVER_KEY_LOCATION. Also create a directory to store public certificate, for example, CLIENT_KEY_LOCATION.
```
mkdir -p $SERVER_KEY_LOCATION ; mkdir -p $CLIENT_KEY_LOCATION
```
For example:
```
ssh host1.hwx.com “mkdir -p /etc/security/serverKeys ; mkdir -p /etc/security/clientKeys ; ” 
```

For each host, create a keystore file.

cd $SERVER_KEY_LOCATION ; keytool -genkey -alias $hostname -keyalg RSA -keysize 1024 -dname \"CN=$hostname,OU=hw,O=hw,L=paloalto,ST=ca,C=us\" -keypass $SERVER_KEYPASS_PASSWORD -keystore $KEYSTORE_FILE -storepass $SERVER_STOREPASS_PASSWORD\”

For each host, export the certificate public key to a certificate file.

cd $SERVER_KEY_LOCATION ; keytool -export -alias $hostname -keystore $KEYSTORE_FILE -rfc -file $CERTIFICATE_NAME -storepass $SERVER_STOREPASS_PASSWORD\”

For each host, import the certificate into truststore file.

cd $SERVER_KEY_LOCATION ; keytool -import -noprompt -alias $hostname -file $CERTIFICATE_NAME -keystore $TRUSTSTORE_FILE -storepass $SERVER_TRUSTSTORE_PASSWORD

Create a single truststore file containing the public key from all certificates. Login to host1 and import the truststore file for host1.
```
keytool -import -noprompt -alias $host -file $CERTIFICATE_NAME -keystore $ALL_JKS -storepass $CLIENT_TRUSTSTORE_PASSWORD
```

Copy $ALL_JKS from host1 to other hosts, and repeat the above command. For example, for a 2-node cluster with host1 and host2:

Create $ALL_JKS on host1.

keytool -import -noprompt -alias $host -file $CERTIFICATE_NAME -keystore $ALL_JKS -storepass $CLIENT_TRUSTSTORE_PASSWORD

Copy over $ALL_JKS from host1 to host2. $ALL_JKS already has the certificate entry of host1.
Import certificate entry of host2 to $ALL_JKS using same command as before:
```
keytool -import -noprompt -alias $host -file $CERTIFICATE_NAME -keystore $ALL_JKS -storepass $CLIENT_TRUSTSTORE_PASSWORD
```

Copy over the updated $ALL_JKS from host2 to host1.

	Note
	Repeat these steps each time for each node in the cluster. When you are finished, the $ALL_JKS file on host1 will have the certificates of all nodes.

Copy over the $ALL_JKS file from host1 to all the nodes.

Validate the common truststore file on all hosts.

keytool -list -v -keystore $ALL_JKS -storepass $CLIENT_TRUSTSTORE_PASSWORD

Set permissions and ownership on the keys:

chgrp -R $YARN_USER:hadoop $SERVER_KEY_LOCATION
chgrp -R $YARN_USER:hadoop $CLIENT_KEY_LOCATION
chown 755 $SERVER_KEY_LOCATION
chown 755 $CLIENT_KEY_LOCATION
chown 440 $KEYSTORE_FILE
chown 440 $TRUSTSTORE_FILE 
chown 440 $CERTIFICATE_NAME
chown 444 $ALL_JKS

	Note
	The complete path of the `$SEVER_KEY_LOCATION` and the `CLIENT_KEY_LOCATION` from the root directory /etc must be owned by the $YARN_USER user and the hadoop group.

7.1.2. Use a CA Signed Certificate

Run the following command to create a self-signing rootCA and import the rootCA into client truststore:
```
openssl genrsa -out $clusterCA.key 2048
openssl req -x509 -new -key $clusterCA.key -days 300 -out $clusterCA.pem
keytool -importcert -alias $clusterCA -file $clusterCA.pem -keystore $clustertruststore -storepass $clustertruststorekey
```
Note
Ensure that the ssl-client.xml on every host configure to use this ‘$clustertrust’ store.

On each host, run the following command to create a certifcate and a keystore for each server:

keytool -genkeypair -alias `hostname -s` -keyalg RSA -keysize 1024 -dname "CN=`hostname -f`,OU=foo,O=corp” -keypass $hostkey -keystore $hostkeystore -storepass $hoststorekey -validity 300

On each host, run the following command to export a certreq file from the host’s keystore:
```
keytool -keystore keystore -alias `hostname -s` -certreq -file $host.cert -storepass $hoststorekey -keypass $hostkey
```

On each host, sign certreq file with the rootCA:

openssl x509 -req -CA $clusterCA.pem -CAkey $clusterCA.key -in $host.cert -out $host.signed -days 300 -CAcreateserial

On each host, import both rootCA and the signed cert back in:

keytool -keystore $hostkeystore -storepass $hoststorekey -alias $clusterCA -import -file cluseter1CA.pem
keytool -keystore $hostkeystore -storepass $hoststorekey -alias `hostname -s` -import -file $host.signed -keypass $hostkey

Legal notices

	Note
	Ensure that the ssl-client.xml on every host configure to use this ‘$clustertrust’ store.