Hadoop Security Guide
Also available as:
PDF
loading table of contents...

Create and Set Up an Internal CA (OpenSSL)

OpenSSL provides tools to allow you to create your own private certificate authority.

Considerations:

  • The encryption algorithms may be less secure than a well-known, trusted third-party.

  • Unknown CAs require that the certificate be installed in corresponding client truststores.

    [Note]Note

    When accessing the service from a client application such as HiveCLI or cURL, the CA must resolve on the client side or the connection attempt may fail. Users accessing the service through a browser will be able to add an exception if the certificate cannot be verified in their local truststore.

Prerequisite: Install openssl. For example, on CentOS run yum install openssl.

To create and set up a CA:

  1. Generate the key and certificate for a component process.

    The first step in deploying HTTPS for a component process (for example, Kafka broker) is to generate the key and certificate for each node in the cluster. You can use the Java keytool utility to accomplish this task. Start with a temporary keystore, so that you can export and sign it later with the CA.

    Use the following keytool command to create the key and certificate:

    $ keytool -keystore <keystore-file> -alias localhost -validity <validity> -genkey

    where:

    <keystore-file> is the keystore file that stores the certificate. The keystore file contains the private key of the certificate; therefore, it needs to be kept safely.

    <validity> is the length of time (in days) that the certificate will be valid.

    Make sure that the common name (CN) matches the fully qualified domain name (FQDN) of the server. The client compares the CN with the DNS domain name to ensure that it is indeed connecting to the desired server, not a malicious server.

  2. Create the Certificate Authority (CA)

    After step 1, each machine in the cluster has a public-private key pair and a certificate that identifies the machine. The certificate is unsigned, however, which means that an attacker can create such a certificate to pretend to be any machine.

    To prevent forged certificates, it is very important to sign the certificates for each machine in the cluster.

    A CA is responsible for signing certificates, and associated cryptography guarantees that a signed certificate is computationally difficult to forge. Thus, as long as the CA is a genuine and trusted authority, the clients have high assurance that they are connecting to the machines that they are attempting to connect with.

    Here is a sample openssl command to generate a CA:

    openssl req -new -x509 -keyout ca-key -out ca-cert -days 365

    The generated CA is simply a public-private key pair and certificate, intended to sign other certificates.

  3. Add the generated CA to the server's truststore:

    keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
  4. Add the generated CA to the client's truststore, so that clients know that they can trust this CA:

    keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert

    In contrast to the keystore in step 1 that stores each machine’s own identity, the truststore of a client stores all of the certificates that the client should trust. Importing a certificate into one’s truststore also means trusting all certificates that are signed by that certificate.

    Trusting the CA means trusting all certificates that it has issued. This attribute is called a "chain of trust," and is particularly useful when deploying SSL on a large cluster. You can sign all certificates in the cluster with a single CA, and have all machines share the same truststore that trusts the CA. That way all machines can authenticate all other machines.

  5. Sign all certificates generated in Step 1 with the CA generated in Step 2:

    1. Export the certificate from the keystore:

      keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file
    2. Sign the certificate with the CA:

      openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days <validity> -CAcreateserial -passin pass:<ca-password>
  6. Import the CA certificate and the signed certificate into the keystore. For example:

    $ keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert 
    $ keytool -keystore server.keystore.jks -alias localhost -import -file cert-signed

    The parameters are defined as follows:

    ParameterDescription
    keystoreThe location of the keystore
    ca-certThe certificate of the CA
    ca-keyThe private key of the CA
    ca-passwordThe passphrase of the CA
    cert-fileThe exported, unsigned certificate of the server
    cert-signedThe signed certificate of the server

All of the preceding steps can be placed into a bash script.

In the following example, note that one of the commands assumes a password of test1234. Specify your own password before running the script.

#!/bin/bash
                        
#Step 1
keytool -keystore server.keystore.jks -alias localhost -validity 365 -genkey

#Step 2
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert

#Step 3 
keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 365 -CAcreateserial -passin pass:test1234
keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.keystore.jks -alias localhost -import -file cert-signed

To finish the setup process:

  1. Set up the CA directory structure:

     mkdir -m 0700 /root/CA /root/CA/certs /root/CA/crl /root/CA/newcerts /root/CA/private
  2. Move the CA key to  /root/CA/private  and the CA certificate to  /root/CA/certs.

    mv ca.key /root/CA/private;mv ca.crt /root/CA/certs
  3. Add required files:

    touch /root/CA/index.txt; echo 1000 >> /root/CA/serial 
  4. Set permissions on the ca.key:

    chmod 0400 /root/ca/private/ca.key
  5. Open the OpenSSL configuration file:

    vi /etc/pki/tls/openssl.cnf
  6. Change the directory paths to match your environment:

    [ CA_default ]
    
    dir             = /root/CA                  # Where everything is kept
    certs           = /root/CA/certs            # Where the issued certs are kept
    crl_dir         = /root/CA/crl              # Where the issued crl are kept
    database        = /root/CA/index.txt        # database index file.
    #unique_subject = no                        # Set to 'no' to allow creation of
                                                # several certificates with same subject.
    new_certs_dir   = /root/CA/newcerts         # default place for new certs.
    
    certificate     = /root/CA/cacert.pem       # The CA certificate
    serial          = /root/CA/serial           # The current serial number
    crlnumber       = /root/CA/crlnumber        # the current crl number
                                                # must be commented out to leave a V1 CRL
    crl             = $dir/crl.pem               # The current CRL
    private_key     = /root/CA/private/cakey.pem # The private key
    RANDFILE        = /root/CA/private/.rand     # private random number file
    
    x509_extensions = usr_cert              # The extensions to add to the cert
  7. Save the changes and restart OpenSSL.

Example of setting up an OpenSSL internal CA:

openssl genrsa -out ca.key 8192; openssl req -new -x509 -extensions v3_ca -key ca.key -out ca.crt -days 365

Generating RSA private key, 8192 bit long modulus
.......................................................................................++
......................++
e is 65537 (0x10001)
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.

Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:California
Locality Name (eg, city) [Default City]:SantaClara
Organization Name (eg, company) [Default Company Ltd]:Hortonworks
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:nn
Email Address []:it@hortonworks.com

mkdir -m 0700 /root/CA /root/CA/certs /root/CA/crl /root/CA/newcerts /root/CA/private
ls /root/CA
certs  crl  newcerts  private