Enabling HDFS Encryption Using the Wizard

To accommodate the security best practice of separation of duties, enabling HDFS encryption using the wizard requires different Cloudera Manager user roles for different steps.

Launch the Set up HDFS Data At Rest Encryption wizard in one of the following ways:

Cluster > Set up HDFS Data At Rest Encryption
Minimum Required Role: Key Administrator or Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.
Administration > Security > Set up HDFS Data At Rest Encryption
Minimum Required Role: Key Administrator or Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.
HDFS service > Actions > Set up HDFS Data At Rest Encryption
Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

On the first page of the wizard, select the root of trust for encryption keys:

Cloudera Navigator Key Trustee Server
A file-based password-protected Java KeyStore

Cloudera strongly recommends using Cloudera Navigator Key Trustee Server as the root of trust for production environments. The file-based Java KeyStore root of trust is insufficient to provide the security, scalability, and manageability required by most production systems. More specifically, the Java KeyStore KMS does not provide:

Scalability, so you are limited to only one KMS, which can result in bottlenecks
High Availability (HA)
Recoverability, so if you lose the node where the Java KeyStore is stored, then you can lose access to all the encrypted data

Ultimately, the Java KeyStore does not satisfy the stringent security requirements of most organizations for handling master encryption keys.

Choosing a root of trust displays a list of steps required to enable HDFS encryption using that root of trust. Each step can be completed independently. The Status column indicates whether the step has been completed, and the Notes column provides additional context for the step. If your Cloudera Manager user account does not have sufficient privileges to complete a step, the Notes column indicates the required privileges.

Available steps contain links to wizards or documentation required to complete the step. If a step is unavailable due to insufficient privileges or a prerequisite step being incomplete, no links are present and the Notes column indicates the reason the step is unavailable.

Continue to the section for your selected root of trust for further instructions:

Enabling HDFS Encryption Using Navigator Key Trustee Server

Enabling HDFS encryption using Key Trustee Server as the key store involves multiple components. For an overview of the components involved in encrypting data at rest, see Encrypting Data at Rest. For guidelines on deploying the Navigator Key Trustee Server in production environments, Data at Rest Encryption Requirements.

Before continuing, make sure the Cloudera Manager server host has access to the internal repository hosting the Key Trustee Server software.

After selecting Cloudera Navigator Key Trustee Server as the root of trust, the following steps are displayed:

1. Enable Kerberos

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

2. Enable TLS/SSL

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

3. Add a dedicated cluster for the Key Trustee Server

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

If you haven't already done so, you must create an internal repository to install Cloudera Navigator before you can set up and use Navigator Key Trustee Server.

This step creates a new cluster in Cloudera Manager for the Key Trustee Server hosts to isolate them from other enterprise data hub (EDH) services for increased security and durability. For more information, see Data at Rest Encryption Reference Architecture.

To complete this step:

Click Add a dedicated cluster for the Key Trustee Server.
Leave Enable High Availability checked to add two hosts to the cluster. For production environments, you must enable high availability for Key Trustee Server. Failure to enable high availability can result in complete data loss in the case of catastrophic failure of a standalone Key Trustee Server. Click Continue.
Search for new hosts to add to the cluster, or select the Currently Managed Hosts tab to add existing hosts to the cluster. After selecting the hosts, click Continue.
Select the KEYTRUSTEE_SERVER parcel to install Key Trustee Server using parcels, or select None if you want to use packages. If you do not see a parcel available, click More Options and add the repository URL to the Remote Parcel Repository URLs list. After selecting a parcel or None, click Continue.
If you selected None, click Continue again, and skip to 4. Install Key Trustee Server binary using packages or parcels.
After the KEYTRUSTEE_SERVER parcel is successfully downloaded, distributed, unpacked, and activated, click Continue.
Click Continue to complete this step and return to the main page of the wizard.

4. Install Key Trustee Server binary using packages or parcels

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

If you haven't already done so, you must create an internal repository to install Cloudera Navigator before you can set up and use Navigator Key Trustee Server.

This step is completed automatically during 3. Add a dedicated cluster for the Key Trustee Server if you are using parcels. If the step is incomplete for any reason (such as the wizard being interrupted or a failure installing the parcel), complete it manually:

Click Install Key Trustee Server binary using packages or parcels.
Select the KEYTRUSTEE_SERVER parcel to install Key Trustee Server, or select None if you need to install Key Trustee Server manually using packages. If you do not see a parcel available, click More Options and add the repository URL to the Remote Parcel Repository URLs list. After selecting a parcel, click Continue.
After the KEYTRUSTEE_SERVER parcel is successfully downloaded, distributed, unpacked, and activated, click Finish to complete this step and return to the main page of the wizard.

5. Install Parcel for Key Trustee KMS

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

If you haven't already done so, you must create an internal repository to install Cloudera Navigator before you can set up and use Navigator Key Trustee Server.

This step installs the Key Trustee KMS parcel. If you are using packages, skip this step and see Installing Cloudera Navigator Key Trustee Server for instructions. After installing Key Trustee KMS using packages, continue to 6. Add a Key Trustee Server Service.

To complete this step for parcel-based installations:

Click Install Parcel for Key Trustee KMS.
Select the KEYTRUSTEE parcel to install Key Trustee KMS. If you do not see a parcel available, click More Options and add the repository URL to the Remote Parcel Repository URLs list. After selecting a parcel, click Continue.
After the KEYTRUSTEE parcel is successfully downloaded, distributed, unpacked, and activated, click Finish to complete this step and return to the main page of the wizard.

6. Add a Key Trustee Server Service

Minimum Required Role: Key Administrator (also provided by Full Administrator)

This step adds the Key Trustee Server service to Cloudera Manager. To complete this step:

Click Add a Key Trustee Server Service.
Click Continue.
On the Customize Role Assignments for Key Trustee Server page, select the hosts for the Active Key Trustee Server and Passive Key Trustee Server roles. Make sure that the selected hosts are not used for other services (see Resource Planning for Data at Rest Encryption for more information), and click Continue.
The Entropy Considerations page provides commands to install the rng-tools package to increase available entropy for cryptographic operations. For more information, see Data at Rest Encryption Requirements. After completing these commands, click Continue.
The Synchronize Active and Passive Key Trustee Server Private Keys page provides instructions for generating and copying the Active Key Trustee Server private key to the Passive Key Trustee Server. Cloudera recommends following security best practices and transferring the private key using offline media, such as a removable USB drive. For convenience (for example, in a development or testing environment where maximum security is not required), you can copy the private key over the network using the provided rsync command.
After you have synchronized the private keys, run the ktadmin init command on the Passive Key Trustee Server as described in the wizard. After the initialization is complete, check the box to indicate you have synchronized the keys and click Continue in the wizard.
The Setup TLS for Key Trustee Server page provides instructions on replacing the auto-generated self-signed certificate with a production certificate from a trusted Certificate Authority (CA). For more information, see Managing Key Trustee Server Certificates. Click Continue to view and modify the default certificate settings.
On the Review Changes page, you can view and modify the following settings:
- Database Storage Directory (db_root)
  Default value: /var/lib/keytrustee/db
  The directory on the local filesystem where the Key Trustee Server database is stored. Modify this value to store the database in a different directory.
- Active Key Trustee Server TLS/SSL Server Private Key File (PEM Format) (ssl.privatekey.location)
  Default value: /var/lib/keytrustee/.keytrustee/.ssl/ssl-cert-keytrustee-pk.pem
  The path to the Active Key Trustee Server TLS certificate private key. Accept the default setting to use the auto-generated private key. If you have a CA-signed certificate, change this path to the CA-signed certificate private key file. This file must be in PEM format.
- Active Key Trustee Server TLS/SSL Server Certificate File (PEM Format) (ssl.cert.location)
  Default value: /var/lib/keytrustee/.keytrustee/.ssl/ssl-cert-keytrustee.pem
  The path to the Active Key Trustee Server TLS certificate. Accept the default setting to use the auto-generated self-signed certificate. If you have a CA-signed certificate, change this to the path to the CA-signed certificate. This file must be in PEM format.
- Active Key Trustee Server TLS/SSL Server CA Certificate (PEM Format) (ssl.cacert.location)
  Default value: (none)
  The path to the file containing the CA certificate and any intermediate certificates (if any intermediate certificates exist, then they are required here) used to sign the Active Key Trustee Server certificate. If you have a CA-signed certificate, set this value to the path to the CA certificate or certificate chain file. This file must be in PEM format.
- Active Key Trustee Server TLS/SSL Private Key Password (ssl.privatekey.password)
  Default value: (none)
  The password for the Active Key Trustee Server private key file. Leave this blank if the file is not password-protected.
- Passive Key Trustee Server TLS/SSL Server Private Key File (PEM Format) (ssl.privatekey.location)
  Default value: /var/lib/keytrustee/.keytrustee/.ssl/ssl-cert-keytrustee-pk.pem
  The path to the Passive Key Trustee Server TLS certificate private key. Accept the default setting to use the auto-generated private key. If you have a CA-signed certificate, change this path to the CA-signed certificate private key file. This file must be in PEM format.
- Passive Key Trustee Server TLS/SSL Server Certificate File (PEM Format) (ssl.cert.location)
  Default value: /var/lib/keytrustee/.keytrustee/.ssl/ssl-cert-keytrustee.pem
  The path to the Passive Key Trustee Server TLS certificate. Accept the default setting to use the auto-generated self-signed certificate. If you have a CA-signed certificate, change this to the path to the CA-signed certificate. This file must be in PEM format.
- Passive Key Trustee Server TLS/SSL Server CA Certificate (PEM Format) (ssl.cacert.location)
  Default value: (none)
  The path to the file containing the CA certificate and any intermediate certificates (if any intermediate certificates exist, then they are required here) used to sign the Passive Key Trustee Server certificate. If you have a CA-signed certificate, set this value to the path to the CA certificate or certificate chain file. This file must be in PEM format.
- Passive Key Trustee Server TLS/SSL Private Key Password (ssl.privatekey.password)
  Default value: (none)
  The password for the Passive Key Trustee Server private key file. Leave this blank if the file is not password-protected.
After reviewing the settings and making any changes, click Continue.
After all commands complete successfully, click Continue. If the Generate Key Trustee Server Keyring appears stuck, make sure that the Key Trustee Server host has enough entropy. See Data at Rest Encryption Requirements for more information.
Click Finish to complete this step and return to the main page of the wizard.

7. Add a Key Trustee KMS Service

Minimum Required Role: Key Administrator (also provided by Full Administrator)

This step adds a Key Trustee KMS service to the cluster. The Key Trustee KMS service is required to enable HDFS encryption to use Key Trustee Server for cryptographic key management. Key Trustee KMS high availability uses ZooKeeper to automatically configure load balancing.

To complete this step:

Click Add a Key Trustee KMS Service.
Select an existing Key Trustee Server pair or specify an external Key Trustee Server pair. If you have an existing Key Trustee Server pair outside of Cloudera Manager control, select the External Key Trustee Server option and specify the fully-qualified domain names (FQDNs) of the Key Trustee Server pair. Click Continue.
Select cluster hosts for the Key Trustee KMS service. For production environments, select at least two hosts for high availability. If you proceed with only one host, you can enable high availability later.
Make sure that the selected hosts are not used for other services (see Resource Planning for Data at Rest Encryption for more information), and click Continue.
The Entropy Considerations page provides commands to install the rng-tools package to increase available entropy for cryptographic operations. For more information, see Data at Rest Encryption Requirements. After completing these commands, click Continue.
The Setup Organization and Auth Secret page generates the necessary commands to create an organization in Key Trustee Server. An organization is required to be able to register the Key Trustee KMS with Key Trustee Server.
Enter an organization name and click Generate Instruction. Run the displayed commands to generate an organization and obtain the auth_secret value for the organization. Enter the secret in the auth_secret field and click Continue.
The Setup Access Control List (ACL) page allows you to generate ACLs for the Key Trustee KMS or to provide your own ACLs. To generate the recommended ACLs, enter the username and group responsible for managing cryptographic keys and click Generate ACLs. To specify your own ACLs, select the Use Your Own kms-acls.xml File option and enter the ACLs. For more information on the KMS Access Control List, see Configuring KMS Access Control Lists (ACLs).
After generating or specifying the ACL, click Continue.
The Setup TLS for Key Trustee KMS page provides high-level instructions for configuring TLS communication between the Key Trustee KMS and the Key Trustee Server, as well as between the EDH cluster and the Key Trustee KMS. See Securing the Key Management Server (KMS) for more information.
Click Continue.
The Review Changes page lists all of the settings configured in this step. Click the ? icon next to any setting for information about that setting. Review the settings and click Continue.
After the First Run commands have successfully completed, click Continue.
The Synchronize Private Keys and HDFS Dependency page provides instructions for copying the private key from one Key Management Server Proxy role to all other roles.
caution
It is very important that you perform this step. Failure to do so leaves Key Trustee KMS in a state where keys are intermittently inaccessible, depending on which Key Trustee KMS host a client interacts with, because cryptographic key material encrypted by one Key Trustee KMS host cannot be decrypted by another. If you are already running multiple Key Trustee KMS hosts with different private keys, immediately back up (Backing up Key Trustee Server and Clients) all Key Trustee KMS hosts, and contact Cloudera Support for assistance correcting the issue.

To determine whether the Key Trustee KMS private keys are different, compare the MD5 hash of the private keys. On each Key Trustee KMS host, run the following command:
```
md5sum /var/lib/kms-keytrustee/keytrustee/.keytrustee/secring.gpg
```
If the outputs are different, contact Cloudera Support for assistance. Do not attempt to synchronize existing keys. If you overwrite the private key and do not have a backup, any keys encrypted by that private key are permanently inaccessible, and any data encrypted by those keys is permanently irretrievable. If you are configuring Key Trustee KMS high availability for the first time, continue synchronizing the private keys.
Cloudera recommends following security best practices and transferring the private key using offline media, such as a removable USB drive. For convenience (for example, in a development or testing environment where maximum security is not required), you can copy the private key over the network using the provided rsync command.
After you have synchronized the private keys, check the box to indicate you have done so and click Continue.
After the Key Trustee KMS service starts, click Finish to complete this step and return to the main page of the wizard.

For parcel-based Key Trustee KMS releases 5.8 and higher, Cloudera Manager automatically backs up Key Trustee KMS (using the ktbackup.sh script) after adding the Key Trustee KMS service. It does not schedule automatic backups using cron. For package-based installations, you must manually back up Key Trustee Server and configure a cron job.

The backup is stored in /var/lib/kms-keytrustee in cleartext. For more information about using the backup script and configuring the cron job (including how to encrypt backups), see Backing up Key Trustee Server and Clients.

8. Restart stale services and redeploy client configuration

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

This step restarts all services which were modified while enabling HDFS encryption. To complete this step:

Click Restart stale services and redeploy client configuration.
Click Restart Stale Services.
Ensure that Re-deploy client configuration is checked, and click Restart Now.
After all commands have completed, click Finish.

9. Validate Data Encryption

Minimum Required Role: Key Administrator or Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

This step launches a tutorial with instructions on creating an encryption zone and putting data into it to verify that HDFS encryption is enabled and working.

Enabling HDFS Encryption Using a Java KeyStore

After selecting A file-based password-protected Java KeyStore as the root of trust, the following steps are displayed:

1. Enable Kerberos

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

2. Enable TLS/SSL

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

3. Add a Java KeyStore KMS Service

Minimum Required Role: Key Administrator (also provided by Full Administrator)

This step adds the Java KeyStore KMS service to the cluster. The Java KeyStore KMS service uses a password-protected Java KeyStore for cryptographic key management. To complete this step:

Click Add a Java KeyStore KMS Service.
Select a cluster host for the Java KeyStore KMS service. Click Continue.
The Setup TLS for Java KeyStore KMS page provides high-level instructions for configuring TLS communication between the EDH cluster and the Java KeyStore KMS.
Click Continue.
The Review Changes page lists the Java KeyStore settings. Click the ? icon next to any setting for information about that setting. Enter the location and password for the Java KeyStore and click Continue.
Click Continue to automatically configure the HDFS service to depend on the Java KeyStore KMS service.
Click Finish to complete this step and return to the main page of the wizard.

4. Restart stale services and redeploy client configuration

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

This step restarts all services which were modified while enabling HDFS encryption. To complete this step:

Click Restart stale services and redeploy client configuration.
Click Restart Stale Services.
Ensure that Re-deploy client configuration is checked, and click Restart Now.
After all commands have completed, click Finish.

5. Validate Data Encryption

Minimum Required Role: Key Administrator or Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

This step launches a tutorial with instructions on creating an encryption zone and putting data into it to verify that HDFS encryption is enabled and working.