Cloudera Docs

Installing Ranger KMS backed by a Database and HA

The tasks and steps for installing the Ranger Key Management System (KMS) with High Availability (HA) service that uses a database as the backing key store.

This task uses the Set up HDFS Data At Rest Encryption wizard to install a Ranger KMS with HA service that uses a database as the backing key store.

The following image shows the Set up HDFS Data At Rest Encryption page. When you select your encryption keys root of trust option, a list of tasks that you must do to enable encryption to-and-from HDFS is displayed.

You complete each task independently from the other tasks. Where, the task’s Status column indicates whether the step has been completed and the Notes column provides additional context for the task. If your Cloudera Manager user account does not have sufficient privileges to complete a task, the Notes column indicates the privileges that are required.

When selected, each task contains links to wizards or documentation that help you complete the task. If a task is unavailable, due to insufficient privileges or an incomplete prerequisite step, no links are present and the Notes column displays the reason.


The Wizard steps are as follows and must be completed in the order listed:
  1. Enable Kerberos
  2. Enable TLS/SSL
  3. Add a Ranger KMS Service
  4. Restart the stale services and redeploy the client configuration
  5. Validate the Data Encryption
The following lists the post installation tasks for Installing the Ranger KMS backed by a Database and HA:
  • Update the Ranger KMS backed by a Database service's URL
  • Create a Ranger Audit Directory
Verify the following:
  • The cluster in which Cloudera Manager and the Cloudera Ranger service is installed, is up and running.
  • A Ranger KMS database has been created as the underlying keyStore mechanism. This database must be separate from the Ranger database.
  • Communication through secure connections is enabled with the Transport Layer Security (TLS) protocol and your network authentication is enabled with the Kerberos protocol.
  • You have securely recorded the following backing key store database access credentials, as you will be required to supply them during the installation steps:
    • The Database name.
    • The Database hostname.
    • The user name and password that has full administrative privileges to the backing key store database.
  1. In a supported web browser on the cluster in which the Ranger service is installed, log in to Cloudera Manager as a user with full administrative privileges.
  2. From the Cloudera Manager navigation side-bar, select Administration > Security.
  3. On the Security Status page, click Set up HDFS Data At Rest Encryption.
  4. In the Set up HDFS Data At Rest Encryption page, select the Ranger Key Management Service backed by Database option.
    A list of tasks are displayed at the bottom of the page. To successfully set up HDFS Data at Rest encryption, these tasks must be completed.
  5. To set up HDFS Encryption, follow the instructions as described below for each of the Set up HDFS Data At Rest Encryption Wizard's steps.

Installing the Ranger KMS Service

The Set up HDFS Data At Rest Encryption wizard's installation step that installs the Ranger Key Management System (KMS) with High Availability (HA) service on your cluster.

Describes the steps that install the Ranger Key Management System (KMS) service on your cluster and associates it with your backing key store database.

  1. From the Step column in the Set up HDFS Data at Rest Encryption for Cluster page, click Add Ranger KMS Service.
    The Add Ranger KMS Service to Cluster Wizard opens.
  2. In the Assign Roles page, verify that the hostname is the required server on which to install the Ranger KMS service by clicking inside the listed server field. By default, this field is populated by the Wizard.
    The Hosts Selected page opens.
  3. In the Hosts Selected page, scroll down and from the Hostname column, locate the hostname that was selected by the Wizard. Notice in the Added Roles column the Ranger KMS Server (RK) role icon. This role is added during the installation.
  4. Do one of the following:
    • If the pre-selected host is correct, confirm the Wizard's choice by clicking OK.
    • If the pre-selected host is incorrect, deselect the check box of the Wizard's choice, select the hostname check box of the required server, and then click OK.
  5. Back in the Assign Roles page, click Continue.
    The Setup Database page opens.
  6. In the Setup Database page, do the following:
    1. In the Database Hostname field, enter the hostname of the backing key store database.
    2. In the Database Name field, enter the name of the backing key store database.
    3. In the Username field, enter the user name that has full administrative privileges to the backing key store database.
    4. In the Password field, enter the password of the user name that has full administrative privileges to the backing key store database.
    5. (Optional) Verify that the credentials you entered are correct by clicking Test Connection.
    6. Click Continue.

    The Review Changes page opens.

  7. In the required Ranger KMS Master Key Password field, enter the password that will be used to encrypt the master key.
  8. Review the rest of the settings before clicking Continue.
  9. In the Command Details page, monitor the installation of the Ranger KMS server. When the Status displays Finished the Ranger KMS is installed and tested.
  10. Click Continue.
    The Summary page opens.
  11. Click Finish, which returns you to the Set up HDFS Data at Rest Encryption for Cluster page.
  12. Verify that the Ranger KMS service appears in the Cloudera Manager Clusters components list and that the service has been started.
    If the Ranger KMS service was not started by the installation wizard, do the following:
    1. Go to Cloudera Manager's Home page by clicking the Cloudera Manager icon.
    2. In the Cloudera Manager Clusters components list, locate and click Ranger KMS.
    3. From the Actions menu, click Start.
Adding Ranger KMS to a cluster triggers additional property updates for other services. Cloudera Manager may flag these with stale configuration warnings. Restart the stale services and redeploy the client configuration.

Restarting the Stale Services and Redeploying the Client Configuration

The Set up HDFS Data At Rest Encryption wizard's step for restarting stale services and redeploying the client configuration.

Describes the steps that restart stale services after installing the Data-at-Rest HDFS Ranger KMS service option on your cluster.

  1. From the Step column in the Set up HDFS Data at Rest Encryption for Cluster page, click Restart stale services and redeploy client configuration..
    The Stale Configurations page opens.
  2. Click Restart Stale Services.
    The restart Stale Services page opens.
  3. Verify that the Re-deploy client configuration check box is selected and click Restart Now.
  4. In the Command Details page, monitor the restart process. When the Status displays Finished, click Continue, which returns you to the Set up HDFS Data at Rest Encryption for Cluster page.
Validate that the Data-at-Rest HDFS Ranger KMS service option can successfully encrypt your data to-and-from HDFS.

Validating Data Encryption to-and-from HDFS

The Set up HDFS Data At Rest Encryption wizard's step for validating the data encryption to-and-from HDFS.

Describes the steps which verify that the Data-at-Rest HDFS Ranger KMS service option can successfully encrypt your data to-and-from HDFS.

  1. From the Step column in the Set up HDFS Data at Rest Encryption for Cluster page, click Validate Data Encryption.
    The Validate Data Encryption page opens, which displays a list of commands and instructions for creating an encryption zone and adding data.
  2. In a terminal, log in to one of the hosts in your cluster and run each of the following commands:
    1. Create a key and directory by entering the following: 
      kinit KEY_ADMIN_USER 
hadoop key create mykey1 
hdfs dfs -mkdir /tmp/zonel
      Where, KEY_ADMIN_USER is the key administrator whose role can perform the following actions:
      • Configure HDFS encryption, administer Key Trustee Server, and manage encryption keys
      • Start, stop, and restart Ranger KMS
      • Configure Ranger KMS Policies
      • View configuration and monitoring information in Cloudera Manager
      • View service and monitoring information
      • View events and logs
      • View Replication jobs and snapshot policies
      • View YARN applications and Impala queries
    2. Create a zone and link to the key, by entering the following: 
      kinit hdfs hdfs 
hdfs crypto -createZone -keyName mykey1 -path /tmp/zone1
    3. Create a file, put it in your zone, and verify that the file can be decrypted, by entering the following: 
      kinit KEY ADMIN_USER
echo "Hello World" > /tmp/helloWorld.txt
hdfs dfs -put /tmp/helloWorld.txt /tmp/zone1
hdfs dfs -cat /tmp/zonel/helloWorld.txt
rm /tmp/helloWorld.txt
    4. Verify that the stored file is encrypted, by entering the following: 
      kinit hdfs
hdfs dfs -cat /.reserved/raw/tmp/zonel/helloWorld.txt
hdfs dfs -rm -R / tmp/zone1
hdfs crypto -listZones
  3. When completed, click Close, which returns you to the Set up HDFS Data at Rest Encryption for Cluster page.

Post-Tasks for the Data-at-Rest HDFS Ranger KMS Service

The post-tasks that you must perform after you have set up the Data-at-Rest HDFS Ranger KMS service option.

Describes the post-task steps.

Depending on which Data-at-Rest HDFS Ranger KMS service option was set up, two or more of the following post-tasks must be completed:
  • Update the Data-at-Rest HDFS Ranger KMS service's URL
  • Create a Ranger Audit Directory
  • (Ranger KMS with Key Trustee Server service only) Update the Authentication Properties and KMS Hadoop cache settings
  1. Update the Data-at-Rest HDFS Ranger KMS service's URL by doing the following:
    1. In the Cloudera Manager Clusters components list, locate and click the Ranger service.
    2. Log in to the Ranger Web UI as the Ranger KMS user, whose default user name credential is keyadmin and default password is admin123.
    3. In the cm_kms service, click the Edit icon and update the KMS URL field value as follows:
      1. In the KMS URL field, enter the URL value using the following syntax:

        kms://http@kms_host1;kms_host2:kms_port/kms

        Where,
        • kms_host is the host where either the Ranger KMS with Key Trustee Server or the Ranger KMS backed by a database is installed.
        • kms_port is the port number. By default, this is 9292. For example,

          kms://http@kms_host1;kms_host2:9292/kms

      2. To confirm your URL setting, click Test Connection.
      3. Click Save.
  2. Create a Ranger Audit Directory by doing the following:
    1. Depending on which Data-at-Rest HDFS Ranger KMS service you set up, in the Cloudera Manager Clusters components list, locate and click either the Ranger KMS with Key Trustee Server service or the Ranger KMS service.
    2. From the Actions menu, click Create Ranger Plugin Audit Directory.
    3. When the Create Ranger Plugin Audit Directory message appears, confirm its creation by clicking Create Ranger Plugin Audit Directory.
    4. Monitor the creation process. When the Status displays Finished, click Close.