Cloudera Docs

Integrating Ranger KMS with Google Cloud HSM

This document provides technical instructions for configuring Apache Ranger Key Management Service (KMS) to use Google Cloud HSM for key management. It covers both a fresh installation scenario and an upgrade/migration scenario for existing installations.

Prerequisites and Google Cloud HSM setup

Before configuring Ranger KMS, you must establish the necessary infrastructure and access tokens within your Google Cloud Platform (GCP) environment:
  1. Create a service account:
    1. Log in to the Google Cloud Console.
    2. Select your project and create a dedicated service account.
    3. Download its authentication key in JSON format.

      This file is required for authentication.

  2. Create a key ring:
    1. In the GCP Console search bar, search for Key Management (Cloud KMS).
    2. Create a new key ring (for example, RangerKmsRing).

Fresh installation: Configuring Ranger KMS with GCP

Follow these steps if you are performing a clean configuration of Ranger KMS to natively use GCP for master key generation:
  1. Stop the KMS service.

    Ensure that the Ranger KMS service is completely stopped before applying configurations.

  2. Configure the Safety Valve:
    1. Log in to Cloudera Manager.
    2. Navigate to Ranger KMS > Configuration and search for dbks-site.xml.
    3. Add the following property definitions into the dbks-site.xml safety valve:
      <property> 
  <name>ranger.kms.gcp.enabled</name> 
  <value>true</value> 
</property> 
<property> 
  <name>ranger.kms.gcp.keyring.id</name> 
  <value>RangerKmsRing</value> 
</property> 
<property> 
  <name>ranger.kms.gcp.cred.file</name> 
  <value>/opt/cloudera/parcels/CDH/lib/ranger-kms/your_cred_file.json</value> 
</property> 
<property> 
  <name>ranger.kms.gcp.project.id</name> 
  <value>gcp-eng-sdx-daily</value> 
</property> 
<property> 
  <name>ranger.kms.gcp.location.id</name> 
  <value>global</value> 
</property> 
<property> 
  <name>ranger.kms.gcp.masterkey.name</name> 
  <value>ApacheMasterKey1</value>
</property>
      Alternatively, you can copy the downloaded GCP service account credential JSON file to the following directory on your KMS host:
      /opt/cloudera/parcels/CDH/lib/ranger-kms
  3. Save the configuration changes and start the KMS service.

    The master key will automatically be created inside your Google Cloud HSM ecosystem.

  4. Verify that the standard KMS zone encryption and decryption tasks successfully route through the newly established GCP master key.

Upgrade scenario: Migrating master key storage from KMS database to Google Cloud HSM

Follow these steps to migrate an existing Ranger KMS master key stored in a local relational database to Google Cloud HSM:
  2. Move to your Ranger KMS home directory:
    /opt/cloudera/parcels/CDH/lib/ranger-kms
  3. SSH to the KMS host and run the following command:
    ls -ltrh /var/run/cloudera-scm-agent/process/ | grep kms

    You will get kms process directories.

    1. Identify the latest directory entry from the timestamps provided.
  4. Export the following variable (process number and password will vary in the following snippet):
    export JAVA_HOME=/usr/java/<jdk-version>-cloudera/
export RANGER_KMS_HOME=/opt/cloudera/parcels/CDH/lib/ranger-kms
export RANGER_KMS_CONF=/var/run/cloudera-scm-agent/process/1546338415-ranger_kms-RANGER_KMS_SERVER/conf
export SQL_CONNECTOR_JAR=/usr/share/java/postgresql-connector-java.jar
export HADOOP_CREDSTORE_PASSWORD=sbpiuybywid78552ory169p6
  5. Migrate the master key to GCP by using the MigrateMKeyStorageDbToGCP.sh utility script.
    Use the following command for this purpose:
    bash MigrateMKeyStorageDbToGCP.sh <key_name_for_gcp> <gcp_project_name> <gcp_key_ring_name>  <gcp_key_ring_location> <cred json file>
    Upon successful execution, the following message appears:
    Getting Master KeyMaster Key from Ranger KMS DB has been successfully migrated to GCP.
  6. Now navigate to Cloudera Manager > Ranger KMS > Configuration, and search for dbks-site.xml.
  7. Add the following in the Ranger KMS Server Advanced Configuration Snippet (Safety Valve) for conf/dbks-site.xml and save the changes:
    <property> 
    <name>ranger.kms.gcp.enabled</name> 
    <value>true</value> 
</property> 

<property> 
    <name>ranger.kms.gcp.keyring.id</name> 
    <value>RangerKmsRing</value> 
</property> 

<property> 
    <name>ranger.kms.gcp.cred.file</name> 
    <value>/full/path/to/downloadedCredJsonFile.json</value> 
</property> 

<property> 
    <name>ranger.kms.gcp.project.id</name> 
    <value>gcp-eng-sdx-daily</value> 
</property> 

<property> 
    <name>ranger.kms.gcp.location.id</name> 
    <value>global</value> 
</property> 

<property> 
   <name>ranger.kms.gcp.masterkey.name</name>
   <value>vbmasterkeyupmar201</value>
</property>

    Alternatively, you can copy the cred json file in the following location: /opt/cloudera/parcels/CDH/lib/ranger-kms

  8. Add the following in the Ranger KMS Service Environment Advanced Configuration Snippet (Safety Valve) and save the changes:
    Key: JAVA_OPTS
Value: --add-opens java.base/java.util=ALL-UNNAMED
  9. Restart the Ranger KMS service, and encrypt and decrypt new zones.

    You should be able to access previously encrypted zones as well.