Securing the Key Management Server (KMS)

This topic contains information on securing the Hadoop KMS using Kerberos, SSL-communication and Access Control Lists for operations on encryption keys.

Enabling Kerberos Authentication

Using Cloudera Manager

Minimum Required Role: Full Administrator

To enable Kerberos for the KMS using Cloudera Manager:
  1. Open the Cloudera Manager Admin Console and navigate to the KMS service.
  2. Click Configuration.
  3. Set the Authentication Type property to kerberos.
  4. Click Save Changes.
  5. Since Cloudera Manager does not automatically create the principal and keytab file for the KMS, you will need to manually run the Generate Credentials command. Using the top navigation bar, go to Administration > Kerberos > Credentials and click Generate Credentials.

Kerberos Mapping Rules:

The hadoop.kms.authentication.kerberos.name.rules property can only be changed using the Key Management Server Advanced Configuration Snippet (Safety Valve) for kms-site.xml property.

To modify, the value should be copied from the hadoop.security.auth_to_local configuration property, found in the core-site.xml file of the service.

Using the Command Line

Configure the etc/krb5.conf file with information for your KDC server. Create an HTTP principal and keytab file for the KMS.

Configure the etc/hadoop/kms-site.xml with the following properties: 
<property>
    <name>hadoop.kms.authentication.type</name>
    <value>kerberos</value>
</property>

<property>
    <name>hadoop.kms.authentication.kerberos.keytab</name>
    <value>${user.home}/kms.keytab</value>
</property>

<property>
    <name>hadoop.kms.authentication.kerberos.principal</name>
    <value>HTTP/localhost</value>
</property>

<property>
    <name>hadoop.kms.authentication.kerberos.name.rules</name>
    <value>DEFAULT</value>
</property>
Restart the KMS for the configuration changes to take effect.

Configuring the KMS Proxyuser

Each proxyuser must be configured in etc/hadoop/kms-site.xml using the following properties: 
<property>
    <name>hadoop.kms.proxyuser.#USER#.users</name>
    <value>*</value>
</property>

<property>
    <name>hadoop.kms.proxyuser.#USER#.groups</name>
    <value>*</value>
</property>

<property>
    <name>hadoop.kms.proxyuser.#USER#.hosts</name>
    <value>*</value>
</property>

Where #USER# is the username of the proxyuser to be configured.

The hadoop.kms.proxyuser.#USER#.users property indicates the users that can be impersonated. The hadoop.kms.proxyuser.#USER#.groups property indicates the groups to which the users being impersonated must belong. At least one of these properties must be defined. If both are defined, then the configured proxyuser will be able to impersonate any user in the users list and any user belonging to a group listed in the groups list.

The hadoop.kms.proxyuser.#USER#.hosts property indicates the host from which the proxyuser can make impersonation requests. "*" means there are no restrictions for the #USER# regarding users, groups or hosts.

Configuring SSL for the KMS

Using Cloudera Manager

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

The steps for configuring and enabling Hadoop SSL for the KMS are as follows:
  1. Open the Cloudera Manager Admin Console and navigate to the KMS service.
  2. Click Configuration.
  3. In the Search field, type SSL to show the KMS SSL properties (found under the Key Management Server Default Group > Security category).
  4. Edit the following SSL properties according to your cluster configuration.
    KMS SSL Properties
    Property Description
    Key Management Server SSL Enabled Enable SSL for requests to Key Management Server.
    Key Management Server SSL Server JKS Keystore File Password Location of the keystore file on the local file system.
    Key Management Server SSL Server JKS Keystore File Location Password for the keystore.
  5. Click Save Changes.
  6. Restart the KMS service.

Using the Command Line

To configure KMS to work over HTTPS by setting the following properties in the etc/hadoop/kms_env.sh script:

  • KMS_SSL_KEYSTORE_FILE
  • KMS_SSL_KEYSTORE_PASS

In the KMS tomcat/conf directory, replace the server.xml file with the provided ssl-server.xml file.

You need to create an SSL certificate for the KMS. As the kms user, use the Java keytool command to create the SSL certificate: 
$ keytool -genkey -alias tomcat -keyalg RSA

You will be asked a series of questions in an interactive prompt. It will create the keystore file, which will be named .keystore and located in the kms user's home directory. The password you enter for the keystore must match the value of the KMS_SSL_KEYSTORE_PASS environment variable set in the kms-env.sh script in the configuration directory.

The answer to "What is your first and last name?" (CN) must be the hostname of the machine where the KMS will be running.

Configuring Access Control Lists for the KMS

By default, the KMS ACLs are fully permissive, that is, everyone is allowed to perform all operations. However, a key design requirement is to be able to restrict HDFS superusers from having access to key material. This prevents a malicious superuser from having access to both (a) all the key material and (b) all the encrypted data, and thus being able to decrypt everything.

The KMS supports both whitelist and blacklist ACLs where the blacklist overrides the whitelist. A user accessing KMS is first checked for inclusion in the ACL for the requested operation and then checked for exclusion in the blacklist for the operation before access is granted. Hence, add HDFS superusers to the blacklist, while allowing everyone else.

Using Cloudera Manager

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

  1. Open the Cloudera Manager Admin Console and navigate to the KMS service.
  2. Click Configuration.
  3. In the Search field, type acl to show the KMS ACL properties (found under the Key Management Server Default Group category).
  4. Edit the ACL properties according to your cluster configuration.
  5. Click Save Changes.
  6. Restart the KMS service.

Using the Command Line

KMS ACLs configuration are defined in the KMS etc/hadoop/kms-acls.xml configuration file. This file is hot-reloaded when it changes.

<property>
    <name>hadoop.kms.acl.CREATE</name>
    <value>*</value>
    <description>
      ACL for create-key operations.
      If the user does is not in the GET ACL, the key material is not returned
      as part of the response.
    </description>
</property>

<property>
    <name>hadoop.kms.blacklist.CREATE</name>
    <value>hdfs,foo</value>
    <description>
      Blacklist for create-key operations.
      If the user does is in the Blacklist, the key material is not returned
      as part of the response.
    </description>
</property>

<property>
    <name>hadoop.kms.acl.DELETE</name>
    <value>*</value>
    <description>
      ACL for delete-key operations.
    </description>
</property>

<property>
    <name>hadoop.kms.blacklist.DELETE</name>
    <value>hdfs,foo</value>
    <description>
      Blacklist for delete-key operations.
    </description>
</property>

<property>
    <name>hadoop.kms.acl.ROLLOVER</name>
    <value>*</value>
    <description>
      ACL for rollover-key operations.
      If the user does is not in the GET ACL, the key material is not returned
      as part of the response.
    </description>
</property>

<property>
    <name>hadoop.kms.blacklist.ROLLOVER</name>
    <value>hdfs,foo</value>
    <description>
      Blacklist for rollover-key operations.
    </description>
</property>

<property>
    <name>hadoop.kms.acl.GET</name>
    <value>*</value>
    <description>
      ACL for get-key-version and get-current-key operations.
    </description>
</property>

<property>
    <name>hadoop.kms.blacklist.GET</name>
    <value>hdfs,foo</value>
    <description>
      ACL for get-key-version and get-current-key operations.
    </description>
</property>

<property>
    <name>hadoop.kms.acl.GET_KEYS</name>
    <value>*</value>
    <description>
      ACL for get-keys operation.
    </description>
</property>

<property>
    <name>hadoop.kms.blacklist.GET_KEYS</name>
    <value>hdfs,foo</value>
    <description>
      Blacklist for get-keys operation.
    </description>
</property>

<property>
    <name>hadoop.kms.acl.GET_METADATA</name>
    <value>*</value>
    <description>
      ACL for get-key-metadata and get-keys-metadata operations.
    </description>
</property>

<property>
    <name>hadoop.kms.blacklist.GET_METADATA</name>
    <value>hdfs,foo</value>
    <description>
      Blacklist for get-key-metadata and get-keys-metadata operations.
    </description>
</property>

 <property>
    <name>hadoop.kms.acl.SET_KEY_MATERIAL</name>
    <value>*</value>
    <description>
        Complimentary ACL for CREATE and ROLLOVER operation to allow the client
        to provide the key material when creating or rolling a key.
    </description>
</property>

<property>
    <name>hadoop.kms.blacklist.SET_KEY_MATERIAL</name>
    <value>hdfs,foo</value>
    <description>
        Complimentary Blacklist for CREATE and ROLLOVER operation to allow the client
        to provide the key material when creating or rolling a key.
    </description>
</property>

<property>
    <name>hadoop.kms.acl.GENERATE_EEK</name>
    <value>*</value>
    <description>
      ACL for generateEncryptedKey
      CryptoExtension operations
    </description>
</property>

<property>
    <name>hadoop.kms.blacklist.GENERATE_EEK</name>
    <value>hdfs,foo</value>
    <description>
      Blacklist for generateEncryptedKey
      CryptoExtension operations
    </description>
</property>

<property>
    <name>hadoop.kms.acl.DECRYPT_EEK</name>
    <value>*</value>
    <description>
      ACL for decrypt EncryptedKey
      CryptoExtension operations
    </description>
</property>

<property>
    <name>hadoop.kms.blacklist.DECRYPT_EEK</name>
    <value>hdfs,foo</value>
    <description>
      Blacklist for decrypt EncryptedKey
      CryptoExtension operations
    </description>
  </property>

Configuring Key Access Control

KMS supports access control for all non-read operations at the Key level. All Key Access operations are classified as:

  • MANAGEMENT - createKey, deleteKey, rolloverNewVersion
  • GENERATE_EEK - generateEncryptedKey, warmUpEncryptedKeys
  • DECRYPT_EEK - decryptEncryptedKey
  • READ - getKeyVersion, getKeyVersions, getMetadata, getKeysMetadata, getCurrentKey
  • ALL - all of the above.
These can be defined in the etc/hadoop/kms-acls.xml as follows: 
  <property>
    <name>key.acl.testKey1.MANAGEMENT</name>
    <value>*</value>
    <description>
      ACL for create-key, deleteKey and rolloverNewVersion operations.
    </description>
  </property>

  <property>
    <name>key.acl.testKey2.GENERATE_EEK</name>
    <value>*</value>
    <description>
      ACL for generateEncryptedKey operations.
    </description>
  </property>

  <property>
    <name>key.acl.testKey3.DECRYPT_EEK</name>
    <value>*</value>
    <description>
      ACL for decryptEncryptedKey operations.
    </description>
  </property>

  <property>
    <name>key.acl.testKey4.READ</name>
    <value>*</value>
    <description>
      ACL for getKeyVersion, getKeyVersions, getMetadata, getKeysMetadata,
      getCurrentKey operations
    </description>
  </property>

  <property>
    <name>key.acl.testKey5.ALL</name>
    <value>*</value>
    <description>
      ACL for ALL operations.
    </description>
  </property>

  <property>
    <name>default.key.acl.MANAGEMENT</name>
    <value>user1,user2</value>
    <description>
      default ACL for MANAGEMENT operations for all keys that are not
      explicitly defined.
    </description>
  </property>

  <property>
    <name>default.key.acl.GENERATE_EEK</name>
    <value>user1,user2</value>
    <description>
      default ACL for GENERATE_EEK operations for all keys that are not
      explicitly defined.
    </description>
  </property>

  <property>
    <name>default.key.acl.DECRYPT_EEK</name>
    <value>user1,user2</value>
    <description>
      default ACL for DECRYPT_EEK operations for all keys that are not
      explicitly defined.
    </description>
  </property>

  <property>
    <name>default.key.acl.READ</name>
    <value>user1,user2</value>
    <description>
      default ACL for READ operations for all keys that are not
      explicitly defined.
    </description>
  </property>

For all keys for which an ACL has not been explicitly configured, you can configure a default key ACL for a subset of the operation types.

If no ACL is configured for a specific key, and no default ACL is configured for the requested operation, access will be denied.

KMS Delegation Token Configuration

KMS delegation token secret manager can be configured using the following properties: 
  <property>
    <name>hadoop.kms.authentication.delegation-token.update-interval.sec</name>
    <value>86400</value>
    <description>
      How often the master key is rotated, in seconds. Default value 1 day.
    </description>
  </property>

  <property>
    <name>hadoop.kms.authentication.delegation-token.max-lifetime.sec</name>
    <value>604800</value>
    <description>
      Maximum lifetime of a delegation token, in seconds. Default value 7 days.
    </description>
  </property>

  <property>
    <name>hadoop.kms.authentication.delegation-token.renew-interval.sec</name>
    <value>86400</value>
    <description>
      Renewal interval of a delegation token, in seconds. Default value 1 day.
    </description>
  </property>

  <property>
    <name>hadoop.kms.authentication.delegation-token.removal-scan-interval.sec</name>
    <value>3600</value>
    <description>
      Scan interval to remove expired delegation tokens.
    </description>
  </property>