Migrating from a Key Trustee KMS to an HSM KMS

In cases where your enterprise requires the highest level of encryption zone key (EZ key) security, or, it must adhere to strict security certification compliance standards, you have the option of migrating from a Key Trustee KMS (KT KMS) to a Hardware Security Module KMS (HSM KMS). Migrating from KT KMS to HSM KMS essentially means that you are moving EZ key metadata from one KMS to another.

You should be aware of the following important points associated with the migration of key metadata from a KT KMS to an HSM KMS:

Only EZ key metadata (name, cipher, length, description, created, and versions) is migrated. Encryption zone key material is not migrated.
EZ key versions on the destination KMS are distinct and disjoint from the EZ key versions on the source KMS.
The key isolation standards of the HSM KMS require that keys originate on and never leave the HSM. However, importing key material (which includes the corresponding unique identifier and key version) would violate this requirement. If you want to verify metadata after migrating to an HSM KMS, note that the key versions will be different than before. With the Java Keystore KMS and KT KMS, this is not the case.

Before attempting a KT KMS to HSM KMS migration, you should be familiar with basic concepts about EZ keys and the HSM KMS. For more details, see:

Requirements and Limitations

The following requirements and limitations apply to the KT KMS to HSM KMS migration of EZ key metadata:

The KT KMS and HSM KMS must be on separate hosts. Running each KMS on a distinct node ensures that there will be no conflicts, especially when both are running at the same time.
Key Trustee Server must be configured with Key HSM using either the SafeNet Luna or Thales HSM.
The KT KMS must remain active until all encrypted data encryption keys (EDEKs) are re-encrypted using an EZ key version from the destination KMS. This is necessary so that you can continue to use EDEKs encrypted with an EZ key version from the source KMS.
It is recommended that the HSM KMS is installed, but not running as the default KMS for HDFS during the actual migration.
The latency between the physical HSM and HSM KMS must be low. It is recommended that the measurable latency be no higher than 30 milliseconds. Note that this latency can result in timeout conditions, as the EDEK must be sent to the HSM to be encrypted and/or decrypted.

Migrating from a KT KMS to an HSM KMS

Copy the configuration file (kts-site.xml) from the KT KMS node to all HSM KMS nodes and, if not already the owner, change the owner to 'kms':

root@kms-1.example.com> scp /var/run/cloudera-scm-agent/process/##--keytrustee-KMS_KEYTRUSTEE/kts-site.xml root@hsm-1.example.com:/tmp/kts-site.xml
# Where ## is the latest number that is present for keytrustee-KMS_KEYTRUSTEE in the process folder.
root@hsm-1.example.com> chown kms:kms /tmp/kts-site.xml

The kts-site configuration file includes required connection information about the Key Trustee servers to which you will need to connect.

Copy the /var/lib/kms-keytrustee directory from the KT KMS node to all HSM KMS nodes:
```
root@kms-1.example.com> scp -r /var/lib/kms-keytrustee root@hsm-1.example.com:/var/lib/kms-keytrustee/
root@hsm-1.example.com> chown -R kms:kms /var/lib/kms-keytrustee
```
If you plan to re-encrypt with new keys, then you will require a way to decrypt the old keys first, before you can re-encrypt them. Copying this directory ensures that you will have all the GPG keys necessary to contact, read, and decrypt the EZ keys from the KT KMS.

If TLS is enabled, then import the Key Trustee Server certificates into the HSM KMS truststore on both nodes:

hsm-1.example.com> echo -n | openssl s_client -connect kts-1.example.com>11371 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/keytrustee_certificate.pem
# Use the default Oracle Java keytool to import the certificate:
/usr/java/jdk1.7.0_67/bin/keytool -importcert -keystore /path_to_truststore_file/truststore.jks -file /tmp/keytrustee_certificate.pem -alias kts1
# If Key Trustee Server HA is configured, then import certificates from both nodes.

Set the following Advanced Configuration Snippets for kms-site.xml (also known as "safety valves") on all HSM KMS nodes, unless otherwise specified. Refer to Custom Configuration for details about safety valves.

Configure the re-encryption source URI file. The configuration information provided here instructs the HSM to check the KT KMS for a key version (if it cannot be found in the HSM KMS).
Note: If TLS is enabled, use HTTPS. Otherwise, use HTTP.
```
<property>
<name>hadoop.kms.key.provider.reencryption.source.uri</name>
<value>kms://https@kms-1.example.com:16000/kms</value>
</property>

<property>
<name>hadoop.kms.key.provider.reencryption.source.uri</name>
<value>kms://http@kms-1.example.com:16000/kms</value>
</property>
```
This new HSM KMS URI includes the old KT KMS configuration as a secondary source. If the HSM KMS does not have that key version, then it reaches out to the KT KMS to try and locate it there. In cases where you have several KMS nodes, specify them using a semi-colon (;).

Enable re-encryption:

<property>
<name>hadoop.kms.reencryption.crypto.enable</name>
<value>true</value>
</property>

Configure the location of the kts-site.xml file:

<property>
<name>cloudera.hsmkp.ktkms.config.dir</name>
<value>/path/to/kts-site.xml</value>
</property>

Configure HSM-specific settings for the hadoop.kms.key.provider.uri:

For Luna, use the luna.keystore:
<property>
<name>hadoop.kms.key.provider.uri</name>
<value>cloudera.hsmkp://file@/${user.home}/luna.keystore,keytrustee://file@/var/lib/kms-keytrustee/keytrustee/.keytrustee</value>
</property>

For Thales, use the ncipher.keystore:
<property>
<name>hadoop.kms.key.provider.uri</name>
<value>cloudera.hsmkp://file@/${user.home}/ncipher.keystore,keytrustee://file@/var/lib/kms-keytrustee/keytrustee/.keytrustee</value>
</property>

Configure the following safety valve in kms-site.xml for a single Navigator HSM KMS Proxy role (for details, refer to Overriding Configuration Properties):
```
<property>
<name>cloudera.hsmkp.initiatemigration</name>
<value>true</value>
</property>
```

(Optional) If the HSM KMS is operating in a topology where there is significant latency between the HSM KMS and HSM (for example, an HSM KMS in the cloud with an HSM on premise), then you can configure cache sizes to prevent timeouts (the values here are suggestions; you may need to fine tune them for your specific topology):

In hdfs-site.xml on HDFS:
<property>
<name>hadoop.security.kms.client.encrypted.key.cache.low-watermark</name>
<value>.05</value>
</property>
<property>
<name>hadoop.security.kms.client.encrypted.key.cache.size</name>
<value>40</value>
</property>

In kms-site.xml on all HSM KMS nodes:
<property>
<name>hadoop.security.kms.encrypted.key.cache.low-watermark</name>
<value>.05</value>
</property>
<property>
<name>hadoop.security.kms.encrypted.key.cache.size</name>
<value>40</value>
</property>

If Kerberos is enabled, in the HSM KMS service configuration, set Zookeeper Authentication Type for Secret Signer to sasl.
In the HDFS configuration, replace KT KMS with HSM KMS.
Start the HSM KMS, and then restart HDFS. Upon startup, key migration automatically occurs.
Post migration, it is recommended that you set cloudera.hsmkp.initiatemigration to false after key migration completes (set in Step 5), and then restart the HSM KMS.
It is highly recommended that you re-encrypt encryption zones after migration is complete. For details about re-encrypting EDEKs, refer to Re-encrypting Encrypted Data Encryption Keys (EDEKs).
Before operating the new HSM KMS, it is recommended that you restore the properties (hadoop.kms.key.provider.uri), and also remove and clean up the safety valves that you set previously, as they are no longer necessary:
- cloudera.hsmkp.ktkms.config.dir
- hadoop.kms.key.provider.reencryption.source.uri
- hadoop.kms.reencryption.crypto.enable
(For Luna HSM) To restore hadoop.kms.key.provider.uri: cloudera.hsmkp://file@/${user.home}/luna.keystore
(For Thales HSM) To restore hadoop.kms.key.provider.uri: cloudera.hsmkp://file@/${user.home}/ncipher.keystore

Configuring KMS Access Control Lists (ACLs)

Migrating Keys from a Java KeyStore to Cloudera Navigator Key Trustee Server