Upgrading Cloudera Navigator Key Trustee Server 3.x to 5.4.x

Navigator Key Trustee Server 5.4.x is the first release that supports installation using Cloudera Manager. If you are using Cloudera Manager, then you must upgrade Key Trustee Server using the command line before you can migrate Key Trustee Server to Cloudera Manager control.

To upgrade Key Trustee Server to 5.5 or higher, see Upgrading Cloudera Navigator Key Trustee Server 3.8 to 5.5 Using the ktupgrade Script.
CAUTION:
Cloudera recommends that you do not create new encryption keys, encryption zones, or clients during an upgrade.

Upgrading Key Trustee Server 3.x to 5.4.x Using the Command Line

The following instructions apply to both standalone and high availability Key Trustee Servers. For standalone Key Trustee Server, follow the instructions that refer to the active Key Trustee Server. For high availability Key Trustee Servers, follow the instructions on all Key Trustee Servers, unless otherwise indicated.

Upgrade Key Trustee Server

  1. Stop the httpd service: 
    sudo service httpd stop
  2. Install the EPEL Repository
    Dependent packages are available through the Extra Packages for Enterprise Linux (EPEL) repository. To install the EPEL repository, install the epel-release package:
    1. Copy the URL for the epel-release-<version>.noarch file for RHEL 6 or RHEL 7 located in the How can I use these extra packages? section of the EPEL wiki page.
    2. Run the following commands to install the EPEL repository: 
      sudo wget <epel_rpm_url>
sudo yum install epel-release-<version>.noarch.rpm

      Replace <version> with the version number of the downloaded RPM (for example, 6-8).

    If the epel-release package is already installed, you see a message similar to the following: 
    Examining /var/tmp/yum-root-jmZhL0/epel-release-6-8.noarch.rpm: epel-release-6-8.noarch
/var/tmp/yum-root-jmZhL0/epel-release-6-8.noarch.rpm: does not update installed package.
Error: Nothing to do
    Confirm that the EPEL repository is installed: 
    sudo yum repolist | grep -i epel
  3. Install the Cloudera Repository
    Create or edit the /etc/yum.repos.d/gazzang.repo file (for example, sudo vi /etc/yum.repos.d/gazzang.repo) and add the following text. Replace USER and PASSWD with the username and password provided by Cloudera. If you do not know your username or password, contact your Cloudera account team. 
    [gazzang_stable]
name=RHEL $releasever - gazzang.com - base
baseurl=https://USER:PASSWD@archive.gazzang.com/redhat/stable/$releasever
enabled=1
gpgcheck=1
gpgkey=http://archive.gazzang.com/gpg_gazzang.asc
    Import the GPG key by running the following command: 
    sudo rpm --import http://archive.gazzang.com/gpg_gazzang.asc
  4. Upgrade Key Trustee Server using yum: 
    sudo yum update keytrustee-server python-keytrustee
  5. Start the httpd service: 
    sudo service httpd start

Migrate Apache Web Server to CherryPy

For versions 5.4.0 and higher, Key Trustee Server uses CherryPy for the front end web interface; lower versions use the Apache web server. The CherryPy service is managed using the keytrusteed service. The Apache web server is managed using the httpd service. Run the following commands to migrate the web server from Apache to CherryPy.

  1. On the active Key Trustee Server, run the ktadmin db --configure command as follows: 
    sudo -u keytrustee ktadmin db --configure --port 11381 --pg-rootdir /var/lib/keytrustee/db --slave keytrustee02.example.com

    Replace keytrustee02.example.com with the hostname of the passive Key Trustee Server. For standalone Key Trustee Server, omit the --slave keytrustee02.example.com portion of the command.

  2. Export the active Key Trustee Server database. Run the following commands on the active Key Trustee Server: 
    sudo -u postgres pg_dump keytrustee > /var/lib/keytrustee/ktdbexport.pgsql
chown keytrustee:keytrustee /var/lib/keytrustee/ktdbexport.pgsql
  3. Start the Key Trustee Server database and import ktdbexport.pgsql: 
    sudo -u keytrustee /usr/pgsql-9.3/bin/pg_ctl -D /var/lib/keytrustee/db start --log /var/lib/keytrustee/db/pg_ctl.log
sudo -u keytrustee /usr/pgsql-9.3/bin/createdb --host /tmp --port 11381 -O keytrustee keytrustee
sudo -u keytrustee psql -d keytrustee -h /tmp -p 11381 < /var/lib/keytrustee/ktdbexport.pgsql
  4. (High Availability Key Trustee Servers Only) Start the passive Key Trustee Server. Run the following commands on the passive Key Trustee Server: 
    sudo -u keytrustee ktadmin --confdir /var/lib/keytrustee/.keytrustee init-slave --master keytrustee01.example.com --pg-rootdir /var/lib/keytrustee/db --no-import-key --master-host-port 11381 --logdir /var/lib/keytrustee/.keytrustee/logs --postgres-config=local --no-start
sudo -u keytrustee /usr/pgsql-9.3/bin/pg_ctl -D /var/lib/keytrustee/db start
  5. Edit /var/lib/keytrustee/.keytrustee/keytrustee.conf on the active and passive Key Trustee Servers to reference the new database and port. Set the DB_CONNECT parameter as follows: 
        "DB_CONNECT": "postgresql://localhost:11381/keytrustee?host=/tmp",
  6. Restart the Apache web server. Run this command on all Key Trustee Servers: 
    sudo service httpd restart
  7. Start the Key Trustee daemon (which starts the CherryPy web server). Run this command on all Key Trustee Servers: 
    sudo /etc/init.d/keytrusteed start
  8. After verifying that the Key Trustee daemon and CherryPy web server are running, stop the Apache web server and original database and prevent them from starting after reboots. Run these commands on all Key Trustee Servers: 
    sudo service httpd stop
sudo -u postgres /usr/pgsql-9.3/bin/pg_ctl -D /var/lib/pgsql/9.3/keytrustee stop
sudo chkconfig httpd off
sudo chkconfig postgresql-9.3 off

(High Availability Key Trustee Servers Only) Enable Synchronous Replication

Run the following command on the active Key Trustee Server to enable synchronous replication after upgrading: 
sudo -u keytrustee ktadmin enable-synchronous-replication --pg-rootdir /var/lib/keytrustee/db

Migrating Unmanaged Key Trustee Server to Cloudera Manager

For simplified and centralized administration, perform the following steps to move Key Trustee Server under Cloudera Manager control after upgrading Key Trustee Server:
  1. Download the Key Trustee Server CSD from the following location: 
    https://archive.gazzang.com/parcels/cloudera/keytrustee-server/5.4.9/stable/latest/csd/

    When prompted, enter your credentials. If you do not know your credentials, contact your Cloudera account team.

  2. Install the CSD into Cloudera Manager as instructed in Custom Service Descriptor Files. The CSD can only be installed on parcel-deployed clusters.
  3. Add the following parcel repository to Cloudera Manager following the instructions in Configuring Cloudera Manager Server Parcel Settings: 
    https://<username>:<password>@archive.gazzang.com/parcels/cloudera/keytrustee-server/5.4.9/stable/latest

    Replace <username> and <password> with your credentials. If you do not know your credentials, contact your Cloudera account team.

  4. (Recommended) Create a new cluster in Cloudera Manager containing only the hosts the Key Trustee Server will be installed on. Cloudera strongly recommends installing Key Trustee Server in a dedicated cluster to enable multiple clusters to share the same Key Trustee Server and to avoid restarting the Key Trustee Server when restarting a cluster. See Adding and Deleting Clusters for instructions on how to create a new cluster in Cloudera Manager.
  5. Download, distribute, and activate the Key Trustee Server parcel, following the instructions in Managing Parcels. After you activate the Key Trustee Server parcel, Cloudera Manager prompts you to restart the cluster. Click the Close button to ignore this prompt. You do not need to restart the cluster after installing Key Trustee Server.
  6. Stop the active and passive Key Trustee Server web servers using the command that corresponds to your backing web server. See Migrate Apache Web Server to CherryPy for more information.
    For Apache web servers: 
    sudo service httpd stop
    For CherryPy web servers: 
    sudo service keytrusteed stop
  7. Stop the active Key Trustee Server database. Run the following command on the active Key Trustee Server: 
    sudo -u keytrustee /usr/pgsql-9.3/bin/pg_ctl -D /var/lib/keytrustee/db stop
  8. Add the Key Trustee Server service to your cluster, following the instructions in Adding a Service. When customizing role assignments, assign the Active Key Trustee Server and Active Database roles to the active Key Trustee Server host, and the Passive Key Trustee Server and Passive Database roles to the passive Key Trustee Server host.
  9. Stop the passive Key Trustee Server database. Run the following command on the passive Key Trustee Server: 
    sudo -u keytrustee /usr/pgsql-9.3/bin/pg_ctl -D /var/lib/keytrustee/db stop
  10. Restart the Key Trustee Server service (Key Trustee Server service > Actions > Restart).
  11. (High Availability Key Trustee Servers Only) Enable synchronous replication. Run the following command on the active Key Trustee Server: 
    sudo -u keytrustee ktadmin enable-synchronous-replication --pg-rootdir /var/lib/keytrustee/db

Updating Key Trustee Server Clients

After upgrading Key Trustee Server to 5.4 or higher, you must configure Key Trustee Server clients (namely Key Trustee KMS and Cloudera Navigator Encrypt) to communicate with Key Trustee Server over the new ports:

  • Key Trustee KMS

    Add the following entries to the Key Trustee KMS advanced configuration snippet (Key Trustee KMS service > Configuration > Advanced > Key Management Server Advanced Configuration Snippet (Safety Valve) for kms-site.xml):

    <property>
    <name>cloudera.trustee.keyprovider.hkpport</name>
    <value>hkp_port_number</value>
    <description>
      Indicates the HTTP port on which Key Trustee Server clients should request public keys.
      On Key Trustee Server 3.8 (Apache webserver-based) servers, this is usually port 80 (unencrypted).
      On Key Trustee Server 5.4 and higher (CherryPy-based) servers, this is usually port 11371 (SSL-encrypted).
    </description>
</property>
<property>
    <name>cloudera.trustee.keyprovider.ktsport</name>
    <value>kts_port_number</value>
    <description>
      Indicates the HTTPS port on which the client sends and receives Key Trustee Server protocol messages.
      On Key Trustee Server 3.8 (Apache webserver-based) servers, this is usually port 443 (SSL-encrypted).
      On Key Trustee Server 5.4 and higher (CherryPy-based) servers, this is usually port 11371 (SSL-encrypted).
    </description>
</property>
<property>
    <name>cloudera.trustee.keyprovider.hkpssl</name>
    <value>boolean</value>
    <description>
      Indicates whether the client should communicate with the HKP server over an SSL-encrypted (true) or unencrypted (false) channel.
      On Key Trustee Server 3.8 (Apache webserver-based) servers, this is usually false (unencrypted).
      On Key Trustee Server 5.4 and higher (CherryPy-based) servers, this is usually true (SSL-encrypted).
    </description>
</property>
  • Cloudera Navigator Encrypt

    See Updating Key Trustee Server Ports for instructions on updating Cloudera Navigator Encrypt to use the new ports.

Validating Key Operations

Verify that the upgrade was successful by running the following command on all Key Trustee Servers. The output should be similar to the following. If high availability is enabled, the output should be identical on all Key Trustee Servers: 
curl -k https://keytrustee.example.com:11371/?a=fingerprint
4096R/4EDC46882386C827E20DEEA2D850ACA33BEDB0D1

Replace keytrustee.example.com with the fully qualified domain name (FQDN) of each Key Trustee Server you are validating.

If you are using Key Trustee Server as the backing key store for HDFS Transparent Encryption, run the following commands to verify that Hadoop key operations are successful:

hadoop key create hadoop_test_key
hadoop key list
hadoop key delete hadoop_test_key