Known Issues and Workarounds in Cloudera Navigator Key Trustee Server

Key Trustee Server active database setup fails when command line configuration specifies non-default database port

If the Key Trustee Server is configured from the command line to use a non-default database port (the default port is 11381), then when the Key Trustee Server service is added to Cloudera Manager, the first database startup fails.

Affected Versions: 5.15.0, 5.16.0

Cloudera Bug: KT-6238

Workaround:
  1. Log in to the Key Trustee Server and manually stop/start the databases: 
    service keytrustee-db stop
    Make sure that the postmaster.pid file, which is located in the database directory, no longer exists. If necessary, replace /var/lib/keytrustee/db in the following command with the appropriate database directory for your system: 
    # ls /var/lib/keytrustee/db/postmaster.pid
    If postmaster.pid has not been cleaned up, then use the pg_ctl utility to stop the database directly: 
    # pg_ctl stop
  2. Return to the Cloudera Manager home page, where the Key Trustee Server service is listed.
  3. Go into the Key Trustee Server service configuration and for Key Trustee Server Database Port, specify the port that was configured during the command line configuration.
  4. Redeploy the Key Trustee Server configuration and restart the service.

Key Trustee Server 5.4.x and lower is not supported by Cloudera Manager 5.10.x and higher

Key Trustee Server 5.4.x and lower is not supported by Cloudera Manager 5.10.x and higher.

Affected Versions: 5.4.x

Fixed in Version(s): 5.10.1 through 5.14.0

Cloudera Bug: KT-6178

Workaround: Upgrade to a version of Key Trustee Server higher than 5.4.x.

Malicious user can access database used to store HDFS encryption zone keys

This vulnerability, which impacts Key Trustee Server, allows anyone with access to the host on which Key Trustee Server is running to use a non-libpq based connection driver to connect to the Postgres database used by Key Trustee Server without specifying a password.

The keys stored in the database are encrypted, which means access to data cannot be gained by this method. However, the vulnerability could provide a way for someone to alter encryption zone key metadata, delete the encryption zone keys, or delete the contents of the entire database.

Affected Versions: 5.3.0, 5.4.0, 5.5.0, 5.6.0, 5.7.0, 5.8.0, 5.9.0, 5.10.0, 5.11.0, 5.12.0, 5.13.0

Fixed in Version: 5.14.0

Cloudera Bug: DOCS-2788

Workaround: To remove the connection capability to Postgres (via the local network from all Key Trustee Server roles):

  1. Log in as root to the host running Key Trustee Server.
  2. Make a backup of the original Postgres configuration file: 
    cp /var/lib/keytrustee/db/pg_hba.conf \ /var/lib/keytrustee/db/pg_hba.conf.bak
  3. Edit the Postgres configuration file to comment out the use of network connections for Postgres.
  4. Using a preferred editor, edit /var/lib/keytrustee/db/pg_hba.conf, and add a # character to the beginning of the following lines in the file: 
    host keytrustee keytrustee 127.0.0.1/32 md5
host keytrustee keytrustee ::1/128 md5

After editing, the files should appear as:

#host keytrustee keytrustee 127.0.0.1/32 md5
#host keytrustee keytrustee ::1/128 md5
  5. Save the file.
  6. Perform a rolling restart of Key Trustee Server service.

The encryption wizard continues to fail if there is a failure during initial configuration run

The encryption wizard continues to fail if there was a failure during the initial run configuring HSM KMS.

Affected Versions: 5.12.0, 5.13.0, 5.14.0, 5.15.0, 5.16.0

Cloudera Bug: KT-4909

Workaround: Open Cloudera Manager in another browser tab, and manually stop the installed KMS by clicking the arrow next to the KMS and selecting Stop. Then retry the installation in the new tab after correcting the cause of the install failure.

Before installing the Thales backed HSM KMS, you must add the KMS user to the nfast group

After installation of the Thales HSM client, and before installing Navigator HSM KMS backed by Thales HSM, you must add the KMS user to the nfast group..

Affected Versions: 5.12.0, 5.13.0, 5.14.0, 5.15.0, 5.16.0

Cloudera Bug: KT-4618

Workaround: Run the following command to manually add the KMS user to the nfast group:usermod -a -G nfast kms

Navigator Key HSM may become unresponsive when there are network delays or connection issues with the HSM

This issue manifests itself as job failures within CDH. To verify that this issue is the source of the failed CDH jobs and that Key HSM has become unresponsive log into the Key HSM server and look at the log file in /var/log/keyhsm/. If no new entries are being written to the log file while key requests are being made, e.g. CDH jobs continue to retry, then follow the directions in the Workaround section.

Workaround: If threads accumulate and Navigator Key HSM has become unresponsive then restart Key HSM by executing the following commands: 
$ sudo service keyhsm shutdown
$ sudo service keyhsm start

Key Trustee KMS cannot connect to Key Trustee Server using TLS versions other than 1.0 on JDK 7

If you have configured Key Trustee Server to use a TLS version other than 1.0, Key Trustee KMS fails to connect to Key Trustee Server, and key operations fail when using JDK 7.

Workaround: Use TLS version 1.0 only, or JDK 8.

Key Trustee Server cannot use TLS version 1.2 on RHEL 6

Configuring Key Trustee Server to use TLS version 1.2 causes Key Trustee Server to be unable to start.

Workaround: Use your operating system package manager to upgrade the pyOpenSSL package to version 1.4 or higher, or do not configure Key Trustee Server to use TLS version 1.2.

Upgraded passive Key Trustee Server fails to start due to incorrect ownership of recovery.conf

Passive Key Trustee Servers upgraded from Key Trustee Server 3.8.x or lower fail to start with the following error: 
WARNING:root:stdout pg_basebackup: directory "/var/lib/pgsql/9.3/keytrustee" exists but is not empty
:
Traceback (most recent call last):
  File "/usr/bin/ktadmin", line 484, in <module>
    main()
  File "/usr/bin/ktadmin", line 473, in main
    init_slave()
  File "/usr/bin/ktadmin", line 349, in init_slave
    pgsetup.base_backup(ARGS.pg_rootdir, ARGS.master, PKG, run_as=ARGS.postgres_user)
  File "/usr/lib/python2.6/site-packages/keytrustee/server/setup/postgres.py", line 206, in base_backup
    run([pg_basebackup, '-D', dest, '--host=%s' % master_ip, '--port=%d' % port, '--username=%s' % db_user], run_as=run_as)
  File "/usr/lib/python2.6/site-packages/keytrustee/util.py", line 145, in run
    raise subprocess.CalledProcessError(p.returncode, cmd)
subprocess.CalledProcessError: Command '['/usr/pgsql-9.3/bin/pg_basebackup', '-D', '/var/lib/pgsql/9.3/keytrustee', '--host=kt01.example.com', '--port=5432', '--username=keytrustee']' returned non-zero exit status 1
Workaround: Change the owner and group of /var/lib/pgsql/9.3/keytrustee/recovery.conf to postgres: 
$ sudo chown postgres:postgres /var/lib/pgsql/9.3/keytrustee/recovery.conf

Key Trustee Server PKCS8 private key cannot communicate with Key HSM

If its private key is in PKCS8 format, Key Trustee Server cannot communicate with Key HSM.

Cloudera Bug: KT-3172

Workaround: Convert the Key Trustee Server private key to raw RSA format.

Key Trustee Server backup script fails if PostgreSQL versions lower than 9.3 are installed

If PostgreSQL versions lower than 9.3 are installed on the Key Trustee Server host, the ktbackup.sh script fails with an error similar to the following:

pg_dump: server version: 9.3.11; pg_dump version: 9.2.14
pg_dump: aborting because of server version mismatch

Workaround: Uninstall the lower PostgreSQL version.