Fixed Issues in Cloudera Manager 6.1.0

The following sections describes issue fixed in Cloudera Manager 6.1.0:

ZooKeeper JMX did not support TLS when managed by Cloudera Manager

Technical Service Bulletin 2019-310 (TSB)

The ZooKeeper service optionally exposes a JMX port used for reporting and metrics. By default, Cloudera Manager enables this port, but prior to Cloudera Manager 6.1.0, it did not support mutual TLS authentication on this connection. While JMX has a password-based authentication mechanism that Cloudera Manager enables by default, weaknesses have been found in the authentication mechanism, and Oracle now advises JMX connections to enable mutual TLS authentication in addition to password-based authentication. A successful attack may leak data, cause denial of service, or even allow arbitrary code execution on the Java process that exposes a JMX port. Beginning in Cloudera Manager 6.1.0, it is possible to configure mutual TLS authentication on ZooKeeper’s JMX port.

Products affected: ZooKeeper

Releases affected: Cloudera Manager 6.1.0 and lower, Cloudera Manager 5.16 and lower

Users affected: All

Date/time of detection: June 7, 2018

Severity (Low/Medium/High): 9.8 High (CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H)

Impact: Remote code execution

CVE: CVE-2018-11744

Immediate action required: Upgrade to Cloudera Manager 6.1.0 and enable TLS for the ZooKeeper JMX port by turning on the configuration settings “Enable TLS/SSL for ZooKeeper JMX” and “Enable TLS client authentication for JMX port” on the ZooKeeper service and configuring the appropriate TLS settings. Alternatively, disable the ZooKeeper JMX port via the configuration setting “Enable JMX Agent” on the ZooKeeper service.

Addressed in release/refresh/patch: Cloudera Manager 6.1.0

Upgrade fails during checkJavaComponent (DbHostHeartbeat.java:177)

Fixed an issue with Java version parsing during Cloudera Manager upgrade.

Cloudera Issue: OPSAPS-47620

CDH 6 upgrade validator fails when no Sentry service is available

Fixed an issue where a CDH 5 cluster with a Keystore Indexer without a Sentry service. When attempting to upgrade to CDH 6 an empty error message displays. Note that when Sentry Policy File is enabled, users must either disable it or add a Sentry service, so that the policy file can be migrated automatically.

Cloudera Issue: OPSAPS-47617

creds.localjceks marked stale with an empty diff

Fixed an issue where creds.localjceks, the encrypted keystore used for the Hadoop Credentials provider, might be shown under the list of stale configuration files, but the contents did not actually change. When a role instance is shown as stale and its files include creds.localjceks, then this file will also be marked stale. This fix eliminates false reports of staleness.

Cloudera Issue: OPSAPS-47511

Search upgrade reinitialize does not use config for hdfs command

Fixed an issue where on CDH upgrade, Solr index files were not getting deleted from HDFS. This caused Solr to fail to start since files had an old index scheme.

Cloudera Issue: OPSAPS-47502

Export cluster template API returns failure

Fixed the cluster template export failure when the Hue configuration HDFS Web Interface Role (webhdfs_url) is using or pointing to the httpfs load balancer rather than to an HDFS role.

Cloudera Issue: OPSAPS-47060

Kafka should use the Garbage first garbage collector by default

Fixed an issue where Kafka broker and MirrorMaker processes did not use the Garbage-First (G1) garbage collector.

Cloudera Issue: OPSAPS-45956

Externally authenticated users cannot view their roles or previous session

Fixed a display issue where a user could not see their assigned roles and most recent successful login by navigating to <Username> > My Profile in the Cloudera Manager Admin Console. This issue did not affect functionality.

Cloudera Issue: OPSAPS-46996, OPSAPS-47025

Cloudera Manager not detecting available physical memory correctly

Fixed an issue with incorrect reporting of used physical memory on host nodes with a significant amount of Shared Memory in use. Cloudera Manager now takes usage of Shared Memory into account when reporting the physical memory used on a host node.

Cloudera Issue: OPSAPS-47396

HDFS_CLIENT_CONFIG_JAVA_OPTS has hbase in the template name

Changed the API name to fix the wrong name in the parameter.

Old Name New Name
hbase_client_java_opts hdfs_client_java_opts

This parameter configures the Client Java Configuration Options found under the HDFS Gateway role configuration. Any API scripts or cluster templates referencing these old names need to be updated to use the new names.

Cloudera Issue: OPSAPS-24569

Fix typos in a "detecton_window" API names

Changed the API names to fix typos in the following parameters:

Old Name New Name
hbase_active_master_detecton_window hbase_active_master_detection_window
hdfs_active_namenode_detecton_window hdfs_active_namenode_detection_window
mapreduce_active_jobtracker_detecton_window mapreduce_active_jobtracker_detection_window
yarn_active_resourcemanager_detecton_window yarn_active_resourcemanager_detection_window

These parameters tune the behavior of health test checking. The affected entities are: HBase Master, HDFS NameNode, MapReduce JobTracker, YARN ResourceManager. Any API scripts or cluster templates referencing these old names need to be updated to use the new names.

Cloudera Issue: OPSAPS-39223

CDH 6 Spark CSD does not support Auto-TLS

Fixed an issue where Auto-TLS settings were not applied to the Spark service when Auto-TLS was enabled.

Cloudera Issue: OPSAPS-47925

Impala shell does not display the port number

Fixed an issue where the Impala shell command in the Cloudera Manager Admin Console was missing the port number required to connect to the Impala shell.

Cloudera Issue: OPSAPS-47589

Enable ZooKeeper fix for CVE-2018-8012

Enable Kerberos Authentication and Enable Server to Server SASL Authentication settings in ZooKeeper have been linked together since both should be either turned on or off. If either is switched on or off, the other automatically follows.

This change automates steps that address CVE-2018-8012. Previously, the solution required manual steps.

Cloudera Issue: OPSAPS-46628

Combine audit entries

Fixed an issue that occurs when the API is accessed at a rapid rate. This can cause the Audits database table to grow rapidly, negatively impacting Cloudera Manager performance.

Cloudera Manager logs events in the Audits database table when the API is accessed either from the Admin Console or from any other client. You can now configure a time period during which similar events are combined into one log entry. For more information, see Audit Events.

Cloudera Issue: OPSAPS-46898

CMF_SERVER_ARGS if given a configuration file results in staleness for Cloudera Manager

Fixed an issue where applying a configuration change with CMF_SERVER_ARGS arguments (using the /etc/default/cloudera-scm-server configuration file) led to a staleness warning after a Cloudera Manager server restart.

Cloudera Issue: OPSAPS-47240

Kudu package missing from libs/common/src/main/java/com/cloudera/cmf/CDHResources

Fixes an issue where Cloudera Manager did not install Kudu packages when CDH was installed with packages instead of parcels.

Cloudera Issue: OPSAPS-45692

Restart warnings are incorrect after starting role with outdated configuration

Fixed an issue where some roles that required restarts were not correctly identified after starting a role marked as Started with Outdated Configuration.

Cloudera Issue: OPSAPS-45237

Typo in HiveServer2 load balancer API name

Fixed typos in the following parameter. This change affects Hive services when Hive Server 2 is configured for High Availability.

API Names
Old Name New Name
hiverserver2_load_balancer hiveserver2_load_balancer

Any API scripts or cluster templates referencing these old names will need to be updated to use the new names.

Cloudera Issue: OPSAPS-33266

Traceback seen in ImpalaRoleDiagnosticsCollection and HBaseRoleDiagnosticsCollectionprocess

Fixed an issue that caused an exception to occur in the Cloudera Manager Agent during diagnostic bundle collection if the process had exited previously.

Cloudera Issue: OPSAPS-47354

Fix kafka_network_processor_avg_idle metric

Fixed an issue where the kafka_network_processor_avg_idle metric shows NO DATA.

Cloudera Issue: OPSAPS-45816

Sentry fails on first run, due to a pending command

When starting Sentry for the first time after the service was added, the "Creating Sentry Database Tables" step in the Start Service command may fail with the error: "There is already a pending command on this entity". This issue has been fixed and starting Sentry for the first time after the service was added no longer fails due to a pending command.

Cloudera Issue: OPSAPS-48426

HDFS Canary with HA nameservice in a non-federated cluster fails

The HDFS canary no longer erroneously reports UNKNOWN health status.

Cloudera Issue: OPSAPS-48337

Server and Daemon RPM installation scripts do not work well with Puppet installs

If you have installed the JDK at a non-standard location, set the JAVA_HOME environment variable before installing Cloudera Manager. If you cannot set JAVA_HOME in your environment, create an empty file with the path /etc/cloudera-pre-install/CLOUDERA_SKIP_JAVA_INSTALL_CHECK to skip any Java checks during package installation of Cloudera Manager Server and Daemon packages.

Cloudera Issue: OPSAPS-47908

Cannot stop Kafka broker

Fixed an Issue where the Kafka Broker could not be stopped if Automatically Restart Process is enabled. Because of a misconfiguration in process monitoring, the Cloudera Manager Agent would also restart the process when a legitimate stop was requested. Additionally, without automatic restarts, once the process was stopped, the health check for Unexpected Exits would eventually show the process in bad health. Note that this bug affected all CSD-based services where a graceful stop behavior was enabled at the role-level.

Cloudera Issue: OPSAPS-45029

Database connection error.

Fixed a database connection leak issue that caused the following error: java.lang.IllegalStateException: currentCmfEntityManager already in transaction.

Cloudera Issue: OPSAPS-45829

CSD role creation logic fixed for second instance of service

Fixes the automatic role creation logic when adding a second instance of a service. Adding a second instance of a service could result in extra roles being generated for the first instance of a service.

Cloudera Issue: OPSAPS-47766

Agent should download key bundles when behind proxy (plain HTTP)

Even if a proxy server was configured for Cloudera Manager, it was not used to download the package signing key during host installs, leading to installation failures. This has been fixed so that downloading the package signing key will use the configured proxy, but only if it is a plain HTTP proxy. Proxies requiring authentication or HTTPS are not currently supported. As a workaround, you can mirror the package repository locally to avoid needing a proxy.

Cloudera Issue: OPSAPS-47830