Known Issues and Limitations in Cloudera Manager 6.3.1
The following sections describe known issues and limitations for Cloudera Manager 6.3.1:
- Cloudera Manager 6.x issue with the service role Resume
- Cloudera Manager installation fails on MariaDB 10.2.8 and later
- BDR - Hive restore failing during import
- TLS does not work for Agent heartbeat
- Upgrade Failure: active NameNode not found
- HDFS Replication with Sentry causes additional NameNode heap usage
- Restart Kafka after upgrading Cloudera Manager
- Limited IE 11 browser support
- Cloudera Manager API Failure serving multiple API calls
Cloudera Manager 6.x issue with the service role Resume
If a selected service role on a node is restarted and fails, and the customer clicks the "Resume" button in Cloudera Manager, the service role on all of the nodes will be restarted concurrently.
Products affected: Cloudera Manager
- Cloudera Manager 5.5 and later
- Cloudera Manager 6.0 until 6.3.3
- Cloudera Manager 7.1.x
Users affected: Users with admin role in Cloudera Manager can impact end users of the service.
Impact:In production clusters this can result in a cluster-wide service outage; Already observed for the YARN service and the HDFS service in a few clusters.
Severity: High
- A workaround exists where instead of performing a restart we recommend performing a stop/start of the services.
- Issue is fixed in CM-6.3.4, CM-7.2.1 and above.
Knowledge article: For the latest update on this issue see the corresponding Knowledge article: Cloudera Customer Advisory: Cloudera Manager 6.x issue with service role Resume
Cloudera Manager installation fails on MariaDB 10.2.8 and later
When installing Cloudera Manager using MariaDB 10.2.8 or later, the Cloudera Manager web server doesn't come up and the install process ends with a failed status. The cloudera-scm-server.log includes the following SQL error:
2019-08-28 04:37:10,171 FATAL main:org.hsqldb.cmdline.SqlFile: SQL Error at 'UTF-8' line 57: "alter table ROLE_CONFIG_GROUPS drop column REVISION_ID" Key column 'REVISION_ID' doesn't exist in table 2019-08-28 04:37:10,171 FATAL main:org.hsqldb.cmdline.SqlFile: Rolling back SQL transaction. 2019-08-28 04:37:10,172 ERROR main:com.cloudera.enterprise.dbutil.SqlFileRunner: Exception while executing ddl scripts. com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Key column 'REVISION_ID' doesn't exist in table
Note that MariaDB 10.2.8 is provided by default in some operating systems, including SLES 12 SP4.
Workaround: Replace the default MariaDB 10.2.x version with MariaDB 10.2.7.
Affected Versions: MariaDB 10.2.8 and later
Cloudera Issue: OPSAPS-52340
BDR - Hive restore failing during import
When the table filter used during hive cloud restore is different from the table filter used to create the hive cloud backup, the import step fails with the table not found error. Currently it impacts only the cloud restore scenario.
Products affected: Cloudera Manager
- Cloudera Manager 5.15, 5.16
- Cloudera Manager 6.1.x
- Cloudera Manager 6.2.x
- Cloudera Manager 6.3.x
Users affected: BDR, Hive cloud restore, where restore uses a subset of tables from the exported tables
- Limited, the hive cloud restore all tables works properly.
- The hive cloud restore from the hive cloud backup created prior to Cloudera Manager 5.15 would work without any problem.
- No other BDR functionality is affected.
- Workaround: Not available. Importing specific tables would fail. Impoting ALL tables would continue to work properly.
- Upgrade: Upgrade to a Cloudera Manager version containing the fix.
Addressed in release/refresh/patch: Cloudera Manager 7.0 and higher versions
TLS does not work for Agent heartbeat
If you use Add Hosts Wizard to add a host and that host does not have the Linux host command line utility installed, the agent config.ini file on that host will have the Cloudera Manager Server's IP address in the "server_host=" line instead of the hostname. This will cause a problem if you enable TLS for agent to server communication because the agent will connect to the Cloudera Manager Server using the IP address and not the hostname. You can fix this problem by manually changing the "server_host=" to have the correct hostname.
Cloudera also recommends that you install the host utility for future use.
Cloudera Bug: OPSAPS-49273
Upgrade Failure: active NameNode not found
On a Cloudera Manager cluster host running on either CentOS 7.0 or 7.1, performing a curl operation to access the NameNode web UI results in an error similar to the following if the web UI is SSL-enabled:
curl https://nn1.example.com:20102 curl: (35) Cannot communicate securely with peer: no common encryption algorithm(s).
This issue occurs on Cloudera Manager clusters running versions 6.1 and higher that exclude insecure ciphers by default. In addition, an SSL bug with curl on CentOS versions 7.0 and 7.1 prevents negotiation of a more secure SSL cipher with the HDFS NameNode web UI.
Workaround: Update yum packages to the latest available versions by running the following command:
yum update nss-util nss-sysinit nss-tools
curl --tlsv1 --ciphers rsa_aes_256_cbc_sha_256 -k -v <hostname>:<port>
- Log into the Cloudera Manager Admin Console.
- Go to the HDFS Service
- Select the Configuration tab.
- Locate the SSL/TLS Cipher Suite property
- Select Intermediate 2018 (Needed for legacy clients or Redhat 6).
- Restart the HDFS service
Cloudera Bug: CDH-81328
HDFS Replication with Sentry causes additional NameNode heap usage
While performing HDFS replication on clusters where Sentry is in use on either or both of the source and destination clusters, you must set the value of Run on Peer as Username to be the same as Run as Username. This action ensures that Sentry provided ACL data is not copied to the target cluster, which results in additional usage of NameNode heap in the target cluster.
Cloudera Bug: OPSAPS-50649
Restart Kafka after upgrading Cloudera Manager
- num.network.threads=8
- num.recovery.threads.per.data.dir=1
- num.replica.fetchers=4
- producer.metrics.enable
Cloudera Bug: OPSAPS-49741
Limited IE 11 browser support
Many Cloudera Manager wizards, including Installation wizards and Add Service/Role wizards, cannot be completed when using Microsoft Internet Explorer version 11.x. To work around the issue, use another supported browser. See Browser Requirements.
Affected Versions: Cloudera Manager 6.3.0
Cloudera Bug: OPSAPS-51481
Cloudera Manager API Failure serving multiple API calls
2019-07-13 01:45:55,657 WARN C3P0PooledConnectionPoolManager[identityToken->2skykra31sef453dohob7|2b8bd14b]-AdminTaskTim er:com.mchange.v2.async.ThreadPoolAsynchronousRunner: com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector @20fc6e86 – APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
- Log in to the Cloudera Manager server host using ssh.
- Stop the Cloudera Manager server.
- Edit the /etc/cloudera-scm-server/db.properties file and append it with the following line:
com.cloudera.cmf.orm.hibernate.c3p0.max_statements=0
- Start the Cloudera Manager server
- Navigate to the Database Info page of Cloudera Manager at https://CM_Server:Port/cmf/debug/dbinfo in the Cloudera Manager UI.
- Confirm that the max_statements hibernate property is 0.
OPSAPS-51576