New Features and Changes in Cloudera Manager 6.3.0
The following sections describe new and changed features for Cloudera Manager 6.3.0:
- OpenJDK 11 support for Cloudera Manager and CDH 6.3 and higher
- Cloudera Manager Kerberos support for FreeIPA and Red Hat Identity Manager
- SPNEGO/Kerberos support for Cloudera Manager Admin Console and API
- Cloudera Manager Server certificate expiration monitoring and alerting
- Auto-TLS Support for Root CA
- Service and Host Monitor configurations for Garbage Collection
- New Configuration Parameters for JournalNode Syncer
- New Kafka configuration parameters and metrics
- Simplified enabling Kerberos through Cloudera Manager templates
- Cluster Administrator role can now create and delete Data Context
- New YARN Configuration Parameter
- Kudu integration with Sentry
- Network Performance Inspector now includes a bandwidth test
- Kafka support in Compute Clusters
- Improved Backup and Disaster Log Retention behaviour
- Filtered HDFS NameNode and Hive MetaStore role logs included with BDR diagnostic bundles
- Option to disable metric rollup
OpenJDK 11 support for Cloudera Manager and CDH 6.3 and higher
You can now use OpenJDK 11 with Cloudera Enterprise 6.3.
When you install OpenJDK 11 in your cluster, it uses the G1GC method for garbage collection for most services, which may require tuning to avoid overcommitting memory. See Tuning JVM Garbage Collection.
OPSAPS-50993, OPSAPS-49390, OPSAPS-51643
Cloudera Manager Kerberos support for FreeIPA and Red Hat Identity Manager
Cloudera Manager now supports FreeIPA and RedHat Identity Manager as a Kerberos KDC type. See Step 4: Enabling Kerberos Using the Wizard
Cloudera Issue: OPSAPS-45833, OPSAPS-51348
SPNEGO/Kerberos support for Cloudera Manager Admin Console and API
The Cloudera Manager Admin Console and API now support Kerberos authentication (using SPNEGO). Only clients that can use SPNEGO (such as curl) are supported. The Swagger and Python SDKs for Cloudera Manager do not support SPNEGO.
With SPNEGO enabled, the Swagger-based Java and Python SDKs, as well as the older deprecated Java SDK, can still authenticate using HTTP Basic Authentication. The older deprecated Python SDK cannot. Do not enable SPNEGO if you are relying on the deprecated Python client for any operations.
You can enable SPNEGO/Kerberos authentication for the Cloudera Manager Admin Console and API by going to Settings > Administration > External Authentication > "Enable SPNEGO/Kerberos Authentication for the Admin Console and API" and checking the box. This requires a restart of Cloudera Manager to take effect. It also requires that Kerberos is enabled.
For more information, see Configuring External Authentication and Authorization for Cloudera Manager.
Cloudera Issue: OPSAPS-49753, OPSAPS-24297
Cloudera Manager Server certificate expiration monitoring and alerting
Cloudera Manager now alerts you 60 days before the Cloudera Manager Server TLS certificate expires. You can view and modify the threshold values by searching for Expiry in the Cloudera Management Service configuration page.
Cloudera Issue: OPSAPS-35977
Auto-TLS Support for Root CA
For new cluster installations, auto-TLS can create and use an intermediate CA for an existing internal root CA. This creates a chain of trust to your existing internal root CA, and allows internal hosts that trust your root CA to access cluster web resources without browser security warnings.
This feature is not supported for existing clusters.
Cloudera Issue: OPSAPS-50063
Service and Host Monitor configurations for Garbage Collection
The Cloudera Manager Host monitor and Service Monitor now use G1GC garbage collection by default. You can change this by adding Java options in the Java Configuration Options for Host Monitor and Java Configuration Options for Service Monitor configuration properties (Go to and search for "java". See Tuning JVM Garbage Collection.
Cloudera Issue: OPSAPS-50123
New Configuration Parameters for JournalNode Syncer
Enable JournalNode Syncer has been added for CDH 6.3 and later. When enabled, a JournalNode will periodically synchronize edit logs with other JournalNodes.
Shared Edits Directory has been added for HDFS JournalNodes when High Availability is enabled, which allows the JournalNode Syncer to start and work correctly.
Cloudera Issue: OPSAPS-51249
New Kafka configuration parameters and metrics
- num.network.threads=8
- num.recovery.threads.per.data.dir=1
- num.replica.fetchers=4 (default changed from 1)
- broker
- broker level topic
- controller
- fetch session cache
- log cleaner
- network
- replica manager
- session expire listener
- zookeeper client
Cloudera Issue: OPSAPS-49741
Simplified enabling Kerberos through Cloudera Manager templates
"instantiator": { "clusterName": "test", "enableKerberos": { "datanodeTransceiverPort" : <optional/default 1004>, "datanodeWebPort" : <optional/default 1006> }, . .This enables Kerberos on all the services in the template.
Cloudera Issue: OPSAPS-49704
Cluster Administrator role can now create and delete Data Context
Users with the Cluster Administrator role who can access only the base clusters in Virtual Private Cloud setups can now create and delete Data Contexts.
Cloudera Issue: OPSAPS-49478
New YARN Configuration Parameter
A new YARN configuration parameter, Enable Container Launch Debug Information, has been added and is disabled by default. Previously, this configuration had to be made using the NodeManager Advanced Configuration Snippet for yarn-site.xml.
Cloudera Issue: OPSAPS-50709
Kudu integration with Sentry
Cloudera Manager can now configure Kudu integration with Sentry.
Cloudera Issue: OPSAPS-50359
Network Performance Inspector now includes a bandwidth test
You can now test the bandwidth between clusters using the Network Performance Inspector. See Inspecting Network Performance
Cloudera Issue: OPSAPS-49682
Kafka support in Compute Clusters
You can now add the Kafka service to a compute cluster. Kafka will share the Sentry service across multiple Kafka instances.
Cloudera Issue: OPSAPS-49165
Improved Backup and Disaster Log Retention behaviour
When you are using a custom log location for BDR replication, automatic log expiration now purges these custom replication logs and metadata files, based on their purge settings. To maintain information for each replication schedule, it is important that user set valid paths for HDFS that are writable by an existing user.
OPSAPS-50880
Filtered HDFS NameNode and Hive MetaStore role logs included with BDR diagnostic bundles
- Earliest start and latest end timestamps for all the replication runs within a diagnostic bundle
- The configured log level (defaults to INFO) on both the source and destination clusters for HDFS and Hive replication jobs.
- HDFS NameNode (for HDFS and Hive replication jobs)
- Hive MetaStore (Hive replication)
Both source and destination role logs will be capped to a maximum of 201 MBs (this is the normal CDH role log size).
- HDFS Replication Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml
- Hive Replication Advanced Configuration Snippet (Safety Valve) for hive-site.xml
- SKIP_HIVE_LOG_COLLECTION [false]
- SKIP_HDFS_LOG_COLLECITON [false]
- LOG_LEVEL [INFO]
- LOG_SEARCH_TIMEOUT_MILLIS [300000]
- LOG_ROLE_RESULT_LIMIT [10000]
- LOG_TOTAL_BYTES_LIMIT [210763776]
Cloudera Issue: OPSAPS-49988
Option to disable metric rollup
You can now disable rollup of metrics over time. This can help increase performance of the Service Monitor. See Disabling Metric rollup.
Cloudera Issue: OPSAPS-50149