What's New in Cloudera Manager 7.4.3

New features and changed behavior for Cloudera Manager 7.4.3.

Miscellaneous

Add "krb5.conf" location configuration into Cloudera Manager
Cloudera Manager now allows a user to set a path for the Kerberos Configuration file, krb5.conf. The user can use the Adminstration > Settings > krb5.conf file path configuration field in Cloudera Manager to set the path. The default is set to /etc/krb5.conf . For now, the valid paths are limited to /etc/hadoop/* or /etc/krb5.conf only. Note: If the configuration to 'Manage Krb5.conf using Cloudera Manager' property is also set by the Cloudera Manager admin user, it will cause Kerberos Staleness because Cloudera Manager is responsible for handling Kerberos configurations.
Expose extra command line arguments for Auto-TLS
The GenerateCMCA API now provides access to an additional arguments:
  • additionalArguments: This parameter can be provided to pass additional parameters for internal CA certificates.

  • subjectaltnames: Using this parameter, a list of Subject Alt Names can be provided for each host during certificate generation.

For more information, please refer to the Cloudera Manager API documentation: https://archive.cloudera.com/cm-public/7.4.3/generic/jar/cm_api/apidocs/json_ApiGenerateCmcaArguments.html
Upgrade Jetty to 9.4.latest
Jetty Server version has been upgraded to 9.4.35.v20201120, which fixes numerous security vulnerabilities .
Enable mTLS for Ratis in Ozone
New configuration parameters have been added for Ozone to separate Data Node Ratis admin/server traffic from clients:
  • dfs.container.ratis.admin.port, defaults to 9857
  • dfs.container.ratis.server.port, defaults to 9856
A new TLS configuration for Ratis in Data Node and OM has also been added :
  • hdds.grpc.tls.enabled, defaults to false
Make Control settings for Data Hub part of the CSD
The Cruise-Control service has new configurable properties related to self-healing.
Add support for custom ZooKeeper principal in Ranger
Added support in Ranger to use custom zookeeper principal for communication with Solr service.
Add option to CollectDiagnosticDataArguments API to force diagnostic bundle upload
This is a new feature in the Cloudera Manager API. The feature adds a new parameter to ApiCollectDiagnosticDataArguments which adds the ability to force the generated diagnostic bundle to be uploaded to Cloudera. With the previous behaviour, this upload feature was controlled by the PHONE_HOME parameter of Cloudera Manager which is still in use, but can be overridden by this new parameter. The feature is backward compatible. The parameter is optional and-if the parameter is missing, the old behaviour takes place according to the PHONE_HOME setting.
Add config 'ozone.scm.ratis.storage.dir' to Ozone
A new configuration property for Ozone, ozone.scm.ratis.storage.dir, has been added.
Add Hive configuration to set cipher suites for Hive WebUi and HS2
With the fix, Hive WebUI SSL Cipher Suites can be configured, allowing the Web UI and HiveServer2 to work with TLS security on a FIPS-enabled operating system.

Atlas

New feature to enable Atlas Hook Spooling
Atlas hook spooling feature is now available and configurable which can be enabled / disabled from configurations.
Cloudera Manager can now configure Atlas Hooks for Sqoop
Cloudera Manager is now capable of enabling and configuring the Sqoop Atlas Hook. See: http://atlas.apache.org/2.0.0/Hook-Sqoop.html The configuration happens automatically and no extra, manual configuration is required from the users. Cloudera Manager configures the Hook in sqoop-site.xml and it automatically generates the atlas-application.properties file for Sqoop. If you are installing a fresh cluster with Atlas being present, then the Hook will be enabled automatically and the atlas-application.properties will be generated for Sqoop. If you are upgrading from an older cluster then you need to enable the Hook manually: Go to the Sqoop configuration page, search for "Atlas", enable the checkbox and then re-deploy the client configurations.
Atlas : Add hadoop-metrics2.properties in conf directory
Added configuration of hadoop-metrics2.properties in Atlas.
Add support for custom Zookeeper principal in Atlas
Added support to use actual zookeeper principal in Atlas.

Kafka

New configuration parameter for Kafka Connect role
A new property called include.connector.context is added for the KafkaConnect role, that is enabled by default. If it is enabled, additional connector context information is added to Kafka connect file logs.
Add Kafka health test for RequestHandlerAvgIdlePercent and NetworkProcessorAvgIdlePercent
Added two new health tests for Kafka: - If NetworkProcessorAvgIdlePercent is below the threshold 0.3, we advise the user to increase num.network.threads and make the broker health concerning - If RequestHandlerAvgIdlePercent is below the threshold 0.3, we advise 1the user to increase num.io.threads and make the broker health concerning

Knox

Add SSL support to Knox Gateway DB
SSL-related connection properties were not exposed in the Cloudera Manager Admin Console for the KNOX GATEWAY Database.
Add Database support for Knox
Previously, the KNOX_GATEWAY role lacked database support which is needed for the Knox Token generation feature (instead of storing the tokens in Zookeeper or in keystores on the local file system)
Knox autodiscovery for SQLStreamBuilder Service
SQLStreamBuilder was not auto-discovered by Knox. Users had to add the service manually into Knox topologies if they wanted to use SSB in CDP. From now on, auto service-discovery for SQL Stream Builder is available. Any previous manual configurations must be reverted.
Knox principal is not overridable in Streams Messaging Manager and Schema Registry
Custom Knox principal can be set for Schema Registry and Streams Messaging Manager by setting the knox_principal_name property in the Schema Registry Server Advanced Configuration Snippet (Safety Valve) for registry.yaml or the Streams Messaging Manager Rest Admin Server Advanced Configuration Snippet (Safety Valve) for streams-messaging-manager.yaml

Ranger

Token created by user should be exchanged using token exhange API without requiring admin privileges in the environment
A new Ranger policy was created to allow public access to the new tokenexchange Knox topology.
New introduce Max Retention Days configuration parameter for Ranger audits
Users can now update the Solr document expiry ranger.audit.solr.config.ttl and ranger.audit.solr.config.delete.trigger parametersin Cloudera Manager for Ranger configurations and refresh configurations to get the Solr collection for Ranger audits updated with ttl and delete trigger.

Streams Messaging Manager

Streams Messaging Manager is now configurable to allow custom list of ciphers and SSL protocols
Streams Messaging Manager now offers the following1 configurations to customize the SSL configurations of the Streams Messaging Manager Server: streams.messaging.manager.ssl.supportedCipherSuites, streams.messaging.manager.ssl.excludedCipherSuites, streams.messaging.manager.ssl.supportedProtocols, streams.messaging.manager.ssl.excludedProtocols.
In a FIPS enabled environment, to support access from a browser, excludedCipherSuites should be updated to allow ciphers ending with "_SHA".

Streams Replication Manager

Streams Replication Manager should be configurable to allow custom list of ciphers and SSL protocols
Streams Replication Manager now offers the following configurations to customize the SSL configurations of Streams Replication Manager Service: streams.replication.manager.ssl.supportedCipherSuites, streams.replication.manager.ssl.excludedCipherSuites, streams.replication.manager.ssl.supportedProtocols, streams.replication.manager.ssl.excludedProtocols.
Introduce health test for Streams Replication Manager service
New health tests were introduced to the SRM service role which describes the state of the SRM service. With the help of these, when the Streams Application inside the SRM services goes to ERROR state or loses connectivity with the target Kafka Cluster, SRM tries to restart it, and Cloudera Manager shows that SRM Service is not functional.