Cloudera Docs

What's New in Streams Messaging

Learn about the new Streams Messaging features in Cloudera DataFlow for Data Hub 7.2.6.

Kafka

Remove usage of non-FIPS compatible algorithms in Kafka
Murmur3 hashing is introduced for the log cleaner when it builds offset maps. In addition, to the introduction of the new hashing algorithm, the default algorithm used is also changed to Murmur3. The previous default was MD5. If required, MD5 can still be used by adding cloudera.log.cleaner.hashing.algorithm=MD5 to the Kafka Broker Advanced Configuration Snippet (Safety Valve) for kafka.properties property in Cloudera Manager.

Schema Registry

There are no new features for Schema Registry in this release.

Streams Messaging Manager

CSD support to configure caching in SMM authorizer
SMM request processing is sped up by introducing an authorization cache. The default TTL of the cache is 30 seconds and it is configurable in Cloudera Manager. Setting the TTL to 0 disables the cache entirely.
The affected versions are Cloudera Manager 7.2.4 and higher and CDH 7.2.1 and higher, Cloudera Manager 7.3.0 and higher and CDH 7.1.6 and higher.
SMM automatically configures SRM in Cloudera Manager
SMM automatically configures the SRM connection based on a service dependency. Manual configuration options are removed. This feature affects Cloudera Manager versions 7.2.3 and higher and CDH versions 7.2.3 and higher, Cloudera Manager versions 7.3.0 and higher and CDH versions 7.1.6 and higher.

Streams Replication Manager

SRM is available in Data Hub and CDP Public Cloud
SRM can now be provisioned in CDP Public Cloud with Data Hub. The default Streams Messaging cluster definitions are updated to include SRM. For more information, see Streams Messaging cluster layout and Creating your first Streams Messaging cluster.
SRM high availability mode
You can now deploy SRM in high availability mode. For more information, see Enable high availability for Streams Replication Manager.
Replication-specific Kafka Connect REST servers
SRM can now run multiple drivers in the same cluster (high availability). To make this possible, the SRM driver role now deploys a Kafka Connect REST server for each replication that you set up and configure. These REST servers ensure communication between the different instances of the driver role and make replication with multiple drivers in a single cluster possible.
If required, you can configure these REST servers in Cloudera Manager with the Streams Replication Manager's Replication Configs property and two specific prefixes. For more information, see Configuring replication specific Kafka Connect REST servers .
The grace and retention period as well as the collection frequency of SRM Service role metrics are configurable
Configuration properties related to the SRM Service role’s metric processing are added. These properties give users the ability to configure the grace and retention periods as well as the collection frequency of SRM Service role metrics. The grace and retention periods can be configured directly with the following Cloudera Manager properties:

  • SRM Service Metrics Grace Period (streams.replication.manager.service.streams.metrics.grace)

  • SRM Service Metrics Retention Period (streams.replication.manager.service.streams.metrics.retention)

Metric collection frequency can be configured through Streams Replication Manager's Replication Configs with the metrics.period property. The metrics.period property can only be configured on a replication level. For example: 
primary->secondary.metrics.period=60