Fixed Issues in Streams Messaging

Review the list of Streams Messaging issues that are resolved in Cloudera DataFlow for Data Hub 7.2.10.

Kafka

CDPD-24828: AvroConnectTranslator does not handle optional nested records properly

Avro to Connect schema conversion no longer fails when optional nested records are used in the Avro document.

Streams Messaging Manager

CDPD-24173: Restrict the allowed HTTP methods for SMM REST API

The following http methods are allowed in Streams Messaging Manager: GET, POST, PUT, DELETE, HEAD, and OPTIONS. TRACE method is disabled.

CDPD-24698: Configurable ConsumerEmission timeout

New SMM config: cm.metrics.emit.consumer.metrics.timeout.

SMM pushes the Consumer metrics into Cloudera Manager (only when Cloudera Manager is used as a metricsStore). This configuration allows you to configure the timeout of the emission API calls. By default the timeout is 10 seconds.

Schema Registry

CDPD-24415: Schema Registry RAZ NoClassDefFoundError

Due to changes in Ranger RAZ, Schema Registry was unable to upload serdes files to ABFS and S3. We traced the issue to Hadoop - via Ranger - bringing in transitive dependencies which were conflicting with Schema Registry's own classes. As a solution we isolated Hadoop's classes into a separate classpath and load them only when needed. This way future changes in Hadoop and Ranger classpaths will not affect Schema Registry.

CDPD-21913: Rename properties in registry yaml file

Schema Registry uses the Dropwizard framework which allows overriding configuration properties from the command line. Due to implementation specifics, some of the properties could not be overridden. We have fixed this issue now.

In Schema Registry's configuration file, FileStorageConfiguration properties can be directory, fsUrl, kerberosPrincipal and keytabLocation.

In Schema Registry's configuration file, StorageProviderConfiguration properties can be dbtype, queryTimeoutInSecs and properties that can be dataSourceClassName, dataSourceUrl, dataSourceUser, dataSourcePassword and connectionProperties that can be oracleNetSslVersion, oracleNetSslServerDnMatch, trustStore, trustStoreType, keyStore, keyStoreType.

Streams Replication Manager

There are no fixed Streams Replication Manager issues in this release.