Fixed Issues in Streams Messaging

Review the list of Streams Messaging issues that are resolved in Cloudera DataFlow for Data Hub 7.2.15.

Kafka

CDPD-29058: Migrate to log4j2 due to log4j1 end of life
Kafka is migrated and uses log4j2 as a logging library. Additionally, log4j1 dependencies are removed with the exception of the Log4jAppender. Although the appender remains available, Cloudera recommends that you use the log4j2 implementation of the appender that is available in the log4j2 project.
CDPD-29307: Kafka keystore and truststore type is not configured for Cruise Control metrics reporter
The keystore and truststore types are now correctly supported by the Cruise Control metrics reporter in the Kafka broker.
OPSAPS-62548: TopicMetrics get deleted from Cloudera Manager during restart or Kafka partition reassignment
KafkaTopicMetrics are no longer deleted from the ServiceMonitor's Time-series database during a Kafka restart or a partition leader change.

Schema Registry

CDPD-35983: Unique constraint violation on load balanced Schema Registry cluster startup
A concurrency issue in a multi-node Schema Registry setup is fixed where more nodes tried to initialize database state at the same time causing some of them to fail.
CDPD-35469: Schema Registry responds with Internal Server Error when adding more schemas than defined in offset range
Schema Registry responds with HTTP 409 response instead of HTTP 500 response while trying to add more schemas than defined in offset range.
CDPD-33908: Remove or Upgrade Spring framework to 5.3.14+/5.2.19 due to CVE-2021-22060
Removed Spring dependencies from Schema Registry because they were not used at all.
CDPD-32192: First start failed for Schema Registry, with oracle DB, migration failed at CREATE TABLE "atlas_events"
Fixed v009__create_registry_audit.sql to have create index refer to the lower case "atlas_events" object (the table). Made the script re-runnable since the table was already created where the script had already run.
CDPD-31881: Schema Registry L1 test fails with socket timeout
When more than one instance of Schema Registry is running on the same DB, "concurrent update" exceptions might have appeared in the Schema Registry log regarding changes to be sent to Atlas.

Streams Messaging Manager

CDPD-33770: On the topics details page selecting a custom timestamp is broken
Fixed SMM REST throwing an internal server error when custom timestamps are provided while calling the "/api/v2{or v1}/admin/replication-stats" endpoint, or when a custom time period is provided on the ProducerDetail page in SMM UI.
CDPD-33011: Selecting a consumer with no producers should show 0 producers in the filter panel
On the overview page in the filter panel, when a consumer is selected that has no producers associated, the number of producers will be shown to be 0 of T, where T is the total number of producers.
CDPD-32936: Selecting a producer with no consumers should show 0 consumers in the filter panel
On the overview page in the filter panel, when a producer is selected that has no consumers associated, the number of consumers will be shown to be 0 of T, where T is the total number of consumers.
CDPD-29403: When editing the alert, the topic can be chosen for the replication status
Fixed the topic selection dropdown status in the alert editor after various UI events.
OPSAPS-63017: The Kafka Connect tab is missing from the SMM UI
The Kafka Connect tab is now correctly displayed if Kafka Connect is provisioned on the cluster.
OPSAPS-62548: TopicMetrics get deleted from CM during restart or kafka partition reassignment
KafkaTopicMetrics accidentally gets deleted from ServiceMonitor's Timeseries database during a Kafka restart or partition leader change.

Streams Replication Manager

CDPD-31745: SRM Control fails to configure internal topic when target is earlier than Kafka 2.3
SRM now creates all internal topics explicitly. SRM also verifies the essential configurations of internal topics at startup, and fails if the topics do not meet the required configurations.
OPSAPS-63104: The automatically generated password for co-located services is invalid
SRM Service Basic Authentication would not work with the default, random generated password. SRM Service Basic Authentication default password is now identical on all SRM Service role instances.
OPSAPS-62546: Kafka External Account SSL keypassword configuration is used incorrectly by SRM
SRM uses the correct ssl.keystore.key configuration when a Kafka External Account specifies the keystore.

Cruise Control

Support added for keystore and truststore types other than JKS
You are able to configure the keystore and truststore in Kafka brokers for Cruise Control Metrics Reporter. Previously, only the JKS type was supported for the SSL keystore and truststore.
Migrating Cruise Control to Log4j2
You are able to configure the keystore and truststore in Kafka brokers for Cruise Control Metrics Reporter. Previously, only the JKS type was supported for the SSL keystore and truststore.
Cruise Control fails to start after upgrade with Rack Aware Goal configured
You are able to configure the keystore and truststore in Kafka brokers for Cruise Control Metrics Reporter. Previously, only the JKS type was supported for the SSL keystore and truststore.

Technical Service Bulletins

7.2.15.4

TSB 2022-614: Kafka policy "user auto-creation" does not work in Ranger in CDP Public Cloud 7.2.15
For the latest update on this issue see the corresponding Knowledge article: TSB 2022-614: Kafka policy "user auto-creation" does not work in Ranger in CDP Public Cloud 7.2.15