Fixed Issues in Apache Kafka

Review the list of Apache Kafka issues that are resolved in Cloudera Runtime 7.2.16.

CDPD-40985: Expose log directory total and usable space through the Kafka API (KAFKA-13958 backport)

This is a backported improvement, see KIP-827 and KAFKA-13958 for more information.

CDPD-35680: Remove the verification of the existence of the JWT "sub" claim to extend compatibility with OAuth providers (KAFKA-13730 backport)

This is a backported improvement, see KAFKA-13730 for more information.

Topics created with the kafka-topics tool are only accessible by the user who created them when the deprecated --zookeeper option is used

The --zookeeper option has been removed from the kafka-topics tool. As a result, encountering this issue is no longer possible. Use the --bootstrap-server option instead.

Certain Kafka command line tools require direct access to ZooKeeper

There are no longer any Cloudera supported Kafka command line tools that require direct ZooKeeper access or require the usage of the --zookeeper option. Use the --bootstrap-server option instead.

CDPD-39422: Consumer polls for coordinator in tight loop (KAFKA-13917 backport)

The consumer uses a backoff period controlled by retry.backoff.ms to wait before polling for the coordinator if the previous poll was unsuccessful. This is a backported improvement, see KAFKA-13917 for more information.

CDPD-39354: Kafka Connect connectors and tasks fail to start

Kafka Connect initializes the Secrets Storage right at startup. As a result, timing issues no longer occur in the configuration resolution of connectors and tasks.

CDPD-45958: Kafka client JAAS override policy validation is incorrect

The JAAS override filter policy now correctly filters based on the specified rules and does not refuse JAAS configurations because of unknown fields.

CDPD-44252: Exception during normal operation in MirrorSourceTask causes the task to fail instead of shutting down gracefully

Stopping the read of offsets in a worker of a MirrorSourceTask will now cause a graceful shutdown and the task can be restarted automatically at a later point.

CDPD-39391: Amazon S3 Sink fails when validating bucket names

Due to an issue with the AWS S3 bucket name validation of the Amazon S3 Sink connector, the connector encountered an exception when validating bucket names. This issue is now fixed.

OPSAPS-64606: Authorization issues if Kafka Connect is not installed

If the Kafka Connect role is not present on the cluster, then a Ranger policy (connect internal - topic) is created with default, non-empty topic names. As a result, the Ranger policy include list cannot be empty and will not have any side effects on other Kafka operations.

OPSAPS-63526: Kafka's Ranger related log4j2 config generation is not idempotent

If Kafka fails to start up due a configuration change, the original cause for the startup failure is no longer replaced with an error related to log4j2 configuration after the first auto-attempted restart.

OPSAPS-63640: Monitoring a high number of Kafka producers might cause Cloudera Manager to slow down and run out of memory

This issue is fixed.