Cloudera Docs

Release notes

Learn about the new features, improvements, known and fixed issues, limitations, and unsupported features in this release of Cloudera Streams Messaging - Kubernetes Operator.

What's New

Learn about the new features and notable changes in this release.

Rebase on Strimzi 0.43.0 and Kafka 3.8.

This release of Cloudera Streams Messaging - Kubernetes Operator is based on Strimzi 0.43.0 and Kafka 3.8.

See the following upstream resources for more information on these versions.

Upstream highlights
The following is a list of highlighted changes included in the upstream version of Strimzi, Kafka, and other components. For a full list of upstream changes, see the release notes and notable changes above.
  • KAFKA-15905: Restarts of MirrorCheckpointTask should not permanently interrupt offset translation

    Checkpointing task restarts do not reset the offset sync history. As a result, on average, a wider range of offsets can be translated by checkpointing than before.

  • strimzi-kafka-operator #10566: External Configuration Volumes is deprecated

    The spec.externalConfiguration.volumes property in KafkaConnect and KafkaMirrorMaker2 resources are deprecated and will be removed in the future. Use the additional volumes and volume mounts with pod and container templates instead to mount additional Secrets or ConfigMaps. For more information, see Configuring additional volumes and volume mounts.

  • strimzi-kafka-operator #10117: Add support for custom Cruise Control API users

    You can now set up custom REST API users for Cruise Control. Users that you configure will have access to the Cruise Control REST API. Having access to the API enables you to securely, query, monitor, or debug Cruise Control. Only the VIEWER and USER Cruise Control roles are supported. You can not create ADMIN role users. As a result, the users you set up will only be able to make GET operations. For more information, see Accessing the Cruise Control REST API.

Apache Ranger authorization

Support for Apache Ranger authorization is introduced. You can now integrate your Kafka clusters deployed with Cloudera Streams Messaging - Kubernetes Operator with a remote Ranger service that is running on Cloudera Private Cloud Base. If configured, the Ranger service can provide authorization for your Kafka cluster. For more information, see Apache Ranger authorization.

Replication of heartbeat records can now be turned off in the MirrorSourceConnector

A new property, heartbeats.replication.enabled, is introduced for the MirrorSourceConnector. The property controls whether heartbeats topics of a replication flow are replicated. If set to true, heartbeats topics identified by the replication policy will always be replicated, regardless of the topic filter configuration. If set to false, heartbeats topics will only be replicated if the topic filter allows.

This is a backported Kafka improvement. For additional information, see KAFKA-17534.

Performance improvements for the report.sh tool

The report.sh tool can now run its subprocesses in parallel and does so by default. This improves performance and results in the tool running faster. Additionally, the following three new options are introduced that make it possible to fine-tune the tool’s behavior.

  • --parallel-ns: Script can execute 'N' namespace dumps in parallel. It will be 5 by default.
  • --parallel-cluster: Script can execute 'N' Kafka and Connect cluster dumps per namespace in parallel. It will be 3 by default.
  • --parallel-kubectl: Script can execute 'N' Kubernetes client call in parallel in a subsection. It will be 10 by default. This means the overall maximum Kubernetes client call count is equal to parallel-ns * parallel-cluster * parallel-kubectl (so 150 with defaults).

For more information, see Diagnostics.

Fixed Issues

Learn what issues have been fixed since the previous release.

CSMDS-805: The kafka_shell.sh and connect_shell.sh tools do not propagate command return code
The kafka_shell.sh and connect_shell.sh now propagate the return code of the last command which ran inside the shell.

Known Issues

Learn about the known issues in this release.

CSMDS-334: ZooKeeper pods are running but Kafka pods are not created
Under certain circumstances, ZooKeeper pods might not be able to form a quorum. In a case like this, the creation of the Kafka cluster gets stuck in a state where ZooKeeper pods are running, but Kafka pods are not created.
If you encounter this issue, at least one of the ZooKeeper pods logs a WARN entry similar to the following: 
2024-02-23 18:45:00,311 WARN Unexpected exception (org.apache.zookeeper.server.quorum.QuorumPeer) [QuorumPeer[myid=3](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)]
java.lang.InterruptedException: Timeout while waiting for epoch from quorum
	at org.apache.zookeeper.server.quorum.Leader.getEpochToPropose(Leader.java:1443)
	at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:606)
	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1552)

This is caused by a race condition issue in ZooKeeper. ZooKeeper is unable to recover from this state automatically.

Delete the ZooKeeper pods that are unable to form a quorum.
kubectl delete pod [***ZOOKEEPER POD***] -n [***NAMESPACE***]

The Cluster Operator automatically recreates the ZooKeeper pods that are deleted. The newly created ZooKeeper pods are less likely to encounter the issue.

Unsupported features

Learn what features are unsupported in this release.

The following Strimzi features are unsupported in Cloudera Streams Messaging - Kubernetes Operator:

  • Kafka MirrorMaker
  • Kafka MirrorMaker 2
  • Kafka Bridge
  • Kafka cluster creation without using KafkaNodePool resources