Cloudera Docs

Release notes

Learn about the new features, improvements, known and fixed issues, limitations, unsupported features, as well as deprecations and removals in this release of Cloudera Streams Messaging - Kubernetes Operator.

What's New

Learn about the new features and notable changes in this release.

Rebase to Strimzi 0.45.0 and Kafka 3.9.

This release of Cloudera Streams Messaging - Kubernetes Operator is based on Strimzi 0.45.0 and Kafka 3.9.

See the following upstream resources for more information on these versions.

Upstream highlights
The following is a list of highlighted changes included in the upstream version of Strimzi, Kafka, and other components. For a full list of upstream changes, see the release notes and notable changes above.
  • strimzi-kafka-operator #10563: Add mechanism to manage offsets through KafkaConnector and KafkaMirrorMaker2 resources

    You can now manage connector offsets directly by configuring your KafkaConnector resources. Cloudera recommends that you use this feature over the Kafka Connect REST API to manage connector offsets. The recommended method for managing replication offsets when replicating data with Kafka Connect-based replication is also changed. For more information, see Managing connector offsets and Configuring data replication offsets.

  • strimzi-kafka-operator #10583: Auto-rebalancing on scaling the cluster down/up

    You can now enable auto-rebalancing for Kafka clusters. If auto-rebalancing is enabled, the Strimzi Cluster Operator automatically initiates a rebalance with Cruise Control when you scale the Kafka cluster. Cloudera recommends that you enable this feature as it makes scaling easier and faster. For more information, see Scaling brokers.

Supported Kubernetes version update

Starting with this release, Cloudera Streams Messaging - Kubernetes Operator supports the following Kubernetes versions.

  • Kubernetes 1.25 or later:
    • OpenShift 4.12 or later
    • RKE2 (Rancher Kubernetes Engine 2) 1.25 or later

KRaft

KRaft (Kafka Raft) is generally available. You can now deploy Kafka clusters that use KRaft instead of ZooKeeper for metadata management. Additionally, you can migrate existing ZooKeeper-based Kafka clusters to use KRaft.

With the addition of KRaft, ZooKeeper is deprecated. Deploying new or using existing Kafka clusters running in ZooKeeper mode is deprecated. Additionally, ZooKeeper will be removed in a future release. When deploying new Kafka clusters, deploy them in KRaft mode. Cloudera encourages you to migrate existing clusters to KRaft.

Fixed Issues

Learn what issues have been fixed since the previous release.

CSMDS-933: The log-dir-describe.txt and topic-describe.txt files generated by report.sh are empty
The report.sh tool now correctly collects log directory and topic information even if you run multiple concurrent instances of the tool, or run the tool with a high parallelization factor. Both log-dir-describe.txt and topics-describe.txt are correctly populated with information.

Known Issues

Learn about the known issues in this release.

CSMDS-334: ZooKeeper pods are running but Kafka pods are not created
Under certain circumstances, ZooKeeper pods might not be able to form a quorum. In a case like this, the creation of the Kafka cluster gets stuck in a state where ZooKeeper pods are running, but Kafka pods are not created.
If you encounter this issue, at least one of the ZooKeeper pods logs a WARN entry similar to the following: 
2024-02-23 18:45:00,311 WARN Unexpected exception (org.apache.zookeeper.server.quorum.QuorumPeer) [QuorumPeer[myid=3](plain=127.0.0.1:12181)(secure=[0:0:0:0:0:0:0:0]:2181)]
java.lang.InterruptedException: Timeout while waiting for epoch from quorum
	at org.apache.zookeeper.server.quorum.Leader.getEpochToPropose(Leader.java:1443)
	at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:606)
	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1552)

This is caused by a race condition issue in ZooKeeper. ZooKeeper is unable to recover from this state automatically.

Delete the ZooKeeper pods that are unable to form a quorum.
kubectl delete pod [***ZOOKEEPER POD***] -n [***NAMESPACE***]

The Strimzi Cluster Operator automatically recreates the ZooKeeper pods that are deleted. The newly created ZooKeeper pods are less likely to encounter the issue.

CSMDS-953: Kafka and ZooKeeper might experience downtime during upgrades
Under certain circumstances, ZooKeeper pods might not be able to form a quorum during an upgrade. This results in both ZooKeeper and Kafka becoming unavailable for several minutes during an upgrade.

After a certain amount of time, failed ZooKeeper pods are automatically recreated by the Strimzi Cluster Operator, and the upgrade succeeds.

If you encounter this issue, at least one of the ZooKeeper pods will log the following error:
java.net.BindException: Cannot assign requested address

This issue affects ZooKeeper-based deployments only.

Unsupported features

Learn what features are unsupported in this release.

The following Strimzi features are unsupported in Cloudera Streams Messaging - Kubernetes Operator:

  • Kafka MirrorMaker
  • Kafka MirrorMaker 2
  • Kafka Bridge
  • Kafka cluster creation without using KafkaNodePool resources

Deprecations and removals

Learn what is deprecated or removed in this release.

Deprecations

ZooKeeper
ZooKeeper is deprecated. Deploying new or using existing Kafka clusters running in ZooKeeper mode is deprecated. ZooKeeper will be removed in a future release. When deploying new Kafka clusters, deploy them in KRaft mode. Cloudera encourages you to migrate existing clusters to KRaft.

Removals

There are no removals in this release.