Behavioral Changes in Kafka

Cloudera Runtime 7.3.2.10000 SP1

Kafka Connect Secrets Storage key wrap algorithm is configurable

Previous behavior:

The algorithm that Kafka Connect Secrets Storage uses to wrap the encryption key was hardcoded to AESWrap. You could not change this value in Cloudera Manager.

New behavior:

You can configure the wrap algorithm using the Kafka Connect Secrets Storage Secret Key Wrap Algorithm (kafka.connect.secret.key.wrap.algorithm) property in Cloudera Manager. The default remains AESWrap. On FIPS-enabled clusters where the Bouncy Castle FIPS jars are configured, Cloudera recommends setting this property to AESKWP. Other FIPS-approved algorithms supported by the configured security provider may also be used; AESKWP is the algorithm validated by Cloudera. Changing the wrap algorithm does not re-encrypt secret values stored in the secrets topic. If you change the value from the default and need to run the connect-secret-storage-migration tool, pass kafka.connect.secret.key.wrap.algorithm in the source topic configuration file, similar to kafka.connect.secret.global.password and kafka.connect.secret.pbe.salt.

HTTP metrics SSL enabled by default when Encrypt All Ports is on

Previous behavior:

The default value of SSL Encryption For HTTP Metric Reporter (kafka.http.metrics.ssl.enabled) was false on Kafka Broker and KRaft Controller roles.

New behavior:

When you enable the ENCRYPT_ALL_PORTS feature flag, the effective default becomes true. If you previously set a custom value, your setting is preserved. For information about the feature flag, see Transport Layer Security Support and Custom Protocol Configuration.

Cloudera Runtime 7.3.2.100 CHF 1

There are no behavioral changes in this release.

Cloudera Runtime 7.3.2

Cloudera Runtime 7.3.2 introduces functional adjustments, behavioral updates for Kafka, and includes all service packs and cumulative hotfixes from 7.3.1.100 through 7.3.1.706. For a comprehensive record of all functional adjustments in Cloudera Runtime 7.3.1.x, see Behavioral Changes.

Component-level custom Java home configuration removed

Previous behavior:

You could configure a component-specific Java home for Kafka Connect.

New behavior:

The component-level custom Java home configuration options are removed. Kafka Connect now uses the host-level java_home configuration. If you previously set a component-specific Java home for this service, verify the host-level java_home setting after upgrading.

The Kafka service is configured to use KRaft by default

Previous behavior:

Newly deployed Kafka services were configured to use ZooKeeper as their metadata store. The Kafka Metadata Store Service (metadata.store) Kafka property was set to ZooKeeper.

New behavior:

Newly deployed Kafka services are configured to use KRaft as their metadata store. The Kafka Metadata Store Service (metadata.store) Kafka property is set to KRaft.

High Watermark no longer advances when ISR is below MinISR

Previous behavior:

The High Watermark (HWM) advanced regardless of whether the in-sync replica (ISR) count was below min.insync.replicas. When producers used acks=1 or acks=0, messages were written to the leader and became consumable once the HWM advanced, even if the ISR had dropped below the minimum threshold. The min.insync.replicas setting only affected acks=all produce requests, blocking writes when ISR was insufficient, but did not prevent HWM advancement or consumer reads for acks=0/acks=1 messages.

New behavior:

The HWM no longer advances when the ISR count falls below min.insync.replicas. As a result, consumers are blocked from reading new messages in this condition, even if producers with acks=1 or acks=0 are still writing to the leader. This ensures data is only consumable when it meets the cluster's minimum durability requirements. If you use min.insync.replicas=2 or higher, you may see reduced consumer throughput when the ISR count drops below the configured minimum.

For more information, see KIP-966: Eligible Leader Replicas and KAFKA-15583.

Kafka protocol version is set automatically during upgrades

Previous behavior:

The inter.broker.protocol.version property for ZooKeeper-based clusters and the metadata.version property for KRaft-based clusters were not set automatically before an upgrade. Manually configuring these properties to the current protocol and metadata version was required before an upgrade.

New behavior:

During a cluster upgrade, Cloudera Manager now automatically sets the inter.broker.protocol.version property for ZooKeeper-based clusters and the metadata.version property for KRaft-based clusters. Manual configuration is no longer required.