Behavioral Changes in Apache Kafka

Learn about the change in certain functionality of Kafka that has resulted in a change in behavior from the previously released version to this version of Cloudera Runtime.

Summary:

Previously, Kafka data directories were configured as world-readable on the filesystem of the broker host. From now on, they are only readable by Kafka.

Previous behavior:

Kafka data directories were created with 755 permissions.

New behavior:

Kafka data directories are created with 0750 permissions.

Summary:

JMX authentication is enabled by default for the Kafka service. Additionally random passwords are now generated for both JMX users by default.

Previous behavior:

JMX authentication (Enable Authenticated Communication with the JMX Agent) was disabled by default.

New behavior:

JMX authentication (Enable Authenticated Communication with the JMX Agent) is now enabled by default. By default random passwords are generated for both JMX users.

Summary:

The default TLS protocol when using Java 11 or later is updated to TLS 1.3. TLS 1.2 remains the default for earlier Java versions. For more information, see KIP-573.

Previous behavior:

The default TLS protocol used was TLS 1.2.

New behavior:

The default TLS protocol when using Java 11 or later is TLS 1.3.

Summary:

The client.dns.lookup client property is now set to use_all_dns_ips by default. With this option set, clients attempt to connect to the broker using all possible IP addresses of a hostname. This option reduces connection failure rates and is better suited for cloud and containerized environments where a single hostname might resolve to multiple IP addresses. For more information, see KIP-602.

Previous behavior:

The default value of the client.dns.lookup client property was set to default.

New behavior:

The default value of the client.dns.lookup property is set to use_all_dns_ips.

Summary:

The default value for the following Kafka Connect properties is increased to 3:

• Offset Storage Topic Replication Factor (offset.storage.replication.factor)
• Configuration Storage Topic Replication Factor (config.storage.replication.factor)
• Status Storage Topic Replication Factor (status.storage.replication.factor)

Previous behavior:

The default value of the properties was set to 1.

New behavior:

The default value of the properties is set to 3.

Summary:

The FileStream example connectors (FileStreamSourceConnector and FileStreamsSinkConnector) are no longer available for deployment by default using the Kafka Connect Rest API, SMM UI, or SMM REST API. The JAR file of the connector is still shipped with Cloudera Runtime, but the connectors must be installed before they can be deployed. For more information on how to install Kafka Connect connectors, see Installing Connectors

Previous behavior:

TheFileStream example connectors were available by default for deployment. In SMM, the connectors were selectable on the Connect Cluster > Connector Setup page by default.

New behavior:

TheFileStream example connectors must be installed before they can be deployed. In SMM, the connectors are no longer selectable on the Connect Cluster > Connector Setuppage by default.