CDK Powered By Apache Kafka Incompatible Changes and Limitations
This section describes incompatible changes and limitations:
Flume version requires to be matched to a minimum Kafka version
The CDH Flume version requires to be matched to a minimum Kafka version. This means that if you upgrade to Kafka 2.2, you need to upgrade brokers as well as clients.
- In CDH 5.7 and higher, the Flume connector to Kafka only works with Kafka 2.0 and higher.
- In CDH 5.14 and higher, the Flume connector to Kafka only works with Kafka 2.2 and higher.
We recommend that you have the Kafka broker running the same or newer version as the client.
Flume shipped with CDH 5.7 and lower can only send data to CDK 2.0 and higher Powered By Apache Kafka via unsecured transport
Security additions to CDK 2.0 Powered By Apache Kafka are not supported by Flume in CDH 5.7 (or lower versions).Topic Blacklist Removed
The MirrorMaker Topic blacklist setting has been removed in CDK 2.0 and higher Powered By Apache Kafka.
Avoid Data Loss Option Removed
The Avoid Data Loss option from earlier releases has been removed in CDK 2.0 Powered By Apache Kafka in favor of automatically setting the following properties.
- Producer settings
- acks=all
- retries=max integer
- max.block.ms=max long
- Consumer setting
- auto.commit.enable=false
- MirrorMaker setting
- abort.on.send.failute=true