CDH 6 includes Apache Kafka as part of the core package. The documentation includes improved contents for how to set up, install, and administer your Kafka ecosystem. For more information, see the Cloudera Enterprise 6.0.x Apache Kafka Guide. We look forward to your feedback on both the existing and new documentation.
CDK Powered By Apache Kafka Incompatible Changes and Limitations
This section describes incompatible changes and limitations:
CDK 3.0 Requires CDH 5.13 when Co-located
Using version 3.0 and later of CDK Powered by Apache Kafka requires a newer version of Cloudera Manager and/or CDH when Kafka and CDH are in the same logical cluster in Cloudera Manager. For more information on compatibilities among versions, see Product Compatibility Matrix for CDK Powered By Apache Kafka.
Flume shipped with CDH 5.7 and lower can only send data to CDK 2.0 and higher Powered By Apache Kafka via unsecured transport.
Security additions to CDK 2.0 Powered By Apache Kafka are not supported by Flume in CDH 5.7 (or lower versions).Topic Blacklist Removed
The MirrorMaker Topic blacklist setting has been removed in CDK 2.0 and higher Powered By Apache Kafka.
Avoid Data Loss Option Removed
The Avoid Data Loss option from earlier releases has been removed in CDK 2.0 Powered By Apache Kafka in favor of automatically setting the following properties.
- Producer settings
- acks=all
- retries=max integer
- max.block.ms=max long
- Consumer setting
- auto.commit.enable=false
- MirrorMaker setting
- abort.on.send.failute=true