CDH 6 includes Apache Kafka as part of the core package. The documentation includes improved contents for how to set up, install, and administer your Kafka ecosystem. For more information, see the Cloudera Enterprise 6.0.x Apache Kafka Guide. We look forward to your feedback on both the existing and new documentation.

CDK Powered By Apache Kafka Requirements and Supported Versions

The following sections describe software requirements and supported versions of complementary software for CDK Powered By Apache Kafka:

Supported CDH and Cloudera Manager Releases
Supported Operating Systems
Supported JDK Versions
Ports Used by Kafka

Supported CDH and Cloudera Manager Releases

For the list of supported releases of CDH and Cloudera Manager, see CDH and Cloudera Manager Supported Versions.

Supported Integrations

Flume and Spark Connectors to Kafka

Flume and Spark connectors to Kafka are included with CDH 5.7.x and higher and only work with CDK 2.0.x Powered By Apache Kafka and higher.

Sentry Integration with Kafka

Sentry authorization integration with Kafka is available with CDK Powered By Apache Kafka version 2.1.x and higher on CDH 5.9.x and higher.

CDH 5.13.x introduces Sentry-HA; CDK 3.1 provides a Sentry-HA aware client. You can connect with an older version, but it won't be in high-availability mode: if the Sentry server that serves Kafka goes away, Kafka will not have access to authorization information.

Supported Operating Systems

For the list of supported operating systems, see CDH and Cloudera Manager Supported Operating Systems.

SUSE Linux Enterprise Server (SLES)

Unlike CentOS, SLES limits virtual memory by default. Changing this default requires adding the following entries to the /etc/security/limits.conf file:

* hard as unlimited
* soft as unlimited

Supported JDK Versions

CDK 3.0 and higher Powered By Apache Kafka require JDK 8, and do not support JDK 7.

For a listed of supported and tested JDK versions for CDK 2.2.x and below Powered By Apache Kafka, see CDH and Cloudera Manager Supported JDK Versions.

Ports Used by Kafka

Kafka uses the TCP ports listed in the following table. Before deploying Kafka, ensure that these ports are open on each system.

Component	Service	Port	Access Requirement	Comment
Broker	TCP Port	9092	External/Internal	The primary communication port used by producers and consumers; also used for inter-broker communication.
Broker	TLS/SSL Port	9093	External/Internal	A secured communication port used by producers and consumers; also used for inter-broker communication.
Broker	JMX Port	9393	Internal	Internal use only. Used for administration via JMX.
MirrorMaker	JMX Port	9394	Internal	Internal use only. Used to administer the producer and consumer of the MirrorMaker.
Broker	HTTP Metric Report Port	24042	Internal	Internal use only. This is the port via which the HTTP metric reporter listens. It is used to retrieve metrics through HTTP instead of JMX.