Kafka Guide
Also available as:

Chapter 2. Installing and Configuring Kafka

Prerequisite: ZooKeeper must be installed and running before using Kafka.

To install Kafka using Ambari, see Adding a Service to your Hadoop cluster in the Ambari User's Guide.

To configure Kafka for Kerberos security on an Ambari-managed cluster, see Configuring Kafka for Kerberos Over Ambari.


To install Kafka manually, see Installing and Configuring Kafka in the Non-Ambari Cluster Installation Guide.

To configure Ranger-based authorization for Kafka, see the Kafka section of the Ranger Ambari Installation Guide.


HDP 2.4 supports JDK 1.7 and JDK 1.8 for Kafka.


Supported File Systems

The following underlying file systems are supported for use with Kafka:

  • EXT4: supported and recommended

  • EXT3: supported


Encrypted file systems such as SafenetFS are not supported for Kafka. Index file corruption can occur.

Saving Kafka Audits to HDFS

Audits to HDFS are activated automatically when Ranger is enabled for Kafka. You can also manually update the audit settings. For more information, see Manually Updating Ambari HDFS Audit Settings in the HDP Security Guide.

For secure clusters, perform the additional steps described in the note at the bottom of the Security Guide topic.