Non-Ambari Cluster Installation Guide
Also available as:
PDF
loading table of contents...

Install Kafka

Prerequisites and Considerations

When installing Kafka, note the following prerequisites and considerations:

  • Administrators must use Apache ZooKeeper to manage Kafka for an HDP cluster. Verify that you have successfully installed ZooKeeper before installing and configuring Kafka.

  • Kafka does not currently support user authentication and authorization.

  • The following underlying file systems are supported for use with Kafka:

    • EXT4: recommended

    • EXT3: supported

    [Caution]Caution

    Encrypted file systems such as SafenetFS are not supported for Kafka. Index file corruption can occur.

 

Installation

Install the Kafka RPMs or packages by completing the following steps.

[Note]Note

Hortonworks recommends avoiding using multiple brokers in a single node for Kafka. However, if you need to install a multi-node cluster, use the following instructions to install Kafka on another machine, make sure each broker.id is unique on the cluster, and ensure that zookeeper.connect is the same for all brokers.

  1. Run the following command on each client cluster node and gateway node:

    • For RHEL/CentOS/Oracle Linux

      yum install kafka

    • For SLES

      zypper install kafka

    • For Ubuntu and Debian

      apt-get install kafka

  2. Check the JAVA_HOME environment variable. If it has not yet been set, add the following to the PATH variable:

    export JAVA_HOME=<path of installed jdk version folder>