Command Line Installation
Also available as:
PDF
loading table of contents...

Configure Kafka

Use the following procedure to configure Kafka.

  1. By default, Kafka is installed at /usr/hdp/current/kafka-broker.

  2. Verify the values of the following configuration properties in the server.properties file:

Table 22.1. Kafka Configuration Properties

Kafka Configuration Property

Description

log.dirs

Comma-separated list of directories where Kafka log files are stored. The default value is /kafka-logs.

zookeeper.connect

The hostname or IP address of the host running ZooKeeper and the port to which ZooKeeper listens. The default value is localhost:2181.

log.retention.hours

The number of hours to wait before a Kafka log file is eligible for deletion. The default value is 168 hours (7 days).

Listeners

listener - Comma-separated list of URIs on which we listen and their protocols.

Specify hostname as 0.0.0.0 to bind to all interfaces.

Leave hostname empty to bind to default interface.

Examples of legal listener lists:

PLAINTEXT://myhost:9092,SASL_PLAINTEXT://:9091

PLAINTEXT://0.0.0.0:9092, SASL_PLAINTEXT://localhost:9093

File descriptor

Kafka uses a very large number of files and also large number of sockets to communicate with clients. We suggest the following values:

  • The maximum number of open files. Recommended value: 128000

  • The maximum number of processes. Recommended value: 65536