Non-Ambari Cluster Installation Guide
Also available as:
PDF
loading table of contents...

Configure Kafka

Use the following procedure to configure Kafka.

  1. Verify the values of the following configuration properties in the server.properties file:

Table 20.1. Kafka Configuration Properties

Kafka Configuration Property

Description

log.dirs

Comma-separated list of directories where Kafka log files are stored. The default value is /kafka-logs.

zookeeper.connect

The hostname or IP address of the host running ZooKeeper and the port to which ZooKeeper listens. The default value is localhost:2181.

log.retention.hours

The number of hours to wait before a Kafka log file is eligible for deletion. The default value is 168 hours (7 days).

Listeners

listener - Comma-separated list of URIs we will listen on and their protocols.

Specify hostname as 0.0.0.0 to bind to all interfaces.

Leave hostname empty to bind to default interface.

Examples of legal listener lists:

PLAINTEXT://myhost:9092,SASL_PLAINTEXT://:9091

PLAINTEXT://0.0.0.0:9092, SASL_PLAINTEXT://localhost:9093

File descriptors

Kafka uses a very large number of files and also large number of sockets to communicate with clients. We suggest the following values:

  • The maximum number of open files. Recommended value: 128000

  • The maximum number of processes. Recommended value: 65536