kafka-log-dirs

Learn how to use the kafka-log-dirs tool.

The kafka-log-dirs tool allows user to query a list of replicas per log directory on a broker. The tool provides information that is required for optimizing replica assignment across brokers.

To retrieve replica assignment information, run the following command:

kafka-log-dirs --describe --bootstrap-server hostname:port --broker-list broker 1, broker 2 --topic-list topic 1, topic 2

If no topic is specified with the --topic-list option, then all topics are queried. If no broker is specified with the --broker-list option, then all brokers are queried. If a log directory is offline, the log directory will be marked offline in the script output. Error example:

"error":"org.apache.kafka.common.errors.KafkaStorageException"

On successful execution, the tool prints a list of partitions per log directory for the specified topics and brokers. The list contains information on topic partition, size, offset lag, and reassignment state. Example output:

{
    "brokers": [
        {
            "broker": 86,
            "logDirs": [
                {
                    "error": null,
                    "logDir": "/var/local/kafka/data",
                    "partitions": [
                        {
                            "isFuture": false,
                            "offsetLag": 0,
                            "partition": "mytopic1-2",
                            "size": 0
                        }
                    ]
                }
            ]
        },
        ...
    ],
    "version": 1
}

The following table gives an overview of the information provided by the output of the kafka-log-dirs tool.

Table 1. Contents of the kafka-log-dirs Output
Property	Description
`broker`	Displays the ID of the broker.
`error`	Indicates if there is a problem with the disk that hosts the topic partition. If an error is detected, `org.apache.kafka.common.errors.KafkaStorageException` is displayed. If no error is detected, the value is null.
`logDir`	Specifies the location of the log directory. Returns an absolute path.
`isfuture`	The reassignment state of the partition. This property shows whether there is currently replica movement underway between the log directories.
`offsetLag`	Displays the offset lag of the partition.
`partition`	Displays the name of the partition.
`size`	Displays the size of the partition in bytes.