Extracting KRaft metadata

Learn how to extract Kafka metadata from the __cluster_metadata topic. Metadata extracted from this topic can be used for debugging and troubleshooting issues with a Kafka deployment running in KRaft mode.

When Kafka is running in KRaft mode, metadata describing the state of the Kafka cluster is stored in the __cluster_metadata topic. This topic can be found in the /var/local/kraft/data directory on each KRaft Controller service role host.

In case you encounter any issues when running your deployment in KRaft mode, generally the first step is to print the contents of the __cluster_metadata topic. Reviewing the contents of the topic can help in identifying the issues with the cluster.

The contents of the __cluster_metadata topic can be printed using the kafka-dump-log command with the --cluster-metadata-decoder option.

  1. Log in to one of your cluster hosts that has a KRaft service role (KRaft Controller) deployed on it.
  2. Run the kafka-dump-log command with the --cluster-metadata-decoder option. For example:
    kafka-dump-log --cluster-metadata-decoder --files /var/local/kraft/data/__cluster_metadata-0/00000000000000000000.log