CDH 6 includes Apache Kafka as part of the core package. The documentation includes improved contents for how to set up, install, and administer your Kafka ecosystem. For more information, see the Cloudera Enterprise 6.0.x Apache Kafka Guide. We look forward to your feedback on both the existing and new documentation.

Working with Apache Kafka Logs

By default, the CDK Powered By Apache Kafka parcel is configured to log all Kafka log messages to one file per host, or broker, in the following location:

${log.dir}/kafka-broker-<your_hostname>.log

The default log directory is /var/log/kafka. You can view, filter, and search the logs using Cloudera Manager. See Logs for more information about viewing logs in Cloudera Manager.

You can view, filter, and search this log using Cloudera Manager. See Logs.

For debugging purposes, you can create a separate file with TRACE level logs of a specific component (such as the controller) or the state changes.

For example, to restore the default Apache Kafka log4j configuration, do the following:

  1. Navigate to the Kafka Broker Configuration page.
  2. Search for the Kafka Broker Logging Advanced Configuration Snippet (Safety Valve) field.
  3. Copy and paste the configuration snippet from the Apache Kafka log4j.properties file.

Alternatively, you can add only the appenders you need.

For more information on setting properties, see Modifying Configuration Properties Using Cloudera Manager.