Broker log management

Learn more about the configuration properties related to broker log management.

Kafka brokers save their data as log segments in a directory. The logs are rotated depending on the size and time settings.

The most common log retention settings to adjust for your cluster are shown below. These are accessible in Cloudera Manager via the Kafka > Configuration tab.

log.dirs: The location for the Kafka data (that is, topic directories and log segments).
log.retention.{ms|minutes|hours}: The retention period for the entire log. Any older log segments are removed.
log.retention.bytes: The retention size for the entire log.

In addition, there are many more configuration properties available for fine-tuning broker log management. For more information look for the following properties in theBroker Configs section in the upstream Apache Kafka documentation.

log.dirs
log.flush.*
log.retention.*
log.roll.*
log.segment.*