Configuring logging for Kafka cluster components

The logging properties of Kafka cluster components like Kafka brokers, ZooKeeper, Cruise Control, and all other components deployed and managed through the Kafka resource are configured in the Kafka resource.

Logging properties are specified in spec.[***COMPONENT***].logging. Logging properties can be added directly to this property, or can be defined in an external ConfigMap that is referenced in the Kafka using configMapKeyRef property.

You choose the configuration method by setting the logging.type property to either inline or external.

Inline

Inline configuration means that you directly specify the logging properties in the Kafka resource at the spec of each component.

#...
kind: Kafka
spec:
  #...
    logging:
      type: inline
      loggers:
        kafka.root.logger.level: INFO

External

External configuration means that you reference your own ConfigMap that holds the logging properties.

#...
kind: Kafka
spec:
  #...
    logging:
      type: external
      valueFrom:
        configMapKeyRef:
          name: my-config-map
          key: my-config-map-key

A ConfigMap is generated for each Kafka cluster component after pod creation. These ConfigMaps contain the actual logging configuration. Do not edit the generated ConfigMaps directly, as direct changes are ignored.

Depending on the changes made, they are either applied dynamically, or a rolling restart is triggered.

The following Kafka cluster components use log4j configuration:

• Kafka
• ZooKeeper

The following Kafka cluster components use log4j2 configuration:

• CruiseControl
• UserOperator
• EntityOperator