Building Cloudera Manager charts with Kafka metrics

A collection query examples to build Kafka metrics charts within Cloudera Manager.

Cloudera Manager enables you to build charts based on Kafka specific metrics. To access the chart builder go to Charts > Chart Builderin Cloudera Manager. The following are a few specific examples of queries for cloudera metrics:

Controllers across all brokers

This chart shows the active controller across all brokers. It is useful for checking active controller status (should be one at any given time, transitions should be fast).

SELECT
    kafka_active_controller
WHERE
    roleType=KAFKA_BROKER

Network idle rate

>Chart showing the network processor idle rate across all brokers. If idle time is always zero, then probably the num.network.threads property may need to be increased.

SELECT
    kafka_network_processor_avg_idle_rate
WHERE
    roleType=KAFKA_BROKER

Partitions per broker

Chart showing the number of partitions per broker. It is useful for detecting partition imbalances early.

SELECT
    kafka_partitions
WHERE
    roleType=KAFKA_BROKER

Partition activity

Chart tracking partition activity on a single broker.

SELECT
    kafka_partitions, kafka_under_replicated_partitions
WHERE
    hostname=host1.domain.com

Mirror Maker activity

Chart for tracking Mirror Maker behavior. Since Mirror Maker has one or more consumers and a single producer, most consumer or metrics should be usable with this query.

SELECT
    producer or consumer metric
WHERE
    roleType=KAFKA_MIRROR_MAKER