Kafka Metrics

There are metrics available in the various components of Kafka. In addition, there are some metrics specific to how Cloudera Manager and Kafka interact. This table has pointers to both the Apache Kafka metrics names and the Cloudera Manager metric names.

Cloudera Manager records most of these metrics and makes them available via Chart Builder.

Because Cloudera Manager cannot track metrics on any clients (that is, producer or consumer), you may wish to use an alternative JMX console program to check metrics. There are several JMX console options:

• The JDK comes with the jconsole utility.
• VisualVM has a MBeans plugin.

The Charts edit menu looks like a small pencil icon in the Charts page of the Cloudera Manager console. From there, choose Add from Chart Builder and enter a query for the appropriate metric.

SELECT
    metric
WHERE
    filter

Some specific examples of queries for Cloudera Metrics are:

Controllers across all brokers

This chart shows the active controller across all brokers. It is useful for checking active controller status (should be one at any given time, transitions should be fast).

SELECT
    kafka_active_controller
WHERE
    roleType=KAFKA_BROKER

Network idle rate

>Chart showing the network processor idle rate across all brokers. If idle time is always zero, then probably the num.network.threads property may need to be increased.

SELECT
    kafka_network_processor_avg_idle_rate
WHERE
    roleType=KAFKA_BROKER

Partitions per broker

Chart showing the number of partitions per broker. It is useful for detecting partition imbalances early.

SELECT
    kafka_partitions
WHERE
    roleType=KAFKA_BROKER

Partition activity

Chart tracking partition activity on a single broker.

SELECT
    kafka_partitions, kafka_under_replicated_partitions
WHERE
    hostname=host1.domain.com

Mirror Maker activity

Chart for tracking Mirror Maker behavior. Since Mirror Maker has one or more consumers and a single producer, most consumer or metrics should be usable with this query.

SELECT
    producer or consumer metric
WHERE
    roleType=KAFKA_MIRROR_MAKER