Metrics

Learn about Kafka metrics and how to view them.

Kafka uses Yammer metrics to record internal performance measurements. The metrics are exposed via Java Management Extensions (JMX) and can be read with a JMX console.

Metrics Categories

There are metrics available in the various components of Kafka. In addition, there are some metrics specific to how Cloudera Manager and Kafka interact. This table has pointers to both the Apache Kafka metrics names and the Cloudera Manager metric names.

Table 1. Metrics by Category
Category	Cloudera Manager Metrics Doc	Apache Kafka Metrics Doc
Cloudera Manager Kafka Service	Base Metrics
Broker	Broker Metrics, Broker Topic Metrics, Replica Metrics, Broker Topic Partition Metrics	Broker
Common Producer/Consumer		Client Client-to-Broker
Producer	Producer Metrics	Producer Producer Sender
Consumer	Consumer Group Metrics	Consumer Group Consumer Fetch
Mirror Maker	Mirror Maker Metrics	Same as Producer or Consumer tables

Note the following about Cloudera Manager metrics:

Broker Topic Partitions Metrics:: Required for Streams Messaging Manager. These metrics have to be specifically enabled within the host's configuration.

Producer Metrics:: Required for Streams Messaging Manager. These metrics have to be specifically enabled within Kafka's configuration and are different from the metrics available on client instances.

Consumer Groups Metrics: Generated by and required for Streams Messaging Manager. These metrics are different from the metrics available on client instances.

Viewing Metrics

Cloudera Manager records most of these metrics and makes them available via Chart Builder.

Because Cloudera Manager cannot track metrics on any clients (that is, producer or consumer), you may wish to use an alternative JMX console program to check metrics. There are several JMX console options, for example:

JConsole, which comes bundled with the JDK.
VisualVM, with the MBeans plugin.