Learn about Kafka metrics and how to view them.
Kafka uses Yammer metrics to record internal performance measurements. The metrics are exposed via Java Management Extensions (JMX) and can be read with a JMX console.
There are metrics available in the various components of Kafka. In addition, there are some metrics specific to how Cloudera Manager and Kafka interact. This table has pointers to both the Apache Kafka metrics names and the Cloudera Manager metric names.
|Category||Cloudera Manager Metrics Doc||Apache Kafka Metrics Doc|
|Broker||Broker Metrics, Broker Topic Metrics, Replica Metrics, Broker Topic Partition Metrics||Broker|
|Consumer||Consumer Metrics,Consumer Group Metrics|
|Mirror Maker||Mirror Maker Metrics||Same as Producer or Consumer tables|
- Broker Topic Partitions Metrics:
- Required for Streams Messaging Manager. These metrics have to be specifically enabled within the host's configuration.
- Producer Metrics:
- Required for Streams Messaging Manager. These metrics have to be specifically enabled within Kafka's configuration and are different from the metrics available on client instances.
- Consumer and Consumer Groups Metrics
- Generated by and required for Streams Messaging Manager. These metrics are different from the metrics available on client instances.
Cloudera Manager records most of these metrics and makes them available via Chart Builder.
Because Cloudera Manager cannot track metrics on any clients (that is, producer or consumer), you may wish to use an alternative JMX console program to check metrics. There are several JMX console options, for example:
- JConsole, which comes bundled with the JDK.
- VisualVM, with the MBeans plugin.