What's New in Apache Kafka

Learn about the new features of Kafka in Cloudera Runtime 7.2.2.

Detailed GC metrics are exported for brokers

Kafka brokers previously only exported an aggregated run count for garbage collection. From now on, more detailed information is exported. An aggregated time metric called kafka_jvm_gc_time is added. In addition, run count and time metrics are added for a handful of specific collectors which are the following:

G1 Young Generation
G1 Old Generation
ParNew
Concurrent Mark Sweep (CMS)
PS MarkSweep
PS Scavenge

The metrics for a particular collector are only exported if that collector is in use. For example, if you are using the G1 garbage collector, only the G1 collector metrics will be exported. Metrics for other collectors will be empty. The collector metrics are all named with the following naming convention kafka_jvm_gc_<collector_name>_<runs/time>, for example kafka_jvm_gc_parnew_runs.

For a comprehensive list of available Kafka Broker metrics, see Kafka Broker Metrics in the Cloudera Manager Reference.

The kafka-clients library is no longer dependant on Apache Shiro and libpam4j

The LdapPlainServerCallbackHandler and PamPlainServerCallbackHandler custom Cloudera plugins introduced undesired dependencies for users of the kafka-client library. These plugins are now moved to a new module in Kafka that is only used on the broker side. As a result, the kafka-clients library is no longer dependent on Apache Shiro and libpam4j.