What's New in Apache Kafka

Rebase on Kafka 2.5.0

Kafka shipped with this version of Cloudera Runtime is based on Apache Kafka 2.5.0. For more information, see Apache Kafka Notable Changes for version 2.5.0, as well as the Apache Kafka Release Notes for version 2.5.0 in the upstream documentation.

Allowed Replica Time Lag default value change

The default value of the Allowed Replica Time Lag (replica.lag.time.max.ms) property is increased from 10 to 30 seconds.

Detailed GC metrics are exported for brokers

Kafka brokers previously only exported an aggregated run count for garbage collection. From now on, more detailed information is exported. An aggregated time metric called kafka_jvm_gc_time is added. In addition, run count and time metrics are added for a handful of specific collectors which are the following:

G1 Young Generation
G1 Old Generation
ParNew
Concurrent Mark Sweep (CMS)
PS MarkSweep
PS Scavenge

The metrics for a particular collector are only exported if that collector is in use. For example, if you are using the G1 garbage collector, only the G1 collector metrics will be exported. Metrics for other collectors will be empty. The collector metrics are all named with the following naming convention kafka_jvm_gc_<collector_name>_<runs/time>. For example kafka_jvm_gc_parnew_runs.

For a comprehensive list of available Kafka Broker metrics, see Kafka Broker Metrics in the Cloudera Manager Reference.

Kafka centric clusters with Ranger authorization

Long term audit logging in Ranger can now be enabled or disabled. If you do not need to store audit logs for more than 90 days, long term audit logging can be disabled. If this feature is disabled, Ranger does not require HDFS as a dependency. As a result, you can install and deploy a Kafka centric cluster, which has Kafka and Ranger but not HDFS. For more information, see Installing a Kafka centric cluster.