New Features in Cloudera Distribution of Kafka 1.2.0

The new Kafka producer added in Cloudera Distribution of Kafka 1.1.0 combines features of the existing synchronous and asynchronous producers. Send requests are batched, allowing the new producer to perform as well as the asynchronous producer under load. Every send request returns a response object that can be used to retrieve status and exceptions.

You can now delete topics using the kafka-topics --delete command.

In previous versions, consumers that wanted to keep track of which messages were consumed did so by updating the offset of the last consumed message in Zookeeper. With this new feature, Kafka itself tracks the offsets. Using offset management can significantly improve consumer performance.

Each partition starts with a randomly selected leader replica that handles requests for that partition. When a cluster first starts, the leaders are evenly balanced among nodes. When a broker restarts, leaders from that broker are distributed to other brokers, which results in an unbalanced distribution. With this feature enabled, leaders are assigned to the original replica after a restart.

Kafka administrators can limit the number of connections allowed from a single IP address. By default, this limit is 10 connections per IP address. This prevents misconfigured or malicious clients from destabilizing a Kafka broker by opening a large number of connections and using all available file handles.