Flume Feature Updates
Apache Flume version 1.5.2 includes cumulative 1.6 features. The table below indicates the features added to Flume 1.5.2 with each release of Hortonworks Data Platform (HDP).
Table 1.1. Apache Flume Features by HDP Version
| HDP Release | Added Features | Advantages |
|---|---|---|
| 2.5.0 | Kafka Channel | Uses a single Kafka topic. Provides greater reliability and better performance. |
| TailDir Source | Greater data reliability, even with rotating file names. Can restart tailing at the point where Flume stopped, while continuing data ingest. | |
| 2.4.0 | Kafka Source | Reads messages from a Kafka topic. Can have multiple Kafka sources running and configure them to read a unique set of partitions for the topic. |
| Kafka Sink | Publishes data to a Kafka topic. Supports pull-based processing from various Flume sources. | |
| 2.3.0 | Hive Sink | Not recommended for use in production. Streams events containing delimited text or JSON data directly into a Hive table or partition. Provides a preview feature and not. |