Apache Flume Component Guide
Also available as:
PDF

Flume Feature Updates

Apache Flume version 1.5.2 includes cumulative 1.6 features. The table below indicates the features added to Flume 1.5.2 with each release of Hortonworks Data Platform (HDP).

Table 1.1. Apache Flume Features by HDP Version

HDP ReleaseAdded FeaturesAdvantages
2.5.0Kafka ChannelUses a single Kafka topic. Provides greater reliability and better performance.
 TailDir Source Greater data reliability, even with rotating file names. Can restart tailing at the point where Flume stopped, while continuing data ingest.
2.4.0Kafka SourceReads messages from a Kafka topic. Can have multiple Kafka sources running and configure them to read a unique set of partitions for the topic.
 Kafka SinkPublishes data to a Kafka topic. Supports pull-based processing from various Flume sources.
2.3.0Hive SinkNot recommended for use in production. Streams events containing delimited text or JSON data directly into a Hive table or partition. Provides a preview feature and not.