Apache Flume Component Guide
Also available as:
PDF

A Simple Example

The following snippet shows some of the kinds of properties that can be set using the properties file. For more detailed information, see the Apache Flume 1.5.2 documentation.

agent.sources = pstream 
agent.channels = memoryChannel
agent.channels.memoryChannel.type = memory 

agent.sources.pstream.channels = memoryChannel 
agent.sources.pstream.type = exec 
agent.sources.pstream.command = tail -f /etc/passwd 

agent.sinks = hdfsSinkagent.sinks.hdfsSink.type = hdfs 
agent.sinks.hdfsSink.channel = memoryChannel
agent.sinks.hdfsSink.hdfs.path = hdfs://hdp/user/root/flumetest 
agent.sinks.hdfsSink.hdfs.fileType = SequenceFile 
agent.sinks.hdfsSink.hdfs.writeFormat = Text

The source here is defined as an exec source. The agent runs a given command on startup, which streams data to stdout, where the source gets it.

In this case, the command is a Python test script. The channel is defined as an in-memory channel and the sink is an HDFS sink.