Configure, Start, and Validate Apache Flume
Before you can upgrade Apache Flume, you must have first upgraded your HDP components to the latest version (in this case, 2.4.0). This section assumes that you have already upgraded your components for HDP 2.4.0. If you have not already completed these steps, return to Getting Ready to Upgrade and Upgrade 1.3 Components for instructions on how to upgrade your HDP components to 2.4.0.
Upgrade Flume. On the Flume host machine, run the following command:
For RHEL/CentOS/Oracle Linux:
yum upgrade flume
For SLES:
zypper update flume
zypper remove flume
zypper se -s flume
You should see Flume in the output.
Install Flume:
zypper install flume
For Ubuntu/Debian:
HDP support for Debian 6 is deprecated with HDP 2.4.0. Future versions of HDP will no longer be supported on Debian 6.
apt-get install flume
Replace your configuration after upgrading. Copy /etc/flume/conf from the template to the conf directory in Flume hosts.
By default on installation Flume does not start running immediately. To validate, replace your default conf/flume.conf with the provided flume.conf file, and restart Flume. See if the data is flowing by examining the destination.
Use this flume.conf file:
1. Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 2. Describe/configure the source a1.sources.r1.type = seq 3. Describe the sink a1.sinks.k1.type = file_roll a1.sinks.k1.channel = c1 a1.sinks.k1.sink.directory = /tmp/flume 4. Use a channel which buffers events in memory a1.channels.c1.type = memory 5. Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1
After starting Flume, check /tmp/flume to see if there are any files there. The files should contain simple sequential numbers.
After validating, stop Flume and revert changes to flume.conf to prevent your disk from filling up.