This is the documentation for CDH 5.0.x. Documentation for other versions is available at Cloudera Documentation.

Upgrading Flume

Use the instructions that follow to upgrade Flume.
  Note: Running Services

When starting, stopping and restarting CDH components, always use the service (8) command rather than running scripts in /etc/init.d directly. This is important because service sets the current working directory to / and removes most environment variables (passing only LANG and TERM) so as to create a predictable environment in which to administer the service. If you run the scripts in /etc/init.d, any environment variables you have set remain in force, and could produce unpredictable results. (If you install CDH from packages, service will be installed as part of the Linux Standard Base (LSB).)

Upgrading Flume from CDH 4 to CDH 5

Use one of the following sets of instructions to upgrade Flume from CDH 4 to CDH 5, depending on which version of Flume you have been running under CDH 4:

Upgrading Flume 1.x to CDH 5

Use the instructions that follow to upgrade Flume 1.x from CDH 4 to CDH 5. You must remove the CDH 4 version and then install the CDH 5 version.

Step 1: Remove Flume 1.x from your cluster.

  1. Stop the Flume Node processes on each node where they are running: 
    $ sudo service flume-ng-agent stop
  2. Uninstall the old Flume components:

    On Red Hat-compatible systems: 

    $ sudo yum remove flume-ng agent flume-ng

    On SLES systems: 

    $ sudo zypper remove flume-ng agent flume-ng

    On Ubuntu systems: 

    $ sudo apt-get remove flume-ng agent flume-ng

Step 2. Install the new version of Flume

Follow the instructions in the rest of this document to install Flume 1.x under CDH 5.

Migrating from Flume 0.9.x under CDH 4 to Flume 1.x under CDH 5

Flume 1.x is a significant departure from Flume 0.9.x in its implementation although many of the original concepts are the same. If you're already familiar with Flume, here are some significant points.

  • You still have sources and sinks and they still do the same thing. They are now connected by channels.
  • Channels are pluggable, and dictate durability. Flume 1.x ships with an in-memory channel for fast, but non-durable event delivery. There are also JDBC-based and file-based channels for durable event delivery.
  • There are no more logical or physical nodes. All physical nodes are "agents," and agents can run zero or more sources and sinks.
  • There is no longer a Master, and no ZooKeeper dependency. At this time, Flume runs with a simple file-based configuration system.
  • Just about everything is a plugin — some end user facing, some for tool and system developers.
  • Thrift and Avro legacy Flume sources are provided to enable sending events from Flume 0.9.4 to Flume 1.x.

You must uninstall Flume 0.9.x and then install Flume 1.x, as follows.

Step 1: Remove Flume 0.9.x from your cluster.

  1. Stop the Flume Node processes on each node where they are running: 
    $ sudo service flume-node stop
  2. Stop the Flume Master: 
    $ sudo service flume-master stop
  3. Uninstall the old Flume components:

    On Red Hat-compatible systems: 

    $ sudo yum remove flume

    On SLES systems: 

    $ sudo zypper remove flume

    On Ubuntu systems: 

    $ sudo apt-get remove flume

Step 2. Install the new version of Flume

Follow the instructions in the rest of this document to install Flume 1.x from a tarball or packages.

Upgrading Flume from an Earlier CDH 5 release

These instructions assume that you are upgrading Sqoop as part of an upgrade to the latest CDH 5 release, and have already performed the steps under Upgrading from a CDH 5 Beta Release to the Latest Release .

To upgrade Sqoop from an earlier CDH 5 release, install the new version of Flume using one of the methods described below: Installing the Flume RPM or Debian Packages or Installing the Flume Tarball .

  Important: Configuration files
  • If you install a newer version of a package that is already on the system, configuration files that you have modified will remain intact.
  • If you uninstall a package, the package manager renames any configuration files you have modified from <file> to <file>.rpmsave. If you then re-install the package (probably to install a new version) the package manager creates a new <file> with applicable defaults. You are responsible for applying any changes captured in the original configuration file to the new configuration file. In the case of Ubuntu and Debian upgrades, you will be prompted if you have made changes to a file for which there is a new version; for details, see Automatic handling of configuration files by dpkg.
Page generated September 3, 2015.