Command Line Upgrade
Also available as:
PDF
loading table of contents...

Configure and Start Apache Storm

[Note]Note

The su commands in this section use "zookeeper" to represent the ZooKeeper Service user. If you are using another name for your ZooKeeper Service user, you need to substitute your ZooKeeper Service user name for "zookeeper" in each of the su commands.

Apache Storm is fairly independent of changes to the HDP cluster, but you must upgrade Storm for rolling upgrade support in HDP 2.5.3 and be on the latest version of Storm.

  1. Deactivate all running topologies.

  2. Delete all states under zookeeper:

    /usr/hdp/current/zookeeper-client/bin/zkCli.sh (optionally in secure environment specify -server zk.server:port)

    rmr /storm

  3. Delete all states under the storm-local directory:

    rm -rf <value of storm.local.dir>/*

  4. Stop Storm Services on the storm node.

  5. Stop ZooKeeper Services on the storm node.

    su - zookeeper -c "export ZOOCFGDIR=/etc/zookeeper/conf ; export ZOOCFG=zoo.cfg ;source /etc/zookeeper/conf/zookeeper-env.sh ; /usr/lib/zookeeper/bin/zkServer.sh stop"

  6. Remove Storm and zookeeper from the storm node and install the HDP 2.5.3 version:

    • For RHEL/CentOS/Oracle Linux:

      yum erase storm

      yum erase zookeeper

      yum install storm

      yum install zookeeper

    • For SLES:

      zypper rm storm

      zypper rm zookeeper

      zypper install storm

      zypper install zookeeper

    • For Ubuntu/Debian:

      apt-get remove storm --purge

      apt-get remove zookeeper --purge

      apt-get install storm

      apt-get install zookeeper

  7. After upgrading Storm, replace your configuration in /etc/storm/conf/storm.yaml change references backtype.storm to org.apache.storm . This is needed because namespace migration in apache storm.

  8. Replace your ZooKeeper configuration after upgrading. Replace the ZooKeeper template configuration in /etc/zookeeper/conf.

  9. Ensure ZooKeeper is running. On the storm node, run the following command:

    su - zookeeper -c "source /etc/zookeeper/conf/zookeeper-env.sh; export ZOOCFGDIR=/etc/zookeeper/conf; /usr/hdp/current/zookeeper-server/bin/zkServer.sh start >> $ZOO_LOG_DIR/zoo.out\"

    where

    • $ZOO_LOG_DIR is the directory where ZooKeeper server logs are stored. For example, /var/log/zookeeper.

  10. Start Storm processes, using a process controller (like supervisord) or manually. Start nimbus, then supervisor/ui/drpc/logviewer:

    su - storm /usr/hdp/current/storm-nimbus/bin/storm nimbus

    You can use the same command syntax to start supervisor, ui, logviewer and drpc.

    su - storm /usr/hdp/current/storm-supervisor/bin/storm supervisor

    su - storm /usr/hdp/current/storm-supervisor/bin/storm ui

    su - storm /usr/hdp/current/storm-supervisor/bin/storm logviewer

    su - storm /usr/hdp/current/storm-supervisor/bin/storm drpc