Configure and Start Apache Storm

« Prev

	Note
	The `su` commands in this section use "zookeeper" to represent the ZooKeeper Service user. If you are using another name for your ZooKeeper Service user, you need to substitute your ZooKeeper Service user name for "zookeeper" in each of the `su` commands.

Apache Storm is fairly independent of changes to the HDP cluster:

Deactivate all running topologies.

Delete all states under zookeeper:

/usr/hdp/current/zookeeper-client/bin/zkCli.sh (optionally in secure environment specify -server zk.server:port)

rmr /storm
Delete all states under the storm-local directory:
rm -rf <value of stormlocal.dir>
Stop Storm Services on the storm node.
Update the following configs in storm.yaml:
- storm.thrift.transport=org.apache.storm.security.auth.SimpleTransportPlugin
- storm.messaging.transport=org.apache.storm.messaging.netty.Context
- nimbus.topology=org.apache.storm.nimbus.DefaultTopologyValidator
- topology.spout.wait.strategy=org.apache.storm.spout.SleepSpoutWaitStrategy
- topology.kryo.factory=org.apache.storm.serialization.DefaultKryoFactory
- topology.tuple.serializer=org.apache.storm.serialization.types.ListDelegateSerializer
- nimbus.authorizer=org.apache.storm.security.suth.authorizer.SimpleACLAuthorizer (applicable only in a secure cluster)
- drpc.authorizer=org.apache.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer (applicable only in a secure cluster)
- ui.filter=org.apache.storm.secuity.auth.KerberosPrincipalToLocal (applicable only in a secure cluster)

Stop ZooKeeper Services on the storm node.

su -zookeeper -c "export ZOOCFGDIR=/etc/zookeeper/conf ; export ZOOCFG=zoo.cfg ;source /etc/zookeeper/conf/zookeeper-env.sh ; /usr/lib/zookeeper/bin/zkServer.sh stop"

Remove Storm and zookeeper from the storm node and install the HDP 2.6.0 version:
- For RHEL/CentOS/Oracle Linux:
  yum erase storm
  yum erase zookeeper
  yum install storm
  yum install zookeeper
- For SLES:
  zypper rm storm
  zypper rm zookeeper
  zypper install storm
  zypper install zookeeper
- For Ubuntu/Debian:
  apt-get remove storm --purge
  apt-get remove zookeeper --purge
  apt-get install storm
  apt-get install zookeeper
Replace your configuration after upgrading. Copy /etc/storm/conf from the template to the conf directory .
Replace your ZooKeeper configuration after upgrading. Replace the ZooKeeper template configuration in /etc/zookeeper/conf.

Ensure ZooKeeper is running. On the storm node, run the following command:

su - zookeeper -c "source /etc/zookeeper/conf/zookeeper-env.sh; export ZOOCFGDIR=/etc/zookeeper/conf; /usr/hdp/current/zookeeper-server/bin/zkServer.sh start >> $ZOO_LOG_DIR/zoo.out\"

su - zookeeper -c "source /etc/zookeeper/conf/zookeeper-env.sh; export ZOOCFGDIR=/etc/zookeeper/conf; /usr/hdf/current/zookeeper-server/bin/zkServer.sh start >> $ZOO_LOG_DIR/zoo.out\"

where

$ZOO_LOG_DIR is the directory where ZooKeeper server logs are stored. For example, /var/log/zookeeper.

Start nimbus, then supervisor/ui/drpc/logviewer:
/usr/hdp/current/storm-nimbus/bin/storm nimbus.

Start Storm, using a process controller, such as supervisor:

su - storm /usr/hdp/current/storm-supervisor/bin/storm supervisor

You can use the same command syntax to start Storm using nimbus/ui and logviewer.

su - storm /usr/hdp/current/storm-supervisor/bin/storm nimbus
su - storm /usr/hdp/current/storm-supervisor/bin/storm ui
su - storm /usr/hdp/current/storm-supervisor/bin/storm logviewer
su - storm /usr/hdp/current/storm-supervisor/bin/storm drpc

​Configure and Start Apache Storm