Configure and Start Apache Storm
Note | |
---|---|
The |
Apache Storm is fairly independent of changes to the HDP cluster, but you must upgrade Storm for rolling upgrade support in and be on the latest version of Storm.
Deactivate all running topologies.
Delete all states under zookeeper:
/usr/hdp/current/zookeeper-client/bin/zkCli.sh (optionally in secure environment specify -server zk.server:port)
rmr /storm
Delete all states under the storm-local directory:
rm -rf <value of storm.local.dir>/*
Stop Storm Services on the storm node.
Stop ZooKeeper Services on the storm node.
su - zookeeper -c "export ZOOCFGDIR=/etc/zookeeper/conf ; export ZOOCFG=zoo.cfg ;source /etc/zookeeper/conf/zookeeper-env.sh ; /usr/lib/zookeeper/bin/zkServer.sh stop"
Remove Storm and zookeeper from the storm node and install the version:
For RHEL/CentOS/Oracle Linux:
yum erase storm
yum erase zookeeper
yum install storm
yum install zookeeper
For SLES:
zypper rm storm
zypper rm zookeeper
zypper install storm
zypper install zookeeper
For Ubuntu/Debian:
apt-get remove storm --purge
apt-get remove zookeeper --purge
apt-get install storm
apt-get install zookeeper
After upgrading Storm, replace your configuration in
/etc/storm/conf/storm.yaml
change referencesbacktype.storm
toorg.apache.storm
. This is needed because namespace migration in apache storm.Replace your ZooKeeper configuration after upgrading. Replace the ZooKeeper template configuration in
/etc/zookeeper/conf
.Ensure ZooKeeper is running. On the storm node, run the following command:
su - zookeeper -c "source /etc/zookeeper/conf/zookeeper-env.sh; export ZOOCFGDIR=/etc/zookeeper/conf; /usr/hdp/current/zookeeper-server/bin/zkServer.sh start >> $ZOO_LOG_DIR/zoo.out\"
where
$ZOO_LOG_DIR is the directory where ZooKeeper server logs are stored. For example, /var/log/zookeeper.
Start Storm processes, using a process controller (like supervisord) or manually. Start nimbus, then supervisor/ui/drpc/logviewer:
su - storm /usr/hdp/current/storm-nimbus/bin/storm nimbus
You can use the same command syntax to start supervisor, ui, logviewer and drpc.
su - storm /usr/hdp/current/storm-supervisor/bin/storm supervisor
su - storm /usr/hdp/current/storm-supervisor/bin/storm ui
su - storm /usr/hdp/current/storm-supervisor/bin/storm logviewer
su - storm /usr/hdp/current/storm-supervisor/bin/storm drpc