Rolling Upgrade Guide
Also available as:
PDF
loading table of contents...

Upgrade ZooKeeper

In this step you will upgrade and validate each ZooKeeper process, node by node.

  1. Switch to the zookeeper_admin user (default = zookeeper) and run the following stat command for each node in the ZooKeeper quorum until you find the leader:

    su - zookeeper

    echo stat | nc <servername> <port> | grep Mode

    exit

    For example, to check nodes hdp1 and hdp2, assuming they use default ZooKeeper server port 2181:

    [zookeeper@hdp1 ~] echo stat | nc hdp1 2181 | grep Mode
    Mode: follower
    [zookeeper@hdp1 ~] echo stat | nc hdp2 2181 | grep Mode
    Mode: leader
    [zookeeper@hdp1 ~] exit 
    [Note]Note

    This is an optimization step. If the leader changes later, it won’t impact the outcome of the upgrade process.

  2. Upgrade ZooKeeper on each node. Start with follower nodes. Upgrade the leader node last.

    On each follower node, stop, switch, and start the daemon. Validate that the quorum has been reestablished before continuing with the next node:

    1. Stop the ZooKeeper server, switch to the new version, and start the server:

      su - zookeeper -c "source /usr/hdp/current/zookeeper-server/conf/zookeeper-env.sh ; env ZOOCFGDIR=/usr/hdp/current/zookeeper-server/conf ZOOCFG=zoo.cfg /usr/hdp/current/zookeeper-server/bin/zkServer.sh stop"

      hdp-select set zookeeper-server 2.2.9.0-$BUILD

      su - zookeeper -c "source /usr/hdp/current/zookeeper-server/conf/zookeeper-env.sh ; env ZOOCFGDIR=/usr/hdp/current/zookeeper-server/conf ZOOCFG=zoo.cfg /usr/hdp/current/zookeeper-server/bin/zkServer.sh start"

    2. Validate the upgrade: check that the ZooKeeper quorum has been reestablished before upgrading the next node. Create a new znode on the node just upgraded, and then list the created znode.

      For example, start a zkCli process (ZooKeeper shell):

      /usr/hdp/current/zookeeper-server/bin/zkCli.sh

      Create a znode called znode-1. Associate the string "my_data" with the node:

      [zk: localhost:2181(CONNECTED) 20] create /znode-1 my_data
      Created /znode-1

      List the node-1 test directory:

      [zk: localhost:2181(CONNECTED) 21] ls /znode-1
      []

      Remove the test node and exit zkcli:

      [zk: localhost:2181(CONNECTED) 22] delete /znode-1
      [zk: localhost:2181(CONNECTED) 21] quit
      Quitting...
      [hdp@hdp1 /]#
    3. Repeat the validation step until successful, or until you have completed ten attempts. If the test does not succeed after ten tries, see if the ZooKeeper process is running. If not, check ZooKeeper logs for an indication of the problem.

    After upgrading ZooKeeper successfully on each follower node, perform the preceding upgrade steps for the leader node.

If the upgrade is unsuccessful or validations fail, follow the ZooKeeper downgrade steps in Downgrading the Cluster.