2.2. Deploy the ResourceManager HA Cluster

  1. Copy the etc/hadoop/conf/yarn-site.xml file from the primary ResourceManager host to the standby ResourceManager host.

  2. Make sure that the clientPort value set in etc/zookeeper/conf/zoo.cfg matches the port set in the following yarn-site.xml property:

    <property>
            <name>yarn.resourcemanager.zk-state-store.address</name>
            <value>localhost:2181</value>
        </property>

  3. Start Zookeeper. Execute this command on the ZooKeeper host machine(s).

    su - zookeeper -c "export  ZOOCFGDIR=/etc/zookeeper/conf ; export ZOOCFG=zoo.cfg ; source /etc/zookeeper/conf/zookeeper-env.sh ; /usr/lib/zookeeper/bin/zkServer.sh start"
  4. Start HDFS

    1. Execute this command on the NameNode host machine:

      su -l hdfs -c "/usr/lib/hadoop/sbin/hadoop-daemon.sh --config /etc/hadoop/conf start namenode" 
    2. Execute this command on the Secondary NameNode host machine:

      su -l hdfs -c "/usr/lib/hadoop/sbin/hadoop-daemon.sh --config /etc/hadoop/conf start secondarynamenode” 
    3. Execute this command on all DataNodes:

      su -l hdfs -c "/usr/lib/hadoop/sbin/hadoop-daemon.sh --config /etc/hadoop/conf start datanode"
  5. Start YARN

    1. Execute this command to start the primary ResourceManager:

      su - yarn -c "export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf start resourcemanager"
    2. Execute this command to start the standby ResourceManager:

      su - yarn -c "export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf2 start resourcemanager"
    3. Execute this command on the History Server host machine:

      su - mapred -c "export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-mapreduce/sbin/mr-jobhistory-daemon.sh --config /etc/hadoop/conf start historyserver"
    4. Execute this command on all NodeManagers:

      su - yarn -c "export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh --config /etc/hadoop/conf start nodemanager"

      At this point, you should be able to see messages in the console that the NodeManager is trying to connect to the active ResourceManager.

  6. Set the active ResourceManager:

    MANUAL FAILOVER ONLY: If you configured manual ResourceManager failover, you must transition one of the ResourceManagers to Active mode. Execute the following CLI command to transition ResourceManager "rm1" to Active:

    yarn rmadmin -transitionToActive rm1

    You can use the following CLI command to transition ResourceManager "rm1" to Standby mode:

    yarn rmadmin -transitionToStandby rm1

    AUTOMATIC FAILOVER: If you configured automatic ResourceManager failover, no action is required -- the Active ResourceManager will be chosen automatically.

  7. Start all remaining unstarted cluster services using the instructions provided here.


loading table of contents...