Command Line Upgrade
Also available as:
PDF
loading table of contents...

Create Local Directories

You must create local directories for YARN on each NodeManager host in your cluster (in HDP-2, the NodeManager replaces the TaskTracker) and set the appropriate permissions for the YARN log directories.

  1. Set the permissions in the yarn.nodemanager.local-dirs directories. Run these commands on all DataNodes in your cluster.

    chown -R yarn:hadoop ${yarn.nodemanager.local-dirs}

    chmod 755 ${yarn.nodemanager.local-dirs}

    where ${yarn.nodemanager.local-dirs} is your local directory.

  2. Change the permissions of the directories in yarn.nodemanager.log-dirs. If these directories do not exist, you can create them using the instructions in the Create Directories section of the Non-Ambari Cluster Installation Guide. Run these commands on all DataNodes in your cluster.

    chown -R yarn:hadoop ${yarn.nodemanager.log-dirs}

    chmod 755 ${yarn.nodemanager.log-dirs}

    where ${yarn.nodemanager.log-dirs} is your log directory.

  3. Create directories for YARN_LOG_DIR and YARN_PID_DIR.

    • Open /etc/hadoop/conf/yarn-env.sh

    • Write down your values for YARN_LOG_DIR and YARN_PID_DIR, as the following instructions require values for the ${YARN_LOG_DIR} and ${YARN_PID_DIR}.

    For example, in yarn-env.sh:

    YARN_LOG_DIR=/grid/0/var/log/hadoop/yarn

    YARN_PID_DIR=/grid/0/var/run/hadoop/yarn

  4. Make directories for ${YARN_LOG_DIR} and ${YARN_PID_DIR} and set the appropriate permissions for them.

    mkdir ${YARN_LOG_DIR}

    chown yarn:hadoop ${YARN_LOG_DIR}

    chmod 755 ${YARN_LOG_DIR}

    mkdir ${YARN_PID_DIR}

    chown yarn:hadoop ${YARN_PID_DIR}

    chmod 755 ${YARN_PID_DIR}