7. Create Local Directories

You must create local directories for YARN on each NodeManager host in your cluster (in HDP-2, the NodeManager replaces the TaskTracker) and set the appropriate permissions for the YARN log directories.

  1. Set the permissions in the yarn.nodemanager.local-dirs directories. Run these commands on all DataNodes in your cluster.

    chown -R yarn:hadoop ${yarn.nodemanager.local-dirs}

    chmod 755 ${yarn.nodemanager.local-dirs}

    where ${yarn.nodemanager.local-dirs} is your local directory.

  2. Change the permissions of the directories in yarn.nodemanager.log-dirs. If these directories do not exist, you can create them using the instructions in the Create Directories section of the HDP Manual Install Guide. Run these commands on all DataNodes in your cluster.

    chown -R yarn:hadoop ${yarn.nodemanager.log-dirs}

    chmod 755 ${yarn.nodemanager.log-dirs}

    where ${yarn.nodemanager.log-dirs} is your log directory.

  3. Create directories for YARN_LOG_DIR and YARN_PID_DIR.

    • Open /etc/hadoop/conf/yarn-env.sh

    • Write down your values for YARN_LOG_DIR and YARN_PID_DIR, as the following instructions require values for the ${YARN_LOG_DIR} and ${YARN_PID_DIR}.

    For example, in yarn-env.sh:

    YARN_LOG_DIR=/grid/0/var/log/hadoop/yarn

    YARN_PID_DIR=/grid/0/var/run/hadoop/yarn

  4. Make directories for ${YARN_LOG_DIR} and ${YARN_PID_DIR} and set the appropriate permissions for them.

    mkdir ${YARN_LOG_DIR}

    chown yarn:hadoop ${YARN_LOG_DIR}

    chmod 755 ${YARN_LOG_DIR}

    mkdir ${YARN_PID_DIR}

    chown yarn:hadoop ${YARN_PID_DIR}

    chmod 755 ${YARN_PID_DIR}


loading table of contents...