4. Create Local Directories

You must create local directories for YARN on every note of NodeManagers (TaskTrackers) and set the appropriate permissions for your YARN log directories.

  1. Set the permissions on the yarn.nodemanager.local-dirs directories. Run these commands on all DataNodes in your cluster.

    chown -R yarn:hadoop ${yarn.nodemanager.local-dirs}
    chmod 755 ${yarn.nodemanager.local-dirs}

    where ${yarn.nodemanager.local-dirs} is your local directory.

  2. Change permissions of directories of yarn.nodemanager.log-dirs. Run these commands on all DataNodes in your cluster.

    chown -R yarn:hadoop ${yarn.nodemanager.log-dirs}
    chmod 755 ${yarn.nodemanager.log-dirs}

    where ${yarn.nodemanager.log-dirs} is your log directory.

  3. Create directories for YARN_LOG_DIR and YARN_PID_DIR.

    1. Open /etc/hadoop/conf/yarn-env.sh

    2. Write down your values for YARN_LOG_DIR and YARN_PID_DIR as the following instructions require values for the ${YARN_LOG_DIR} and ${YARN_PID_DIR}. For example in yarn-env.sh:

      ${YARN_LOG_DIR}=/grid/0/var/log/hadoop/yarn
       ${YARN_PID_DIR}=/grid/0/var/run/hadoop/yarn

  4. Make directories for ${YARN_LOG_DIR} and ${YARN_PID_DIR} and set the appropriate permissions for them.

    mkdir ${YARN_LOG_DIR}
    chown yarn:hadoop ${YARN_LOG_DIR}
    chown yarn:hadoop ${YARN_LOG_DIR}
    mkdir ${YARN_PID_DIR}
    chown yarn:hadoop ${YARN_PID_DIR}
    chown yarn:hadoop ${YARN_PID_DIR}

loading table of contents...