You must create local directories for YARN on every note of NodeManagers (TaskTrackers) and set the appropriate permissions for your YARN log directories.
Set the permissions on the
yarn.nodemanager.local-dirs
directories. Run these commands on all DataNodes in your cluster.chown -R yarn:hadoop ${yarn.nodemanager.local-dirs} chmod 755 ${yarn.nodemanager.local-dirs}
where
${yarn.nodemanager.local-dirs}
is your local directory.Change permissions of directories of
yarn.nodemanager.log-dirs
. Run these commands on all DataNodes in your cluster.chown -R yarn:hadoop ${yarn.nodemanager.log-dirs} chmod 755 ${yarn.nodemanager.log-dirs}
where
${yarn.nodemanager.log-dirs}
is your log directory.Create directories for YARN_LOG_DIR and YARN_PID_DIR.
Open
/etc/hadoop/conf/yarn-env.sh
Write down your values for YARN_LOG_DIR and YARN_PID_DIR as the following instructions require values for the
${YARN_LOG_DIR}
and${YARN_PID_DIR}
. For example inyarn-env.sh
:${YARN_LOG_DIR}=/grid/0/var/log/hadoop/yarn ${YARN_PID_DIR}=/grid/0/var/run/hadoop/yarn
Make directories for
${YARN_LOG_DIR}
and${YARN_PID_DIR}
and set the appropriate permissions for them.mkdir ${YARN_LOG_DIR} chown yarn:hadoop ${YARN_LOG_DIR} chown yarn:hadoop ${YARN_LOG_DIR} mkdir ${YARN_PID_DIR} chown yarn:hadoop ${YARN_PID_DIR} chown yarn:hadoop ${YARN_PID_DIR}