You must create local directories for YARN on each NodeManager host in your cluster (in HDP-2, the NodeManager replaces the TaskTracker) and set the appropriate permissions for the YARN log directories. If these directories do not exist, you can create them using the instructions on this page.
Set the permissions in the
yarn.nodemanager.local-dirs
directories. Run these commands on all DataNodes in your cluster.chown -R yarn:hadoop ${yarn.nodemanager.local-dirs} chmod 755 ${yarn.nodemanager.local-dirs}
where
${yarn.nodemanager.local-dirs}
is your local directory.Change the permissions of the directories in
yarn.nodemanager.log-dirs
. If these directories do not exist, you can create them using the instructions on this page. Run these commands on all DataNodes in your cluster.chown -R yarn:hadoop ${yarn.nodemanager.log-dirs} chmod 755 ${yarn.nodemanager.log-dirs}
where
${yarn.nodemanager.log-dirs}
is your log directory.Create directories for YARN_LOG_DIR and YARN_PID_DIR.
Open
/etc/hadoop/conf/yarn-env.sh
Write down your values for YARN_LOG_DIR and YARN_PID_DIR as the following instructions require values for the
${YARN_LOG_DIR}
and${YARN_PID_DIR}
. For example inyarn-env.sh
:YARN_LOG_DIR=/grid/0/var/log/hadoop/yarn YARN_PID_DIR=/grid/0/var/run/hadoop/yarn
Make directories for
${YARN_LOG_DIR}
and${YARN_PID_DIR}
and set the appropriate permissions for them.mkdir ${YARN_LOG_DIR} chown yarn:hadoop ${YARN_LOG_DIR} chmod 755 ${YARN_LOG_DIR} mkdir ${YARN_PID_DIR} chown yarn:hadoop ${YARN_PID_DIR} chmod 755 ${YARN_PID_DIR}