Optional: If you are using the
LinuxContainerExecutor
, you must set up
container-executor.cfg
in the config
directory. The file must be owned by root:root
. The settings are
in the form of key=value
with one key per line. There must
entries for all keys. If you do not want to assign a value for a key, you can leave
it unset in the form of key=#
.
The keys are defined as follows:
yarn.nodemanager.linux-container-executor.group
- the configured value ofyarn.nodemanager.linux-container-executor.group
. This must match the value ofyarn.nodemanager.linux-container-executor.group
inyarn-site.xml
.banned.users
- a comma separated list of users who cannot runcontainer-executor
.min.user.id
- the minimum value of user id, this is to prevent system users from runningcontainer-executor
.allowed.system.users
- a comma separated list of allowed system users.
To start YARN, run commands as a YARN user. To start MapReduce, run commands as a MapReduce user.
If you have a secure cluster, create the following principals and keytabs for YARN before you start the YARN service:
yarn.resourcemanager.principal yarn.resourcemanager.keytab yarn.nodemanager.principal yarn.nodemanager.keytab yarn.resourcemanager.webapp.spnego-principal yarn.nodemanager.webapp.spnego-principal mapreduce.jobhistory.webapp.spnego-principal yarn.resourcemanager.webapp.spnego-keytab-file yarn.nodemanager.webapp.spnego-keytab-file mapreduce.jobhistory.webapp.spnego-keytab-file
Start the ResourceManager on your previous JobTracker host.
su $YARN_USER export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh start resourcemanager ps -ef | grep -i resourcemanager
Prepare the NodeManager hosts.
Change permissions for
/usr/lib/hadoop-yarn/bin/container-executor.cfg
chown yarn:hadoop /usr/lib/hadoop-yarn/bin/container-executor chmod -R 650 /usr/lib/hadoop-yarn/bin/container-executor
On all NodeManager hosts, add the yarn user to the hadoop group.
For example if you are using CentOS6:
usermod -a -G hadoop yarn
Start the NodeManager on your previous TaskTracker hosts.
su $YARN_USER export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec /usr/lib/hadoop-yarn/sbin/yarn-daemon.sh start nodemanager ps -ef | grep -i nodemanager
To start MapReduce, run the following commands as MapReduce user:
su $MAPREDUCE_USER export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec /usr/lib/hadoop-mapreduce/sbin/mr-jobhistory-daemon.sh --config /etc/hadoop/conf start historyserver ps -ef | grep -i jobhistoryserver