Upgrade YARN and MapReduce2
Prepare MR2 and YARN for work. Execute HDFS commands on any host.
Create mapreduce dir in hdfs.
su -l <HDFS_USER> -c "hdfs dfs -mkdir -p /hdp/apps/2.3.x.y-z/mapreduce/"Copy new mapreduce.tar.gz to HDFS mapreduce dir.
su -l <HDFS_USER> -c "hdfs dfs -put /usr/hdp/current/hadoop-client/mapreduce.tar.gz /hdp/apps/2.3.x.y-z/mapreduce/."Copy new hadoop-streaming.jar to HDFS mapreduce dir.
su -l <HDFS_USER> -c "hdfs dfs -put /usr/hdp/current/hadoop-mapreduce-client/hadoop-streaming.jar /hdp/apps/2.3.x.y-z/mapreduce/."Grant permissions for created mapreduce dir in hdfs.
su -l <HDFS_USER> -c "hdfs dfs -chown -R <HDFS_USER>:<HADOOP_GROUP> /hdp" su -l <HDFS_USER> -c "hdfs dfs -chmod -R 555 /hdp/apps/2.3.x.y-z/mapreduce" su -l <HDFS_USER> -c "hdfs dfs -chmod -R 444 /hdp/apps/2.3.x.y-z/mapreduce/mapreduce.tar.gz" su -l <HDFS_USER> -c "hdfs dfs -chmod -R 444 /hdp/apps/2.3.x.y-z/mapreduce/hadoop-streaming.jar"
Update YARN Configuration Properties for HDP 2.3
From Ambari Web, browse to >
Services>YARN>Configs. Filter for the following properties, and add these properties if they do not already exist:Name
Value
hadoop.registry.zk.quorum
The comma-separated list of ZooKeeper hosts defining the ZooKeeper quorum.
For example:
ZooKeeperHost1:2181, ZooKeeperHost2:2181
yarn.resourcemanager.zk-address
The comma-separated list of ZooKeeper hosts defining the ZooKeeper quorum.
For example:
ZooKeeperHost1:2181, ZooKeeperHost2:2181
Manually clear the ResourceManager state store.
su -l <YARN_USER> -c "yarn resourcemanager -format-state-store"where <YARN_USER> is the YARN Service user. For example, yarn.
Using Ambari Web, browse to
Services > YARNand select Start from the Service Actions menu. Repeat for MapReduce2.Run the smoke checks. Using Ambari Web, browse to Services > YARN and select Run Service Check from the Service Actions menu. Confirm the check passes. Repeat for MapReduce2.