3. Migrate the HDP Configurations

Back up the following HDP 1.x configurations on all nodes in your clusters.

Edit /etc/hadoop/conf/core-site.xml and set hadoop.rpc.protection from none to authentication.

Copy your /etc/hcatalog/conf configurations to /etc/hive-hcatalog/conf and /etc/hive-webhcat as appropriate.

Download the your HDP 2.x companion files from Download Companion Files and migrate your HDP 1.x configuration.

Copy log4j.properties from the hadoop config directory of the companion files to /etc/hadoop/conf. The file should have owners and permissions similar to other files in /etc/hadoop/conf.

Copy these configurations to all nodes in your clusters.

	Note
	Upgrading the repo using yum or zypper resets all configurations. Prepare to replace these configuration directories each time you perform a yum or zypper upgrade.

Review the following HDP 1.3.2 Hadoop Core configurations and the new configurations or locations in HDP 2.x

	Note
	The hadoop.rpc.protection config needs to specify authentication, integrity and/or privacy. No value defaults to authentication, but an invalid value such as "none" causes an error.

Review the following 1.3.2 HDFS site configations and their new configurations and files in HDP 2.x.

Review the follwing HDP 1.3.2 MapReduce Configs and their new HDP 2.x Mappings

Review the following HDP 1.3.2 Configs and their new HDP 2.x Capacity Scheduler mappings.

Compare the following HDP 1.3.2 configs in hadoop-env.sh with the new configs in HDP 2.x

Add the following properties to the yarn-site.xml file:

<property> 
 <name>yarn.resourcemanager.scheduler.class</name> 
 <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>  
</property>

<property> 
 <name>yarn.resourcemanager.resource-tracker.address</name> 
 <value>$resourcemanager.full.hostname:8025</value>  
 <description>Enter your ResourceManager hostname.</description>
</property>

<property> 
 <name>yarn.resourcemanager.scheduler.address</name> 
 <value>$resourcemanager.full.hostname:8030</value>  
 <description>Enter your ResourceManager hostname.</description>
</property>

<property> 
 <name>yarn.resourcemanager.address</name> 
 <value>$resourcemanager.full.hostname:8050</value>  
 <description>Enter your ResourceManager hostname.</description>
</property>

<property> 
 <name>yarn.resourcemanager.admin.address</name> 
 <value>$resourcemanager.full.hostname:8141</value>  
 <description>Enter your ResourceManager hostname.</description>
</property>

<property> 
 <name>yarn.nodemanager.local-dirs</name> 
 <value>/grid/hadoop/yarn/local,/grid1/hadoop/yarn/local</value>  
 <description>Comma separated list of paths. Use the list of directories from $YARN_LOCAL_DIR.  
                For example, /grid/hadoop/yarn/local,/grid1/hadoop/yarn/local.</description>
</property>

<property> 
 <name>yarn.nodemanager.log-dirs</name> 
 <value>/grid/hadoop/yarn/log</value>
 <description>Use the list of directories from $YARN_LOCAL_LOG_DIR.  
                For example, /grid/hadoop/yarn/log,/grid1/hadoop/yarn/log,/grid2/hadoop/yarn/log</description>
</property>

<property> 
 <name>yarn.log.server.url</name> 
 <value>http://$jobhistoryserver.full.hostname:19888/jobhistory/logs/</value>
 <description>URL for job history server</description>
</property>

<property> 
 <name>yarn.resourcemanager.webapp.address</name> 
 <value>$resourcemanager.full.hostname:8088</value>
 <description>URL for job history server</description>
</property>

Add the following properties to the mapred-site.xml file:

<property>  
 <name>mapreduce.jobhistory.address</name> 
 <value>$jobhistoryserver.full.hostname:10020</value>  
 <description>Enter your JobHistoryServer hostname.</description>
</property>

<property>  
 <name>mapreduce.jobhistory.webapp.address</name>  
 <value>$jobhistoryserver.full.hostname:19888</value>  
 <description>Enter your JobHistoryServer hostname.</description>
</property>

<property>
  <name>mapreduce.shuffle.port</name>
  <value>13562</value>
</property>

<property>
  <name>mapreduce.framework.name</name>
  <value>yarn</value>
</property>                 

For a secure cluster, add the following properties to mapred-site.xml:

<property>
    <name>mapreduce.jobhistory.principal</name>
    <value>jhs/_PRINCIPAL@$REALM.ACME.COM</value>
    <description>Kerberos principal name for the MapReduce JobHistory Server.</description>
</property>
<property>
    <name>mapreduce.jobhistory.keytab</name>
    <value>/etc/security/keytabs/jhs.service.keytab</value>
    <description>Kerberos keytab file for the MapReduce JobHistory Server.</description>
</property>

For a secure cluster, you must also update hadoop.security.auth_to_local in core-site.xml to include a rule regarding the mapreduce.jobhistory.principal value you set in the previous step.

RULE:[2:$1@$0](PRINCIPAL@$REALM.ACME.COM)s/.*/mapred/

where PRINCIPAL and REALM are the kerberos principal and realm you specified in mapreduce.jobhistory.principal.

Delete any remaining HDP1 properties in the mapred-site.xml file.