4. Upgrading HDP 1.3.7 to 1.3.9

Download the appropriate hdp.repo file for your OS:

OR Download the HDP RPMs single repository tarball. (For further information, see the local repository instructions.)

Stop all services.

If you are managing your deployment via Ambari, open Ambari Web, browse to Services and use the Service Actions command to stop each service.

If you have a manually installed cluster, use the Stopping HDP Services instructions.

	Note
	If you are upgrading an HA NameNode configuration, keep your JournalNodes running while performing this upgrade procedure. Upgrade, rollback and finalization operations on HA NameNodes must be performed with all JournalNodes running.

Stop all services (including MapReduce) and client applications deployed on HDFS.

Use the instructions provided here.

Run the fsck command as instructed below and fix any errors. (The resulting file will contain a complete block map of the file system.)

su $HDFS_USER
hadoop fsck / -files -blocks -locations > dfs-old-fsck-1.log 

where $HDFS_USER is the HDFS Service user. For example, hdfs.

Use the following instructions to compare the status before and after the upgrade:

	Note
	The following commands must be executed by the user running the HDFS service (by default, the user is hdfs).

As an HDFS user, execute the following command to save namespace:

su $HDFS_USER
hadoop dfsadmin -safemode enter
hadoop dfsadmin -saveNamespace

where $HDFS_USER is the HDFS Service user. For example, hdfs.

Copy the following checkpoint files into a backup directory:

Stop the HDFS service.

If you are managing your deployment via Ambari, open Ambari Web, browse to Services and use the Service Actions command to stop the service.

If you are installing manually, use the instructions provided here.

	Note
	Verify that all the HDP services in the cluster are stopped.

If you have Oozie installed:

Back up the files in the following directories on the Oozie server host, and make sure all files, including *site.xml files, are copied.

mkdir oozie-conf-bak cp -R /etc/oozie/conf/* oozie-conf-bak 

Remove the old Oozie directories on all Oozie server and client hosts

rm -R /etc/oozie/conf/* oozie-conf-bak 

Back up the Hive Database.

Upgrade the stack on all Agent hosts.

$HIVE_HOME/bin/schematool -upgradeSchema -dbType <$databaseType>  -userName
    <$connectionUserName> -passWord  <$connectionPassWord>

yum upgrade "collectd*" "gccxml*" "pig*" "hadoop*" "oozie" "oozie-client" "sqoop*"
"zookeeper*" "hbase*" "webhcat" "hive*" "hcatalog" hdp_mon_nagios_addons

yum list installed | grep
HDP-$old-stack-version-number               

$HIVE_HOME/bin/schematool -upgradeSchema -dbType <$databaseType>  -userName
    <$connectionUserName> -passWord  <$connectionPassWord>

zypper up "collectd*" "epel-release*" "gccxml*" "pig*" "hadoop*"  "oozie" "oozie-client"
"storm*" "sqoop*" "zookeeper*" "hbase*" "hive*" "hcatalog" hdp_mon_nagios_addons    

zypper up -r HDP-1.3.9.0             

rpm -qa | grep hadoop, rpm -qa | grep hive and rpm -qa| grep hcatalog

If you are upgrading from an HA NameNode configuration, restart all JournalNodes.

On each JournalNode host, enter the following command:

su -l {HDFS_USER} -c “/usr/lib/hadoop/bin/hadoop-daemon.sh start journalnode”

Complete the Stack upgrade.

If this is an Ambari-managed cluster, update the Repository Base URLs to use the HDP 1.3.9 repositories for HDP and HDP-UTILS. For Ambari 1.6.1 or earlier, enter:

ambari-server upgradestack HDP-1.3 //public-repo-1.hortonworks.com/HDP/{$os}/1.x/updates/1.3.9.0 {$os}

where {$os} is the Operating System Family (OS Family). See the following table:

Restart services.

If you are managing your deployment via Ambari, open Ambari Web, browse to Services and use the Service Actions command to start each service.

If you have a manually installed cluster, use the Starting HDP Services instructions.

Finalize the upgrade.

If you are not yet ready to discard your backup, you can start the upgraded HDFS without finalizing the upgrade. (At this stage, you can still roll back if need be.)

Verify your filesystem health. When you are ready to commit to this upgrade (are certain that you do NOT want to roll back), discard your backup and finalize the upgrade.

As $HDFS_USER, execute the following command:

hdfs dfsadmin -finalizeUpgrade