Upgrade HDP 2.2 Components

« Prev

	Important
	See the Release Notes for the HDP 2.4.3.0 repo information.

The upgrade process to HDP 2.4.3 involves the following steps.

Select your OS:

RHEL/CentOS/Oracle 6

On all hosts, clean the yum repository.
yum clean all

Remove your old HDP 2.2 components. This command uninstalls the HDP 2.2 components. It leaves the user data, and metadata, but removes your configurations. Refer to the HDP Component Files for RHEL/CentOS/Oracle6 table for the names of the files that need to be removed for each component and use the following format:

yum erase "accumulo_$VERSION_$BUILD*" "atlas_metadata_$VERSION_$BUILD*" "datafu_$VERSION_$BUILD*" "falcon_$VERSION_$BUILD*" "flume_$VERSION_$BUILD*" "hadoop_$VERSION_$BUILD*" "hadoop_$VERSION_$BUILD-client*" "hadoop_$VERSION_$BUILD-hdfs*" "hadooplzo_$VERSION_$BUILD*" ...

Where:

$VERSION is the HDP version number in the following format: 2_4_3_0
$HUE-VERSION is the Hue version number in the following format : 2.6.1.2.4.3.0
$BUILD is the HDP build number in the following format : 258

So, for example, the file name will look like: accumulo-2_4_3_0_258*

Table 2.4. HDP Component Files for RHEL/CentOS/Oracle6

Component	Associated Files
Accumulo	accumulo_$VERSION_$BUILD*
Atlas	atlas_metadata_$VERSION_$BUILD*
Datafu	datafu_$VERSION_$BUILD*
Falcon	falcon_$VERSION_$BUILD*
Flume	flume_$VERSION_$BUILD*
Hadoop (includes hadoop_client and hadoop_hdfs)	hadoop_$VERSION_$BUILD* hadoop_$VERSION_$BUILD-client* hadoop_$VERSION_$BUILD-hdfs* hadooplzo_$VERSION_$BUILD*
HBase	hbase_$VERSION_$BUILD*
Hive	hive_$VERSION_$BUILD* hive_hbase_$VERSION_$BUILD* hive_hcatalog_$VERSION_$BUILD* hive_jdbc_$VERSION_$BUILD* hive_metastore_$VERSION_$BUILD* hive_server_$VERSION_$BUILD* hive_webhcathive_hbase_$VERSION_$BUILD* ranger_hive_$VERSION_$BUILD*
Hue	hue_$HUE_VERSION_$BUILD* hue_beeswax_$HUE_VERSION_$BUILD* hue_common_$HUE_VERSION_$BUILD* hue_hcatalog_$HUE_VERSION_$BUILD* hue_oozie_$HUE_VERSION_$BUILD* hue_pig_$HUE_VERSION_$BUILD* hue_server_$HUE_VERSION_$BUILD*
Kafka	kafka_$VERSION_$BUILD*
Knox	knox_$VERSION_$BUILD*
Mahout	mahout_$VERSION_$BUILD*
MapReduce	hadoop_$VERSION_$BUILD-mapreduce*
Oozie	oozie_$VERSION_$BUILD*
Phoenix	phoenix_$VERSION_$BUILD
Pig	ranger_$VERSION_$BUILD*
Ranger	ranger_$VERSION_$BUILD*
Slider	slider_$VERSION_$BUILD*
Spark	spark_$VERSION_$BUILD*
Sqoop	sqoop_$VERSION_$BUILD*
Storm	storm_$VERSION_$BUILD*
Tez	tez_$VERSION_$BUILD*
YARN	hadoop_$VERSION_$BUILD-yarn*
ZooKeeper	zookeeper_$VERSION_$BUILD*

Validate that all HDP 2.2 component binaries are uninstalled:
yum list installed | grep @HDP2.2
Remove your old hdp.repo file:
rm /etc/yum.repos.d/hdp.repo

Install the HDP 2.4.3 repo:

Download the hdp.repo file:

wget -nv http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.3.0/hdp.repo -O /etc/yum.repos.d/hdp.repo

Confirm the HDP repository is configured.

yum repolist

You should see something like this. Verify that you have the HDP-2.4.3 directory:

Loaded plugins: fastestmirror, security 
Loading mirror speeds from cached hostfile
* base: mirrors.cat.pdx.edu
* extras: linux.mirrors.es.net
* updates: mirrors.usc.edu 
repo id repo namestatus
HDP-2.4.3.0 Hortonworks Data Platform Version - HDP-2.4.3.0

Install the HDP 2.4.3 versions of the components that you want to upgrade. For example, if you installed and want to upgrade all HDP 2.2 components:

yum install "hadoop" "hadoop-hdfs" "hadoop-libhdfs" "hadoop-yarn" "hadoop-mapreduce" "hadoop-client" "hadoop-lzo" "openssl" "hive-webhcat" "hive-hcatalog" "oozie" "collectd" "gccxml" "pig" "sqoop" "zookeeper" "hbase" "hue" "hive" "tez" "storm" "falcon" "flume" "phoenix" "accumulo" "mahout" "knox" "kafka" "spark" "ranger" "slider" "hdp_mon_nagios_addons"

	Note
	If you installed Apache Ranger, see Upgrade Ranger for more information on the upgrade path.

Verify that the components were upgraded.
yum list installed | grep HDP-<old.stack.version.number>
No component file names should appear in the returned list.

SLES 11 SP 1

On all hosts, clean the yum repository.
zypper clean -all

Remove your old HDP 2.2 components. This command uninstalls the HDP 2.2 components. It leaves the user data, and metadata, but removes your configurations. Refer to the HDP Component Files for SLES11 SP1 table for the names of the files that need to be removed for each component and use the following format:

zypper rm "accumulo_$VERSION_$BUILD*" "datafu_$VERSION_$BUILD*" "falcon_$VERSION_$BUILD*" "flume_$VERSION_$BUILD*" "hadoop_$VERSION_$BUILD*" "hadooplzo_$VERSION_$BUILD*" ...

The following table lists the files that need to be deleted for each of the HDP components. Variables are used to indicate:

$VERSION - HDP version number in the following format: 2_4_3_0
$HUE-VERSION - Hue version number in the following format: 2.6.1.2.3.0.0
$BUILD - the HDP build number in the following format: 258

So, for example, the file name will look like: accumulo-2_4_3_0_258*

Table 2.5. HDP Component Files for SLES11 SP1

Component	Associated Files
Accumulo	accumulo_$VERSION_$BUILD*
Atlas	atlas_metadata_$VERSION_$BUILD*
Datafu	datafu_$VERSION_$BUILD*
Falcon	falcon_$VERSION_$BUILD*
Flume	flume_$VERSION_$BUILD*
Hadoop (includes hadoop_client and hadoop_hdfs)	hadoop_$VERSION_$BUILD* hadoop_$VERSION_$BUILD-client* hadoop_$VERSION_$BUILD-hdfs* hadooplzo_$VERSION_$BUILD*
HBase	hbase_$VERSION_$BUILD*
Hive	hive_$VERSION_$BUILD* hive_hbase_$VERSION_$BUILD* hive_hcatalog_$VERSION_$BUILD* hive_jdbc_$VERSION_$BUILD* hive_metastore_$VERSION_$BUILD* hive_server_$VERSION_$BUILD* hive_webhcathive_hbase_$VERSION_$BUILD* ranger_hive_$VERSION_$BUILD*
Hue	hue_$HUE_VERSION_$BUILD* hue_beeswax_$HUE_VERSION_$BUILD* hue_common_$HUE_VERSION_$BUILD* hue_hcatalog_$HUE_VERSION_$BUILD* hue_oozie_$HUE_VERSION_$BUILD* hue_pig_$HUE_VERSION_$BUILD* hue_server_$HUE_VERSION_$BUILD*
Kafka	kafka_$VERSION_$BUILD*
Knox	knox_$VERSION_$BUILD*
Mahout	mahout_$VERSION_$BUILD*
MapReduce	hadoop_$VERSION_$BUILD-mapreduce*
Oozie	oozie_$VERSION_$BUILD*
Phoenix	phoenix_$VERSION_$BUILD
Pig	ranger_$VERSION_$BUILD*
Ranger	ranger_$VERSION_$BUILD*
Slider	slider_$VERSION_$BUILD*
Spark	spark_$VERSION_$BUILD*
Sqoop	sqoop_$VERSION_$BUILD*
Storm	storm_$VERSION_$BUILD*
Tez	tez_$VERSION_$BUILD*
YARN	hadoop_$VERSION_$BUILD-yarn*
ZooKeeper	zookeeper_$VERSION_$BUILD*

Validate that all HDP 2.2 component binaries are uninstalled:
yum list installed | grep @HDP2.2
Remove your old hdp.repo file:
rm /etc/zypp/repos.d/hdp.repo

Download the HDP 2.4.3 hdp.repo file:

wget -nv http://public-repo-1.hortonworks.com/HDP/sles11sp1/2.x/updates/2.4.3.0/hdp.repo -O /etc/zypp/repos.d/hdp.repo

Install the HDP 2.4.3 versions of the components that you want to upgrade. For example, if you installed and want to upgrade all HDP 2.2 components:
```
zypper install "hadoop" "hadoop-hdfs" "hadoop-libhdfs" "hadoop-yarn" "hadoop-mapreduce" "hadoop-client" "openssl" "hive-webhcat" "hive-hcatalog" "oozie" "collectd" "gccxml" "pig" "sqoop" "zookeeper" "hbase" "hive" "hue" "tez" "storm" "falcon" "flume" "phoenix" "accumulo" "mahout" "knox" "kafka" "spark" "slider" "hdp_mon_nagios_addons"
```
zypper install webhcat-tar-hive webhcat-tar-pig
zypper up -r HDP-2.4
zypper install oozie-client
Note
If you installed Apache Ranger, see Upgrade Ranger for more information on the upgrade path.
Verify that the components were upgraded. For example, to verify hdfs, hive, and hcatalog:
rpm -qa | grep hdfs && rpm -qa | grep hive && rpm -qa | grep hcatalog
No component files names should appear in the returned list.

SLES 11 SP3/SP4

On all hosts, clean the zypper repository.
zypper clean -a

Remove your old HDP 2.2 components. This command uninstalls the HDP 2.2 components. It leaves the user data, and metadata, but removes your configurations. Refer to the HDP Component Files for SLES11 SP3/SP4 table for the names of the files that need to be removed for each component and use the following format:

zypper rm "accumulo-$VERSION-$BUILD*" "datafu-$VERSION-$BUILD*" "falcon-$VERSION-$BUILD*" "flume-$VERSION-$BUILD*" "hadoop-$VERSION-$BUILD*" "hadooplzo-$VERSION-$BUILD*" ...

The following table lists the files that need to be deleted for each of the HDP components. Variables are used to indicate:

$VERSION - HDP version number in the following format: 2_4_3_0
$HUE-VERSION - Hue version number in the following format : 2.6.1.2.3.0.0
$BUILD - the HDP build number in the following format : 258

So, for example, the file name will look like: accumulo-2_4_3_0_258*

Table 2.6. HDP Component Files for SLES11 SP3/SP4

Component	Associated Files
Accumulo	accumulo_$VERSION_$BUILD*
Atlas	atlas_metadata_$VERSION_$BUILD*
Datafu	datafu_$VERSION_$BUILD*
Falcon	falcon_$VERSION_$BUILD*
Flume	flume_$VERSION_$BUILD*
Hadoop (includes hadoop_client and hadoop_hdfs)	hadoop_$VERSION_$BUILD* hadoop_$VERSION_$BUILD-client* hadoop_$VERSION_$BUILD-hdfs* hadooplzo_$VERSION_$BUILD*
HBase	hbase_$VERSION_$BUILD*
Hive	hive_$VERSION_$BUILD* hive_hbase_$VERSION_$BUILD* hive_hcatalog_$VERSION_$BUILD* hive_jdbc_$VERSION_$BUILD* hive_metastore_$VERSION_$BUILD* hive_server_$VERSION_$BUILD* hive_webhcathive_hbase_$VERSION_$BUILD* ranger_hive_$VERSION_$BUILD*
Hue	hue_$HUE_VERSION_$BUILD* hue_beeswax_$HUE_VERSION_$BUILD* hue_common_$HUE_VERSION_$BUILD* hue_hcatalog_$HUE_VERSION_$BUILD* hue_oozie_$HUE_VERSION_$BUILD* hue_pig_$HUE_VERSION_$BUILD* hue_server_$HUE_VERSION_$BUILD*
Kafka	kafka_$VERSION_$BUILD*
Knox	knox_$VERSION_$BUILD*
Mahout	mahout_$VERSION_$BUILD*
MapReduce	hadoop_$VERSION_$BUILD-mapreduce*
Oozie	oozie_$VERSION_$BUILD*
Phoenix	phoenix_$VERSION_$BUILD
Pig	ranger_$VERSION_$BUILD*
Ranger	ranger_$VERSION_$BUILD*
Slider	slider_$VERSION_$BUILD*
Spark	spark_$VERSION_$BUILD*
Sqoop	sqoop_$VERSION_$BUILD*
Storm	storm_$VERSION_$BUILD*
Tez	tez_$VERSION_$BUILD*
YARN	hadoop_$VERSION_$BUILD-yarn*
ZooKeeper	zookeeper_$VERSION_$BUILD*

Validate that all HDP 2.2 component binaries are uninstalled:
zypper search --installed-only --repo HDP-2.2.6.0
Remove your old hdp.repo file:
rm /etc/zypp/repos.d/hdp.repo
Download the HDP 2.4.3.0 hdp.repo file:
http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.4.3.0/hdp.repo -O /etc/zypp/repos.d/hdp.repo

Install the HDP 2.4.3.0 versions of the components that you want to upgrade. For example, if you installed and want to upgrade all HDP 2.2 components:

zypper install "hadoop" "hadoop-hdfs" "hadoop-libhdfs" "hadoop-yarn" "hadoop-mapreduce" "hadoop-client" "openssl" "oozie" "collectd" "gccxml" "pig" "sqoop" "zookeeper" "hbase" "hue" "hive" "tez" "storm" "falcon" "flume" "phoenix" "accumulo" "mahout" "knox" "ranger" "kafka" "spark" "spark-python" "hdp_mon_nagios_addons" "slider" "hive-webcat" "hive-hcatalog""

zypper up -r HDP-2.4

zypper install oozie-client

	Note
	If you installed Apache Ranger, see Upgrade Ranger for more information on the upgrade path.

Verify that the components were upgraded. For example, to verify hdfs, hive, and hcatalog:
rpm -qa | grep hdfs, && rpm -qa | grep hive && rpm -qa | grep hcatalog
No component files names should appear in the returned list.

Ubuntu 12

On all hosts, clean the apt-get repository.
apt-get clean -&-all

Remove your old HDP 2.2 components. This command uninstalls the HDP 2.2 components. It leaves the user data, and metadata, but removes your configurations. Refer to the HDP Component Files for Ubuntu 12 table for the names of the files that need to be removed for each component and use the following format:

apt-get remove "accumulo-$VERSION-$BUILD*" "datafu-$VERSION-$BUILD*" "falcon-$VERSION-$BUILD*" "flume-$VERSION-$BUILD*" "hadoop-$VERSION-$BUILD*" "hadooplzo-$VERSION-$BUILD*" ...

The following table lists the files that need to be deleted for each of the HDP components. Variables are used to indicate:

$VERSION - HDP version in the following format: 2-4-2-0
$HUE-VERSION - Hue version in the following format : 2.6.1.2.3.0.0
$BUILD - version of the HDP build in the following format : 258

So, for example, the file name will look like: accumulo-2-4-2-0-258*

Table 2.7. HDP Component Files for Ubuntu 12

Component	HDP Files to Re-install for Upgrade
Accumulo	accumulo-$VERSION-$BUILD*
Atlas	atlas-metadata-$VERSION-$BUILD*
Datafu	datafu-$VERSION-$BUILD*
Falcon	falcon-$VERSION-$BUILD*
Flume	flume-$VERSION-$BUILD*
Hadoop (includes hadoop-client and hadoop-hdfs)	hadoop-$VERSION-$BUILD* hadoop-$VERSION-$BUILD-client* hadoop-$VERSION-$BUILD-hdfs* hadooplzo-$VERSION-$BUILD*
HBase	hbase-$VERSION-$BUILD*
Hive	hive-$VERSION-$BUILD* hive-hbase-$VERSION-$BUILD* hive-hcatalog-$VERSION-$BUILD* hive-jdbc-$VERSION-$BUILD* hive-metastore-$VERSION-$BUILD* hive-server-$VERSION-$BUILD* hive-webhcathive-hbase-$VERSION-$BUILD* ranger-hive-$VERSION-$BUILD*
Hue	hue-$HUE-VERSION-$BUILD* hue-beeswax-$HUE-VERSION-$BUILD* hue-common-$HUE-VERSION-$BUILD* hue-hcatalog-$HUE-VERSION-$BUILD* hue-oozie-$HUE-VERSION-$BUILD* hue-pig-$HUE-VERSION-$BUILD* hue-server-$HUE-VERSION-$BUILD*
Kafka	kafka-$VERSION-$BUILD*
Knox	knox-$VERSION-$BUILD*
Mahout	mahout-$VERSION-$BUILD*
MapReduce	hadoop-$VERSION-$BUILD-mapreduce*
Oozie	oozie-$VERSION-$BUILD*
Phoenix	phoenix-$VERSION-$BUILD
Pig	ranger-$VERSION-$BUILD*
Ranger	ranger-$VERSION-$BUILD*
Slider	slider-$VERSION-$BUILD*
Spark	spark-$VERSION-$BUILD*
Sqoop	sqoop-$VERSION-$BUILD*
Storm	storm-$VERSION-$BUILD*
Tez	tez-$VERSION-$BUILD*
YARN	hadoop-$VERSION-$BUILD-yarn*
ZooKeeper	zookeeper-$VERSION-$BUILD*

Validate that all HDP 2.2 component binaries are uninstalled:
yum list installed | grep @HDP2.2
Remove your old hdp.repo file:
rm /etc/apt/sources.list.d/hdp.list
Download the HDP 2.4.3.0 hdp.repo file:
wget http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/updates/2.4.3.0/hdp.list - O /etc/apt/sources.list.d/hdp.list
Run an update:
apt-get update

Install the HDP 2.4.3 versions of the components that you want to upgrade. For example, if you installed and want to upgrade all HDP 2.2 components:

apt-get install "hadoop" "hadoop-lzo" "hadoop-hdfs" "libhdfs0" "hadoop-yarn" "hadoop-mapreduce" "hadoop-client" "openssl" "webhcat" "hcatalog" "oozie" "collectd" "gccxml" "pig" "sqoop" "zookeeper" "hbase" "hive" "tez" "storm" "falcon" "flume" "phoenix" "accumulo" "mahout" "knox" "spark" "slider*" "hdp_mon_nagios_addons"

	Note
	If you installed Apache Ranger, see Upgrade Ranger for more information on the upgrade path.

Debian 6 (Deprecated)

HDP support for Debian 6 is deprecated with HDP 2.4.3. Future versions of HDP will no longer be supported on Debian 6.

On all hosts, clean the apt-get repository.
apt-get clean

Remove your old HDP 2.2 components. This command uninstalls the HDP 2.2 components. It leaves the user data, and metadata, but removes your configurations. Refer to the HDP Component Files for Debian 6 (Deprecated)/Debian 7 table for the names of the files that need to be removed for each component and use the following format:

apt-get remove "accumulo-$VERSION-$BUILD*" "datafu-$VERSION-$BUILD*" "falcon-$VERSION-$BUILD*" "flume-$VERSION-$BUILD*" "hadoop-$VERSION-$BUILD*" "hadooplzo-$VERSION-$BUILD*" ...

The following table lists the files that need to be deleted for each of the HDP components. Variables are used to indicate:

$VERSION - HDP version in the following format: 2-4-2-0
$HUE-VERSION - Hue version in the following format : 2.6.1.2.3.0.0
$BUILD - version of the HDP build in the following format : 258

So, for example, the file name will look like: accumulo-2-4-2-0-258*

Table 2.8. HDP Component Files for Debian 6 (Deprecated)/Debian 7

Component	HDP Files to Re-install for Upgrade
Accumulo	accumulo-$VERSION-$BUILD*
Atlas	atlas-metadata-$VERSION-$BUILD*
Datafu	datafu-$VERSION-$BUILD*
Falcon	falcon-$VERSION-$BUILD*
Flume	flume-$VERSION-$BUILD*
Hadoop (includes hadoop-client and hadoop-hdfs)	hadoop-$VERSION-$BUILD* hadoop-$VERSION-$BUILD-client* hadoop-$VERSION-$BUILD-hdfs* hadooplzo-$VERSION-$BUILD*
HBase	hbase-$VERSION-$BUILD*
Hive	hive-$VERSION-$BUILD* hive-hbase-$VERSION-$BUILD* hive-hcatalog-$VERSION-$BUILD* hive-jdbc-$VERSION-$BUILD* hive-metastore-$VERSION-$BUILD* hive-server-$VERSION-$BUILD* hive-webhcathive-hbase-$VERSION-$BUILD* ranger-hive-$VERSION-$BUILD*
Hue	hue-$HUE-VERSION-$BUILD* hue-beeswax-$HUE-VERSION-$BUILD* hue-common-$HUE-VERSION-$BUILD* hue-hcatalog-$HUE-VERSION-$BUILD* hue-oozie-$HUE-VERSION-$BUILD* hue-pig-$HUE-VERSION-$BUILD* hue-server-$HUE-VERSION-$BUILD*
Kafka	kafka-$VERSION-$BUILD*
Knox	knox-$VERSION-$BUILD*
Mahout	mahout-$VERSION-$BUILD*
MapReduce	hadoop-$VERSION-$BUILD-mapreduce*
Oozie	oozie-$VERSION-$BUILD*
Phoenix	phoenix-$VERSION-$BUILD
Pig	ranger-$VERSION-$BUILD*
Ranger	ranger-$VERSION-$BUILD*
Slider	slider-$VERSION-$BUILD*
Spark	spark-$VERSION-$BUILD*
Sqoop	sqoop-$VERSION-$BUILD*
Storm	storm-$VERSION-$BUILD*
Tez	tez-$VERSION-$BUILD*
YARN	hadoop-$VERSION-$BUILD-yarn*
ZooKeeper	zookeeper-$VERSION-$BUILD*

Validate that all HDP 2.2 component binaries are uninstalled:
dpkg -l | grep "^ii" | grep hadoop
Remove your old hdp.repo file:
rm /etc/apt/sources.list.d/hdp.list
Download the HDP 2.4.3 hdp.repo file:
wget -nv http://public-repo-1.hortonworks.com/HDP/debian6/2.x/updates/2.4.3.0/hdp.list - O /etc/apt/sources.list.d/hdp.list
Run an update:
apt-get update

Install the HDP 2.4.3 versions of the components that you want to upgrade. For example, if you installed and want to upgrade all HDP 2.2 components:

apt-get install "hadoop" "hive-webhcat" "hadoop-hdfs" "libhdfs0" "hadoop-yarn" "hadoop-mapreduce" "hadoop-client" "openssl" "hcatalog" "oozie" "collectd" "gccxml" "pig" "sqoop" "zookeeper" "hbase" "hive" "tez" "storm" "falcon" "flume" "phoenix" "accumulo" "mahout" "knox" "spark" "slider*" "hdp_mon_nagios_addons"

	Note
	If you installed Apache Ranger, see Upgrade Ranger for more information on the upgrade path.

​Upgrade HDP 2.2 Components