2. Upgrade Hadoop

  1. On all nodes, clean the yum repository.

    • For RHEL/CentOS:

      yum clean all 
    • For SLES:

      zypper clean --all
    • For Ubuntu or Debian:

      apt-get clean all

  2. Uninstall the HDP 1.x packages.

    • For RHEL/CentOS:

      yum erase hadoop-pipes hadoop-sbin hadoop-native oozie 
    • For SLES:

      zypper rm hadoop-pipes hadoop-sbin hadoop-native oozie hbase hadoop*
    • For Ubuntu or Debian:

      apt-get remove hadoop-pipes hadoop-sbin hadoop-native oozie hbase hadoop* 

  3. Configure your repository.

    The standard HDP install fetches the software from a remote yum repository
 over the Internet. To use this option, you must set up access to the remote
 repository and have an available Internet connection for each of your


    If your cluster does not have access to the Internet, or you are
 creating a large cluster and you want to conserve bandwidth, you can
 instead provide a local copy of the HDP repository that your hosts can
 access. For more information, see Deployment Strategies for Data Centers with
 Firewalls, a separate document in this set.

    1. For each node in your cluster, download the yum repo configuration
 file hdp.repo. From a terminal window,
 enter the following wget command.

      • For RHEL/CentOS/Oracle Linux 5

wget http://public-repo-1.hortonworks.com/HDP/centos5/2.x/updates/ -O /etc/yum.repos.d/hdp.repo

      • For RHEL/CentOS/Oracle Linux 6:

wget http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/ -O /etc/yum.repos.d/hdp.repo

      • For SLES 11 SP1:

wget http://public-repo-1.hortonworks.com/HDP/suse11sp1/2.x/updates/ -O /etc/zypp/repos.d/hdp.repo

      • For SLES 11 SP3:

wget http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/ -O /etc/zypp/repos.d/hdp.repo

      • For Ubuntu 12.04

wget http://public-repo-1.hortonworks.com/HDP/ubuntu12/ -O /etc/apt-get/repos.d/hdp.list

      • For Debian 6:

        wget http://public-repo-1.hortonworks.com/HDP/debian6/ -O /etc/apt-get/repos.d/hdp.list
    2. Confirm the HDP repository is configured by checking the repo

      • For RHEL/CentOS/Oracle Linux:

        yum repolist
      • For SLES:

        zypper repos

      • For Ubuntu or Debian:

        apt-get list

  4. Use the HDP Utility script to calculate memory configuration settings.

  5. Install Hadoop

    • For RHEL/CentOS/Oracle Linux:

      yum upgrade hadoop*

    • For SLES:

      zypper install hadoop* hadoop-hdfs hadoop-lzo
    • For Ubuntu or Debian:

      apt-get update
      apt-get install hadoop hadoop-hdfs libhdfs0 libhdfs0-dev hadoop-yarn hadoop-mapreduce hadoop-client opensslliblzo2-2 liblzo2-dev hadoop-lzo 
  6. Install YARN

    • For RHEL/CentOS/Oracle Linux:

      yum install hadoop-mapreduce hadoop-yarn

    • For SLES:

      zypper install hadoop-mapreduce hadoop-yarn
    • For Ubuntu or Debian:

      apt-get install hadoop-mapreduce hadoop-yarn
  7. Verify HDP 2.x packages have installed successfully.

    • For RHEL/CentOS/Oracle Linux:

      yum list hadoop*|grep HDP-2 

    • For SLES:

      zypper pa|grep HDP-2 

      Verify that you have HDP 2.x installed:

      hadoop version

      You may need to add /etc/hadoop/conf/hadoop-env.sh in /usr/bin/hadoop for $JAVA_HOME.

    • For Ubuntu or Debian:

       dpkg -s HDP-2 | grep Status 

loading table of contents...