This is the documentation for Cloudera Manager 5.0.x. Documentation for other versions is available at Cloudera Documentation.

Upgrading CDH 5 Using Packages

If you originally used Cloudera Manager to install your CDH service using packages, you can upgrade to a new version of CDH 5 using either packages or parcels. Using parcels is recommended, because the upgrade wizard for parcels handles the upgrade almost completely automatically. But if you want to continue to use packages, you can perform an upgrade following the instructions presented here.

To upgrade CDH using packages, the steps are as follows.

  1. Upgrade Unmanaged Components
  2. Stop All Services
  3. Back up the HDFS Metadata on the NameNode
  4. Upgrade Managed Components
  5. Using the "1-Click" Package
  6. Update Symlinks for the Newly Installed Components
  7. Upgrade Sqoop
  8. Upgrade the Oozie Database
  9. Start All Services
  10. Configure Cluster CDH Version for Package Installs
  11. Deploy Client Configurations

Upgrade Unmanaged Components

Upgrade unmanaged components before proceeding to upgrade managed components. Components that you might have installed that are not managed by Cloudera Manager include:
  • Pig
  • Whirr
  • Mahout

For information on upgrading these unmanaged components, see the CDH 5 Installation Guide.

Stop All Services

  1. Stop each cluster.
    1. On the Home page, click to the right of the cluster name and select Stop.
    2. Click Stop in the confirmation screen. The Command Details window shows the progress of stopping services.

      When All services successfully stopped appears, the task is complete and you can close the Command Details window.

  2. Stop the Cloudera Management Service:
    1. Do one of the following:
        1. Select Clusters > Cloudera Management Service > mgmt.
        2. Select Actions > Stop.
        1. On the Home page, click to the right of mgmt and select Stop.
    2. Click Stop to confirm. The Command Details window shows the progress of stopping the roles.
    3. When Command completed with n/n successful subcommands appears, the task is complete. Click Close.

Back up the HDFS Metadata on the NameNode

  1. Stop the NameNode you want to back up.
  2. Go to the HDFS service.
  3. Select Configuration > View and Edit.
  4. In the Search field, search for "NameNode Data Directories". This locates the NameNode Data Directories property.
  5. From the command line on the NameNode host, back up the directory listed in the NameNode Data Directories property. If more than one is listed, then you only need to make a backup of one directory, since each directory is a complete copy. For example, if the data directory is /mnt/hadoop/hdfs/name, do the following as root:
    # cd /mnt/hadoop/hdfs/name
    # tar -cvf /root/nn_backup_data.tar .

    You should see output like this:

    ./
    ./current/
    ./current/fsimage
    ./current/fstime
    ./current/VERSION
    ./current/edits
    ./image/
    ./image/fsimage
      Warning: If you see a file containing the word lock, the NameNode is probably still running. Repeat the preceding steps, starting by shutting down the CDH services.

Upgrade Managed Components

Use one of the following strategies to upgrade CDH 5:
  • Use your operating system's package management tools to update all packages to the latest version using standard repositories. This approach works well because it minimizes the amount of configuration required and uses the simplest commands. Be aware that this can take a considerable amount of time if you have not upgraded the system recently. To update all packages on your system, use the following command:
    Operating System Command
    RHEL
    $ sudo yum update
    SLES
    $ sudo zypper up
    Ubuntu or Debian
    $ sudo apt-get upgrade
  • Use Cloudera's "1-click Install" package. This is the simplest way to upgrade only the Cloudera packages. Follow the instructions in Using the "1-Click" Package.

Using the "1-Click" Package

Follow the instructions below if you have decided to use the "1-click" package.

  1. Check whether you have the CDH 5 "1-click" repository installed.
    • Red Hat/CentOS-compatible and SLES
      rpm -q CDH 5-repository

      If you are upgrading from CDH 5 Beta 1 or later, and you used the "1-click" package for the previous CDH 5 release, you should see:

      CDH5-repository-1-0

      In this case, skip to installing the CDH 5 packages. If instead you see:

      package CDH 5-repository is not installed

      proceed with installing the 1-click package.

    • Ubuntu and Debian
      dpkg -l | grep CDH 5-repository

      If the repository is installed, skip to installing the CDH 5 packages; otherwise proceed with installing the "1-click" package.

  2. If the CDH 5 "1-click" repository is not already installed on each host in the cluster, follow the instructions below for that host's operating system.
    • Red Hat compatible
      1. Download and install the "1-click Install" package.
        1. Download the CDH 5 "1-click Install" package.

          Click the entry in the table below that matches your Red Hat or CentOS system, choose Save File, and save the file to a directory to which you have write access (it can be your home directory).

          OS Version Click this Link
          Red Hat/CentOS/Oracle 5 Red Hat/CentOS/Oracle 5 link
          Red Hat/CentOS/Oracle 6 Red Hat/CentOS/Oracle 6 link
        2. Install the RPM:
          • Red Hat/CentOS/Oracle 5
            $ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm 
          • Red Hat/CentOS/Oracle 6
            $ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm
      2. (Optionally) add a repository key:
        • Red Hat/CentOS/Oracle 5
          $ sudo rpm --import http://archive.cloudera.com/cdh5/redhat/5/x86_64/cdh/RPM-GPG-KEY-cloudera
        • Red Hat/CentOS/Oracle 6
          $ sudo rpm --import http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera
    • SLES
      1. Download and install the "1-click Install" package:
        1. Download the CDH 5 "1-click Install" package.

          Click this link, choose Save File, and save it to a directory to which you have write access (it can be your home directory).

        2. Install the RPM:
          $ sudo rpm -i cloudera-cdh-5-0.x86_64.rpm
        3. Update your system package index by running:
          $ sudo zypper refresh
      2. (Optionally) add a repository key:
        $ sudo rpm --import http://archive.cloudera.com/cdh5/sles/11/x86_64/cdh/RPM-GPG-KEY-cloudera  
    • Ubuntu and Debian
      1. Download and install the "1-click Install" package:
        1. Download the CDH 5 "1-click Install" package:
          OS Version Click this Link
          Wheezy Wheezy link
          Precise Precise link
        2. Install the package. Do one of the following:
          • Choose Open with in the download window to use the package manager.
          • Choose Save File, save the package to a directory to which you have write access (it can be your home directory) and install it from the command line, for example:
            sudo dpkg -i cdh5-repository_1.0_all.deb
      2. (Optionally) add a repository key:
        • Ubuntu Lucid
          $ curl -s http://archive.cloudera.com/cdh5/ubuntu/lucid/amd64/cdh/archive.key | sudo apt-key add -
        • Ubuntu Precise
          $ curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -
        • Debian Wheezy
          $ curl -s http://archive.cloudera.com/cdh5/debian/wheezy/amd64/cdh/archive.key | sudo apt-key add -
  3. Install the CDH packages:
    • Red Hat compatible
      $ sudo yum clean all
      $ sudo yum install avro-tools crunch flume-ng hadoop-hdfs-fuse hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-pig hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell kite llama mahout oozie pig pig-udf-datafu search sentry solr-mapreduce spark-python sqoop sqoop2 whirr
    • SLES
      $ sudo zypper clean --all
      $ sudo zypper install  avro-tools crunch flume-ng hadoop-hdfs-fuse hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-pig hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell kite llama mahout oozie pig pig-udf-datafu search sentry solr-mapreduce spark-python sqoop sqoop2 whirr
    • Ubuntu and Debian
      $ sudo apt-get update
      $ sudo apt-get install avro-tools crunch flume-ng hadoop-hdfs-fuse hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-pig hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell kite llama mahout oozie pig pig-udf-datafu search sentry solr-mapreduce spark-python sqoop sqoop2 whirr
      Note: Installing these packages will also install all the other CDH packages that are needed for a full CDH 5 installation.

Update Symlinks for the Newly Installed Components

Restart all the Cloudera Manager agents to force an update of the symlinks to point to the newly installed components on each host:
$ sudo service cloudera-scm-agent restart

Upgrade Sqoop

  1. Go to the Sqoop service.
  2. Select Actions > Upgrade Sqoop and click Upgrade Sqoop to confirm.
  3. When the command completes, click Close.

Upgrade the Oozie Database

  1. Go to the Sqoop service.
  2. Select Actions > Upgrade Database and click Upgrade Database to confirm.
  3. When the command completes, click Close.

Start All Services

  1. Start each cluster.
  2. Start the Cloudera Management Service.

    When the command completes, click Close.

Configure Cluster CDH Version for Package Installs

Because Cloudera Manager does not manage service software installed as packages, during certain upgrade scenarios Cloudera Manager assigns a default CDH version of a cluster. You must manually configure the cluster CDH version to match the package CDH version following the procedure in Configuring the CDH Version for a Cluster in Managing Clusters with Cloudera Manager. If you do not set the cluster CDH version to the package CDH version, Cloudera Manager will incorrectly enable and disable service features based on the configured CDH version.

Deploy Client Configurations

  1. From the top Actions button that corresponds to the cluster and choose Deploy Client Configuration....
  2. Click the Deploy Client Configuration button in the confirmation pop-up that appears.
Page generated September 3, 2015.