Upgrading CDH 5 Using Packages
If you originally used Cloudera Manager to install your CDH service using packages, you can upgrade to a new version of CDH 5 using either packages or parcels. Using parcels is recommended, because the upgrade wizard for parcels handles the upgrade almost completely automatically. But if you want to continue to use packages, you can perform an upgrade following the instructions presented here.
To upgrade CDH using packages, the steps are as follows.
- Upgrade Unmanaged Components
- Stop All Services
- Back up the HDFS Metadata on the NameNode
- Upgrade Managed Components
- Using the "1-Click" Package
- Update Symlinks for the Newly Installed Components
- Upgrade Sqoop
- Upgrade the Oozie Database
- Start All Services
- Configure Cluster CDH Version for Package Installs
- Deploy Client Configurations
Upgrade Unmanaged Components
- Pig
- Whirr
- Mahout
For information on upgrading these unmanaged components, see the CDH 5 Installation Guide.
Stop All Services
- Stop each cluster.
- On the Home page, click to the right of the cluster name and select Stop.
- Click Stop in the confirmation screen. The Command Details window shows the progress of stopping services.
When All services successfully stopped appears, the task is complete and you can close the Command Details window.
- Stop the Cloudera Management Service:
- Do one of the following:
-
- Select .
- Select .
-
- On the Home page, click to the right of mgmt and select Stop.
-
- Click Stop to confirm. The Command Details window shows the progress of stopping the roles.
- When Command completed with n/n successful subcommands appears, the task is complete. Click Close.
- Do one of the following:
Back up the HDFS Metadata on the NameNode
- Stop the NameNode you want to back up.
- Go to the HDFS service.
- Select .
- In the Search field, search for "NameNode Data Directories". This locates the NameNode Data Directories property.
- From the command line on the NameNode host, back up the directory listed
in the NameNode Data Directories property. If more than one is listed, then you
only need to make a backup of one directory, since each directory is a complete
copy. For example, if the data directory is /mnt/hadoop/hdfs/name, do the
following as root:
# cd /mnt/hadoop/hdfs/name # tar -cvf /root/nn_backup_data.tar .
You should see output like this:
./ ./current/ ./current/fsimage ./current/fstime ./current/VERSION ./current/edits ./image/ ./image/fsimage
Warning: If you see a file containing the word lock, the NameNode is probably still running. Repeat the preceding steps, starting by shutting down the CDH services.
Upgrade Managed Components
Use one of the following strategies to upgrade CDH 5:- Use your operating system's package management tools to update all packages to the latest version using standard repositories. This approach works well because it minimizes the amount of configuration required and uses the simplest commands. Be aware that this can take a considerable amount of time if you have not upgraded the system recently. To update all packages on your system, use the following command:
Operating System Command RHEL $ sudo yum update
SLES $ sudo zypper up
Ubuntu or Debian $ sudo apt-get upgrade
- Use Cloudera's "1-click Install" package. This is the simplest way to upgrade only the Cloudera packages. Follow the instructions in Using the "1-Click" Package.
Using the "1-Click" Package
Follow the instructions below if you have decided to use the "1-click" package.
- Check whether you have the CDH 5 "1-click" repository installed.
- Red Hat/CentOS-compatible and SLES
rpm -q CDH 5-repository
If you are upgrading from CDH 5 Beta 1 or later, and you used the "1-click" package for the previous CDH 5 release, you should see:
CDH5-repository-1-0
In this case, skip to installing the CDH 5 packages. If instead you see:
package CDH 5-repository is not installed
proceed with installing the 1-click package.
- Ubuntu and Debian
dpkg -l | grep CDH 5-repository
If the repository is installed, skip to installing the CDH 5 packages; otherwise proceed with installing the "1-click" package.
- Red Hat/CentOS-compatible and SLES
- If the CDH 5 "1-click" repository is not already installed on each host in the cluster, follow the instructions below for that host's operating system.
- Red Hat compatible
- Download and install the "1-click Install" package.
- Download the CDH 5 "1-click Install" package.
Click the entry in the table below that matches your Red Hat or CentOS system, choose Save File, and save the file to a directory to which you have write access (it can be your home directory).
OS Version Click this Link Red Hat/CentOS/Oracle 5 Red Hat/CentOS/Oracle 5 link Red Hat/CentOS/Oracle 6 Red Hat/CentOS/Oracle 6 link - Install the RPM:
- Red Hat/CentOS/Oracle 5
$ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm
- Red Hat/CentOS/Oracle 6
$ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm
- Red Hat/CentOS/Oracle 5
- Download the CDH 5 "1-click Install" package.
- (Optionally) add a repository key:
- Red Hat/CentOS/Oracle 5
$ sudo rpm --import http://archive.cloudera.com/cdh5/redhat/5/x86_64/cdh/RPM-GPG-KEY-cloudera
- Red Hat/CentOS/Oracle 6
$ sudo rpm --import http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera
- Red Hat/CentOS/Oracle 5
- Download and install the "1-click Install" package.
- SLES
- Download and install the "1-click Install" package:
- Download the CDH 5 "1-click Install" package.
Click this link, choose Save File, and save it to a directory to which you have write access (it can be your home directory).
- Install the RPM:
$ sudo rpm -i cloudera-cdh-5-0.x86_64.rpm
- Update your system package index by running:
$ sudo zypper refresh
- Download the CDH 5 "1-click Install" package.
- (Optionally) add a repository key:
$ sudo rpm --import http://archive.cloudera.com/cdh5/sles/11/x86_64/cdh/RPM-GPG-KEY-cloudera
- Download and install the "1-click Install" package:
- Ubuntu and Debian
- Download and install the "1-click Install" package:
- Download the CDH 5 "1-click Install" package:
OS Version Click this Link Wheezy Wheezy link Precise Precise link - Install the package. Do one of the following:
- Choose Open with in the download window to use the package manager.
- Choose Save File, save the package to a directory to which you have write access (it can be your home directory) and install it from the command line, for example:
sudo dpkg -i cdh5-repository_1.0_all.deb
- Download the CDH 5 "1-click Install" package:
- (Optionally) add a repository key:
- Ubuntu Lucid
$ curl -s http://archive.cloudera.com/cdh5/ubuntu/lucid/amd64/cdh/archive.key | sudo apt-key add -
- Ubuntu Precise
$ curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -
- Debian Wheezy
$ curl -s http://archive.cloudera.com/cdh5/debian/wheezy/amd64/cdh/archive.key | sudo apt-key add -
- Ubuntu Lucid
- Download and install the "1-click Install" package:
- Red Hat compatible
- Install the CDH packages:
- Red Hat compatible
$ sudo yum clean all $ sudo yum install avro-tools crunch flume-ng hadoop-hdfs-fuse hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-pig hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell kite llama mahout oozie pig pig-udf-datafu search sentry solr-mapreduce spark-python sqoop sqoop2 whirr
- SLES
$ sudo zypper clean --all $ sudo zypper install avro-tools crunch flume-ng hadoop-hdfs-fuse hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-pig hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell kite llama mahout oozie pig pig-udf-datafu search sentry solr-mapreduce spark-python sqoop sqoop2 whirr
- Ubuntu and Debian
$ sudo apt-get update $ sudo apt-get install avro-tools crunch flume-ng hadoop-hdfs-fuse hadoop-httpfs hbase-solr hive-hbase hive-webhcat hue-beeswax hue-hbase hue-impala hue-pig hue-plugins hue-rdbms hue-search hue-spark hue-sqoop hue-zookeeper impala impala-shell kite llama mahout oozie pig pig-udf-datafu search sentry solr-mapreduce spark-python sqoop sqoop2 whirr
Note: Installing these packages will also install all the other CDH packages that are needed for a full CDH 5 installation. - Red Hat compatible
Update Symlinks for the Newly Installed Components
$ sudo service cloudera-scm-agent restart
Upgrade Sqoop
- Go to the Sqoop service.
- Select Upgrade Sqoop to confirm. and click
- When the command completes, click Close.
Upgrade the Oozie Database
- Go to the Sqoop service.
- Select Upgrade Database to confirm. and click
- When the command completes, click Close.
Start All Services
- Start each cluster.
- Start the Cloudera Management Service.
When the command completes, click Close.
Configure Cluster CDH Version for Package Installs
Because Cloudera Manager does not manage service software installed as packages, during certain upgrade scenarios Cloudera Manager assigns a default CDH version of a cluster. You must manually configure the cluster CDH version to match the package CDH version following the procedure in Configuring the CDH Version for a Cluster in Managing Clusters with Cloudera Manager. If you do not set the cluster CDH version to the package CDH version, Cloudera Manager will incorrectly enable and disable service features based on the configured CDH version.
Deploy Client Configurations
- From the top Actions button that corresponds to the cluster and choose Deploy Client Configuration....
- Click the Deploy Client Configuration button in the confirmation pop-up that appears.
<< Upgrading CDH 5 Using Parcels | Upgrading to CDH 5 >> | |