Stop All Services

1. Stop each cluster.
   1. On the Home page, click to the right of the cluster name and select Stop.
   2. Click Stop in the confirmation screen. The Command Details window shows the progress of stopping services.

      When All services successfully stopped appears, the task is complete and you can close the Command Details window.

2. Stop the Cloudera Management Service:
   1. Do one of the following:
      • 1. Select Clusters > Cloudera Management Service > mgmt.
        2. Select Actions > Stop.
      • 1. On the Home page, click to the right of mgmt and select Stop.
   2. Click Stop to confirm. The Command Details window shows the progress of stopping the roles.
   3. When Command completed with n/n successful subcommands appears, the task is complete. Click Close.

Back up the HDFS Metadata on the NameNode

1. Stop the NameNode you want to back up.
2. Go to the HDFS service.
3. Select Configuration > View and Edit.
4. In the Search field, search for "NameNode Data Directories". This locates the NameNode Data Directories property.
5. From the command line on the NameNode host, back up the directory listed in the NameNode Data Directories property. If more than one is listed, then you only need to make a backup of one directory, since each directory is a complete copy. For example, if the data directory is /mnt/hadoop/hdfs/name, do the following as root:

   # cd /mnt/hadoop/hdfs/name
   # tar -cvf /root/nn_backup_data.tar .

   You should see output like this:

   ./
   ./current/
   ./current/fsimage
   ./current/fstime
   ./current/VERSION
   ./current/edits
   ./image/
   ./image/fsimage

     Warning: If you see a file containing the word lock, the NameNode is probably still running. Repeat the preceding steps, starting by shutting down the CDH services.

Download, Distribute, and Activate CDH 4, Impala, and Solr Parcels

1. In the Cloudera Manager Admin Console, click the Parcels indicator in the top navigation bar ( or ) to go to the Parcels page.
2. In the parcels page, click Download for the version(s) you want to download. If you want to run both CDH, Cloudera Impala, and Cloudera Search you should download the CDH, Impala, and Solr parcels. If the parcel you want is not shown here — for example, you want to upgrade to version of CDH that is not the most current version — you can make additional parcel repos available through the parcel settings page. You can find the locations of the previous CDH 4 parcels at http://archive.cloudera.com/cdh4/parcels/. If you are upgrading to CDH 4.3 and want to use Sentry, you can find the separate Sentry parcel at http://archive.cloudera.com/sentry/parcels/latest/). Sentry is included with CDH 4.4 or later parcels. If your Cloudera Manager server does not have Internet access, you can obtain the required parcel file(s) and put them into the local repository. See Creating and Using a Parcel Repository for more details.
3. When the download has completed, click Distribute for the version you downloaded.
4. When the parcel has been distributed and unpacked, the button will change to say Activate.
5. Click Activate. A pop-up that offers to restart your services displays. If you are upgrading between major Hive versions (that is, from CDH 4.0 to 4.1 or 4.2, or from CDH 4.1 to 4.2) do not restart the services – you must perform the next step to upgrade your Hive metastore before you restart Hive. Click Close to remove the pop-up.

Upgrade the Hive Metastore

If you are upgrading from CDH 4.2 to CDH 4.3 or later, you do not need to perform this step. If you are upgrading from an earlier version of CDH to CDH 4.2 or later, you must do this.

1. (Strongly recommended) Make a backup copy of your Hive metastore database.
2. Run the metastore upgrade script. The script you run depends on whether you are upgrading to parcels or packages.
   • If you are upgrading to packages, the upgrade script is at /usr/lib/hive/scripts/metastore/upgrade/
   • If you are upgrading to parcels, then the upgrade script is located at /opt/cloudera/parcels/parcel_name/lib/hive/scripts/metastore/upgrade/database, where parcel_name should be the name of the parcel to which you have upgraded and database is the type of database you are running (that is, mysql, postgres, and so on) For example, if you are installing a CDH 4.2.0 parcel using the default location for the local repository, and using the default database (PostgreSQL) the script will be at: /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10-e16.parcel/lib/hive/scripts/metastore/upgrade/postgres
   • You must cd to the directory the scripts are in.
   • Execute the script in the appropriate DB command shell. There are multiple scripts in each directory. You must run the one that corresponds to the versions of Hive you are upgrading between. For example, if you are upgrading with MySQL from Hive 0.9 to 0.10, the command would be similar to:

     mysql -u hive1 -phive1 hive1 < upgrade-0.9.0-to-0.10.0.mysql.sql

     (with the appropriate substitutions for username, etc.).

     If your upgrade spans multiple versions of Hive (for example, upgrading from Hive 0.8 to Hive 0.10) you must run all the relevant scripts in the proper order.

     Important: You must know the password for the Hive metastore database; if you installed Cloudera Manager using the default (embedded PostgreSQL) database, the password was displayed on the Database Setup page during the Cloudera Manager installation wizard. If you do not know the password for your Hive metastore database, you can find it as follows:

   • cat /etc/cloudera-scm-server/db.properties This shows you Cloudera Manager's internal database credentials.
   • Run the following command:

     psql -p 7432 -U cm cm -c "select s.display_name as hive_service_name, s.name as  hive_internal_name, c.value as metastore_password from CONFIGS c, SERVICES s where attr='hive_metastore_database_password' and  c.service_id = s.service_id"

   • Use the password from com.cloudera.cmf.db.password. This will output the passwords for the hive service metastore as follows:

      hive_service_name | hive_internal_name | metastore_password
     -------------------+--------------------+--------------------
      hive1             | hive1              | lF3Cv2zsvI
     (1 row)

3. If you have multiple instances of Hive, run the upgrade script(s) on each metastore database.

(If Upgrading to CDH 4.2) Upgrade the Oozie Sharelib

1. In the Cloudera Manager Admin Console, select Oozie from the Services tab.
2. From the Actions button, choose Stop.
3. When the service has stopped, from the Actions button choose Install Oozie Sharelib. The commands to perform this function are run.

Upgrade Sqoop

1. In the Cloudera Manager Admin Console, go to the Sqoop service.
2. Select Configuration > View and Edit.
3. Under the Sqoop Server Default Group role group, Advanced category, add the following to the Sqoop Server Configuration Advanced Configuration Snippet for sqoop.properties (and Save Changes):

   org.apache.sqoop.repository.schema.immutable=false
   org.apache.sqoop.connector.autoupgrade=true
   org.apache.sqoop.framework.autoupgrade=true
   

   These properties enable the upgrading of the repository, connectors, and framework structures.

4. Restart the Sqoop service. Let it update the database.
5. Select the Status tab and in the Summary section click the Log File link to verify that the upgrade has been done successfully.
6. From the Sqoop service's Action menu, Stop the service.
7. From the Configuration tab, remove the properties you added to the Sqoop Server Configuration Advanced Configuration Snippet for sqoop.properties and Save Changes again.
8. Restart the Sqoop service.

Start the Services

1. On the Home page, click to the right of the cluster name and select Start.
2. Click the Start button in the confirmation pop-up that appears. The Command Details window shows the progress of starting services.

Deploy the New Client Configuration Files

1. On the Home page, click to the right of the cluster name and select Deploy Client Configuration.
2. Click the Deploy Client Configuration button in the confirmation pop-up that appears.

Remove the Previous CDH Version Packages

If your previous installation of CDH 4 (4.0.x or 4.1.x) was done using packages, you must remove those packages on all hosts on which you installed the parcels and refresh the symlinks so that clients will run the new software versions. Skip this step if your previous installation was using parcels.

1. If Hue is configured to use SQLite as its database:
   1. Stop the Hue service.
   2. Back up the desktop.db to a temporary location before deleting the old Hue Common package. The location of the database can be found in the Hue service Configuration tab under Service > Database > Hue's Database Directory.
     Important: Removing the Hue Common package will remove your Hue database; if you do not back it up you may lose all your Hue user account information.
2. Uninstall the CDH packages on each host:
   • Not including Impala and Search

     Operating System	Command
     RHEL	$ sudo yum remove hadoop hue-common bigtop-jsvc bigtop-tomcat
     SLES	$ sudo zypper remove hadoop hue-common bigtop-jsvc bigtop-tomcat
     Ubuntu or Debian	$ sudo apt-get purge hadoop hue-common bigtop-jsvc bigtop-tomcat

   • Including Impala and Search

     Operating System	Command
     RHEL	$ sudo yum remove hadoop hue-common impala-shell solr-server 'bigtop-*'
     SLES	$ sudo zypper remove hadoop hue-common impala-shell solr-server 'bigtop-*'
     Ubuntu or Debian	$ sudo apt-get purge hadoop hue-common impala-shell solr-server 'bigtop-*'

Update Symlinks for the Newly Installed Components

$ sudo service cloudera-scm-agent restart

Restore Backed up Hue Database

1. In the Cloudera Manager Admin Console, select the Hue service.
2. From the Actions button, choose Stop.
3. When the Hue service has stopped, restore the back up you created in Remove the Previous CDH Version Packages to the newly created Hue database directory: /opt/cloudera/parcels/CDH-4.x.0-x.cdh4.x.0.p0.xx/share/hue/desktop.
4. Restart the Hue service.