Before You Begin

• Read the Cloudera Manager Release Notes.
• Make sure there are no Oozie workflows in RUNNING or SUSPENDED status; otherwise the Oozie database upgrade will fail and you will have to reinstall CDH 4 to complete or kill those running workflows.
• Plan downtime. If you are upgrading a cluster that is part of a production system, be sure to plan ahead. As with any operational work, be sure to reserve a maintenance window with enough extra time allotted in case of complications. The Hadoop upgrade process is well understood, but it is best to be cautious. For production clusters, Cloudera recommends allocating up to a full day maintenance window to perform the upgrade, depending on the number of hosts, the amount of experience you have with Hadoop and Linux, and the particular hardware you are using.
• To avoid generating many alerts during the upgrade process, you can enable maintenance mode on your cluster before you start the upgrade. Be sure to exit maintenance mode when you have finished the upgrade, in order to re-enable Cloudera Manager alerts.
• Put the NameNode into safe mode. To upgrade CDH in multiple clusters, repeat this process for each cluster:
  1. In the Cloudera Manager Admin Console, go the HDFS service, NameNode role instance.
  2. Select Actions > Enter Safemode... and confirm that you want to do this.
  3. After the NameNode has successfully entered safemode, select Actions > Save Namespace... and confirm that you want to do this. This will result in a new fsimage being written out with no edit log entries. Leave the NameNode in safe mode while you proceed with the upgrade instructions.
• Ensure Java 7 is installed across the cluster. CDH 5 requires Java 7, and some services may not start if it is not installed. For installation instructions and recommendations for CDH 5, see (CDH 5) Java Development Kit Installation.
• Back up important databases:
  • Cloudera Manager databases. For instructions, see Backing up Databases. You will need to indicate to the upgrade wizard that you have performed this step before the upgrade will proceed.
  • Hive Metastore database (which could be in the embedded database)
  • Hue database
  • Oozie database
  • Sqoop database
• If you have just upgraded to Cloudera Manager 5, you must hard restart the Cloudera Manager Agents as described in the Hard Restart Cloudera Manager Agents task in Upgrading Cloudera Manager 4 to Cloudera Manager 5 in Cloudera Manager Administration Guide.

Stop All Services

1. Stop each cluster.
   1. On the Home page, click to the right of the cluster name and select Stop.
   2. Click Stop in the confirmation screen. The Command Details window shows the progress of stopping services.

      When All services successfully stopped appears, the task is complete and you can close the Command Details window.

2. Stop the Cloudera Management Service:
   1. Do one of the following:
      • 1. Select Clusters > Cloudera Management Service > mgmt.
        2. Select Actions > Stop.
      • 1. On the Home page, click to the right of mgmt and select Stop.
   2. Click Stop to confirm. The Command Details window shows the progress of stopping the roles.
   3. When Command completed with n/n successful subcommands appears, the task is complete. Click Close.

Perform Service-Specific Prerequisite Actions

• Accumulo - if you have installed the Accumulo parcel, deactivate it following the instructions in Managing Parcels.
• HDFS - Back up HDFS metadata on the NameNode:
  1. Stop the NameNode you want to back up.
  2. Go to the HDFS service.
  3. Select Configuration > View and Edit.
  4. In the Search field, search for "NameNode Data Directories". This locates the NameNode Data Directories property.
  5. From the command line on the NameNode host, back up the directory listed in the NameNode Data Directories property. If more than one is listed, then you only need to make a backup of one directory, since each directory is a complete copy. For example, if the data directory is /mnt/hadoop/hdfs/name, do the following as root:

     # cd /mnt/hadoop/hdfs/name
     # tar -cvf /root/nn_backup_data.tar .

     You should see output like this:

     ./
     ./current/
     ./current/fsimage
     ./current/fstime
     ./current/VERSION
     ./current/edits
     ./image/
     ./image/fsimage

       Warning: If you see a file containing the word lock, the NameNode is probably still running. Repeat the preceding steps, starting by shutting down the CDH services.

Run the Upgrade Wizard

Import MapReduce Configuration to YARN

In CDH 5 and Cloudera Manager 5, YARN rather than MapReduce is the default MapReduce computation framework. If you had the MapReduce service configured in CDH 4, you can import the MapReduce configuration to YARN. This does not affect your MapReduce configuration.

  Warning: In addition to importing configuration settings, the import process:

• Configures services to use YARN as the MapReduce computation framework instead of MapReduce.
• Overwrites existing YARN configuration and role assignments.

1. To import the existing configuration from your MapReduce service, select OK, set up YARN to add the YARN service and import the MapReduce settings. To skip the import, select Skip this step now. If you choose to skip this step, you can perform it at a later time from the YARN service.
2. Click Continue to proceed. Cloudera Manager stops the YARN service (if running) and its dependencies. When these commands complete, click Continue.
3. The next page indicates some additional configuration required by YARN. Verify or modify these and click Continue.
4. The Switch Cluster to MR2 step proceeds. When all steps have been completed, click Continue.
5. When all steps have complete, click Continue.

Remove CDH Packages and Update Symlinks

Restart the Reports Manager Role

1. Do one of the following:
   • Select Clusters > Cloudera Management Service > mgmt.
   • On the Status tab of the Home page, in Cloudera Management Service table, click the mgmt link.
2. Click the Instances tab.
3. Check the checkbox next to reportsmanager.
4. Select Actions for Selected > Restart and then Restart to confirm.

Finalize the HDFS Metadata Upgrade