Upgrading Cloudera Manager 5 Using Tarballs

Minimum Required Role: Full Administrator

This topic describes how to upgrade Cloudera Manager 5.x using tarballs. Tarballs contain both the Cloudera Manager Server and Cloudera Manager Agent in a single file.

In most cases it is possible to upgrade Cloudera Manager without shutting down most CDH services, although you may need to stop some dependent services. CDH daemons can continue running, unaffected, while Cloudera Manager is upgraded. The upgrade process does not affect your CDH installation. After upgrading Cloudera Manager you may also want to upgrade CDH 4 clusters to CDH 5, or upgrade to a more recent minor version of CDH.

Steps to upgrade Cloudera Manager 5 using tarballs:
Step Description Link
1 Collect the information you will need to upgrade Cloudera Manager. This includes user accounts, passwords, database URLs and other items. You must gather this information before beginning the upgrade because some of the information is only available from the Cloudera Manager Admin Console, which will not be available during the upgrade process. Step 1: Collect Upgrade Information
2 Complete the pre-upgrade steps and review some special warnings about upgrades. Step 2: Complete Pre-Upgrade Steps
3 If your Cloudera Manager hosts are using an unsupported version of the JDK, you must upgrade to a supported version of the JDK before upgrading Cloudera Manager. If you also plan to upgrade CDH, you must also upgrade the JDK on all cluster hosts. Step 4:

Java Development Kit Installation

Upgrading to Oracle JDK 1.8

4

If the Cloudera Manager host does not have access to the internet, or you wish to install a version other than the latest version of Cloudera Manager, you configure access to the Cloudera Manager software, either from the Cloudera public repository or a local package repository that you create.

Creating and Using a Package Repository for Cloudera Manager
5

If you are upgrading from Cloudera Navigator 2.6 or lower, you must upgrade the Cloudera Navigator data management component. Cloudera Navigator 2.6 is supported on Cloudera Manager 5.7.x and higher.

Upgrading the Component
6 Upgrade the Cloudera Manager server and agent software. Upgrading the Cloudera Manager Server and Agents
7 Verify and test your upgrade. Step 8: Verify and Test the Upgrade
8 Upgrade any required Cloudera Navigator components:
  • Cloudera Manager Key Trustee Server
  • Cloudera Navigator Key HSM
  • Cloudera Navigator Key Trustee KMS
  • Cloudera Navigator Encrypt.

The Cloudera Navigator Data Management Component is upgraded automatically when you upgrade Cloudera Manager.

Upgrading Cloudera Navigator Components
9 (Optional) Upgrade CDH. You are not required to upgrade CDH after upgrading Cloudera Manager. You can upgrade CDH at a later time. Step 10: (Optional) Upgrade CDH

Step 1: Collect Upgrade Information

Before starting an upgrade of Cloudera Manager, collect the following information:
  • Host credentials. You must have SSH access and be able to log in using a root account or an account that has password-less sudo permission.
  • The version of Cloudera Manager used in your cluster. Go to Support > About.
  • The version of CDH. The CDH version number displays next to the cluster name on the Home page.
  • The services enabled in your cluster. Go to Clusters > Cluster name.
  • Whether the cluster was installed using parcels or packages. This information displays next to the CDH version on the Home page of Cloudera Manager.
  • The version of the JDK deployed in the cluster. Go to Support > About.
  • Whether or not you have enabled Cloudera Navigator auditing.
  • For all databases used in your cluster:
    • Type of database (PostgreSQL, Embedded PostgreSQL, MySQL, MariaDB, or Oracle)
    • Hostnames of the databases
    • Credentials for the databases
    To locate database information:
    • Sqoop, Oozie, and Hue – Go to Cluster Name > Configuration > Database Settings.

    • Hive Metastore – Go to the Hive service, select Configuration, and select the Hive Metastore Database category.
    • Sentry – Go to the Sentry service, select Configuration, and select the Sentry Server Database category.
  • Operating system version.
  • Whether the cluster uses AES-256 encryption.

Step 2: Complete Pre-Upgrade Steps

Before beginning a Cloudera Manager upgrade, do the following:

  1. Review the system requirements for the new versions you are upgrading to.

    See CDH 5 and Cloudera Manager 5 Requirements and Supported Versions.

  2. If you are upgrading from Cloudera Manager 5.4.x to Cloudera Manager 5.5 or higher, perform the following steps required for the associated Cloudera Navigator upgrade:
    1. Stop the Navigator Metadata Server role.
    2. Back up the Navigator Metadata Server storage directory.
    3. Make sure that the Navigator Metadata Server has sufficient memory to complete the upgrade.
    4. If you are using an Oracle database, in SQL*Plus, ensure that the following additional privileges are set:
      GRANT EXECUTE ON sys.dbms_crypto TO nav;
      GRANT CREATE VIEW TO nav;
      where nav is the user of the Navigator Audit Server database.
    For further information, see Upgrading the Cloudera Navigator Data Management Component.
  3. Note the following:
    • Cloudera Management Service TLS/SSL configuration

      If you have enabled TLS security for the Cloudera Manager Admin Console, as of Cloudera Manager 5.1, Cloudera Management Service roles try to communicate with Cloudera Manager using TLS, and fail to start until TLS/SSL properties have been configured.

    • Navigator

      If you have enabled auditing with Cloudera Navigator, during the upgrade to Cloudera Manager 5, auditing is suspended and is only restarted when you restart the roles of audited services. You will be instructed to stop some services in a later step.

  4. If you have previously installed Kafka 1.2, and are upgrading from Cloudera Manager 5.4 or lower, remove the Kafka CSD:
    1. Determine the location of the CSD directory:
      1. Select Administration > Settings.
      2. Click the Custom Service Descriptors category.
      3. Retrieve the directory from the Local Descriptor Repository Path property.
    2. Delete the Kafka CSD from the directory.
  5. Review package (RPM) dependencies. A Cloudera Manager upgrade may introduce new package dependencies. If your organization has restrictions or requires prior approval for installation of packages, see the list of Package Dependencies before upgrading Cloudera Manager.

Upgrading the Cloudera Manager Server and Agents

  1. If your cluster is running the embedded PostgreSQL database, stop all services that are using the embedded database. These may include:
    • Hive service and all services such as Impala and Hue that use the Hive metastore
    • Oozie
    • Sentry
  2. Stop Cloudera Manager Server, Database, and Agent:
    1. Use the Cloudera Manager Admin Console to stop any running commands. These include commands a user runs and commands Cloudera Manager automatically triggers in response to a state change or a schedule. You can either wait for commands to complete or abort any running commands. For more information on viewing and aborting running commands, see Viewing Running and Recent Commands.
    2. On the host running the Cloudera Manager Server, stop the Cloudera Manager Server:
      sudo service cloudera-scm-server stop
    3. On the host where the database runs, (usually the Cloudera Manager Server host) stop the database:
      sudo service cloudera-scm-server-db stop
    4. If the Cloudera Manager host is also running the Cloudera Manager Agent, stop the Cloudera Manager Agent:
      sudo service cloudera-scm-agent stop
  3. If you are using JDK 1.6, upgrade the JDK on Cloudera Manager Server JDK 1.7 or 1.8. After you install JDK 7, Cloudera Manager can install JDK 7 on all of the cluster hosts. To upgrade to JDK 1.8, you must manually install the JDK on all cluster hosts. Follow the steps in Java Development Kit Installation, and then return here to continue the upgrade.
  4. Download tarballs from the locations listed in Cloudera Manager Version and Download Information.
  5. Copy the tarballs and unpack them on all hosts on which you intend to install Cloudera Manager Server and Cloudera Manager Agents, in a directory of your choosing. If necessary, create a new directory to accommodate the files you extract from the tarball. For instance, if /opt/cloudera-manager does not exist, create it using a command similar to:
    $ sudo mkdir /opt/cloudera-manager
  6. Extract the contents of the tarball to this directory. For example, to copy a tar file to your home directory and extract the contents of all tar files to the /opt/ directory, use a command similar to the following:
    $ sudo tar xzf cloudera-manager*.tar.gz -C /opt/cloudera-manager

    The files are extracted to a subdirectory named according to the Cloudera Manager version being extracted. For example, files could be extracted to /opt/cloudera-manager/cm-5.0/. This full path is needed later and is referred to as tarball_root directory.

  7. On every Cloudera Manager Agent host, configure the Cloudera Manager Agent to point to the Cloudera Manager Server by setting the following properties in the tarball_root/etc/cloudera-scm-agent/config.ini configuration file:
    Property Description
    server_host Name of the host where Cloudera Manager Server is running.
    server_port Port on the host where Cloudera Manager Server is running.
  8. By default, a tarball installation has a var subdirectory where state is stored. In a non-tarball installation, state is stored in /var. Cloudera recommends that you configure the tarball installation to use an external directory as the /var equivalent (/var or any other directory outside the tarball) so that when you upgrade Cloudera Manager, the new tarball installation can access this state. Configure the installation to use an external directory for storing state by editing tarball_root/etc/default/cloudera-scm-agent and setting the CMF_VAR variable to the location of the /var equivalent. If you do not reuse the state directory between different tarball installations, duplicate Cloudera Manager Agent entries can occur in the Cloudera Manager database.
  9. Start Cloudera Manager Server.
    The way in which you start the Cloudera Manager Server varies depending on which account you want the Server to run under:
    • As root:
      sudo tarball_root/etc/init.d/cloudera-scm-server start 
    • As another user. If you run as another user, ensure the user you created for Cloudera Manager owns the location to which you extracted the tarball including the newly created database files. If you followed the earlier examples and created the directory /opt/cloudera-manager and the user cloudera-scm, you could use the following command to change ownership of the directory:
      sudo chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager

      Once you have established ownership of directory locations, you can start Cloudera Manager Server using the user account you chose. For example, you might run the Cloudera Manager Server as cloudera-service. In this case, you have the following options:

      • Run the following command:
        $ sudo -u cloudera-service tarball_root/etc/init.d/cloudera-scm-server start 
      • Edit the configuration files so the script internally changes the user. Then run the script as root:
        1. Remove the following line from tarball_root/etc/default/cloudera-scm-server:
          export CMF_SUDO_CMD=" "
        2. Change the user and group in tarball_root/etc/init.d/cloudera-scm-server to the user you want the server to run as. For example, to run as cloudera-service, change the user and group as follows:
          USER=cloudera-service
          GROUP=cloudera-service
        3. Run the server script as root:
          $ sudo tarball_root/etc/init.d/cloudera-scm-server start 
    • To start the Cloudera Manager Server automatically after a reboot:
      1. Run the following commands on the Cloudera Manager Server host:
        • RHEL-compatible and SLES
          $ cp tarball_root/etc/init.d/cloudera-scm-server /etc/init.d/cloudera-scm-server
          $ chkconfig cloudera-scm-server on
        • Debian/Ubuntu
          $ cp tarball_root/etc/init.d/cloudera-scm-server /etc/init.d/cloudera-scm-server
          $ update-rc.d cloudera-scm-server defaults
      2. On the Cloudera Manager Server host, open the /etc/init.d/cloudera-scm-server file and change the value of CMF_DEFAULTS from ${CMF_DEFAULTS:-/etc/default} to tarball_root/etc/default.
  10. To stop the Cloudera Manager Agent, run this command on each Agent host:
    $ sudo tarball_root/etc/init.d/cloudera-scm-agent hard_stop_confirmed
    If you are running single user mode, stop Cloudera Manager Agent using the user account you chose. For example, if you are running the Cloudera Manager Agent as cloudera-scm, you have the following options:
    • Run the following command:
      $ sudo -u cloudera-scm tarball_root/etc/init.d/cloudera-scm-agent hard_stop_confirmed 
    • Edit the configuration files so the script internally changes the user, and then run the script as root:
      1. Remove the following line from tarball_root/etc/default/cloudera-scm-agent:
        export CMF_SUDO_CMD=" "
      2. Change the user and group in tarball_root/etc/init.d/cloudera-scm-agent to the user you want the Agent to run as. For example, to run as cloudera-scm, change the user and group as follows:
        USER=cloudera-scm
        GROUP=cloudera-scm
      3. Run the Agent script as root:
        $ sudo tarball_root/etc/init.d/cloudera-scm-agent hard_stop_confirmed 
  11. Start the Cloudera Manager Agent according to the account you want the Agent to run under:
    • To start the Cloudera Manager Agent, run this command on each Agent host:
      $ sudo tarball_root/etc/init.d/cloudera-scm-agent start
      When the Agent starts, it contacts the Cloudera Manager Server.
    • If you are running single user mode, start Cloudera Manager Agent using the user account you chose. For example, to run the Cloudera Manager Agent as cloudera-scm, you have the following options:
      • Run the following command:
        $ sudo -u cloudera-scm tarball_root/etc/init.d/cloudera-scm-agent start 
      • Edit the configuration files so the script internally changes the user, and then run the script as root:
        1. Remove the following line from tarball_root/etc/default/cloudera-scm-agent:
          export CMF_SUDO_CMD=" "
        2. Change the user and group in tarball_root/etc/init.d/cloudera-scm-agent to the user you want the Agent to run as. For example, to run as cloudera-scm, change the user and group as follows:
          USER=cloudera-scm
          GROUP=cloudera-scm
        3. Run the Agent script as root:
          $ sudo tarball_root/etc/init.d/cloudera-scm-agent start 
    • To start the Cloudera Manager Agents automatically after a reboot:
      1. Run the following commands on each Agent host:
        • RHEL-compatible and SLES
          $ cp tarball_root/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent
          $ chkconfig cloudera-scm-agent on
        • Debian/Ubuntu
          $ cp tarball_root/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent
          $ update-rc.d cloudera-scm-agent defaults
      2. On each Agent, open the tarball_root/etc/init.d/cloudera-scm-agent file and change the value of CMF_DEFAULTS from ${CMF_DEFAULTS:-/etc/default} to tarball_root/etc/default.
  12. Log in to the Cloudera Manager Admin Console.
  13. Restart all services:
    1. From the Home > Status tab click next to the cluster name and select Restart.
    2. In the confirmation dialog box that displays, click Restart.
  14. Upgrade any required Cloudera Navigator components:
  15. (Optional) Upgrade CDH.

    See Step 10: (Optional) Upgrade CDH.

Step 8: Verify and Test the Upgrade

If the commands to update and start the Cloudera Manager Server complete without errors, you can assume the upgrade has completed successfully. To verify, you can check that the server versions have been updated.
  1. In the Cloudera Manager Admin Console, click the Hosts tab.
  2. Click Host Inspector. On large clusters, the host inspector may take some time to finish running. You must wait for the process to complete before proceeding to the next step.
  3. Click Show Inspector Results. All results from the host inspector process are displayed, including the currently installed versions. If this includes listings of current component versions, the installation completed as expected.
  4. Verify that the monitoring features are working as expected; follow the instructions in Testing the Installation.

Step 10: (Optional) Upgrade CDH

For more information on upgrading CDH, see Upgrading CDH and Managed Services Using Cloudera Manager.