Upgrading the Cloudera Manager Agents

Minimum Required Role: Cluster Administrator (also provided by Full Administrator) This feature is not available when using Cloudera Manager to manage Data Hub clusters.

Loading Filters ... 7.1.3 7.1.2 7.1.1 7.0.3 6.3.3 6.3.1 6.3.0 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0 6.3.3 6.3.2 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0 7.1.3 7.1.2 7.1.1 7.0.3

Upgrade the Cloudera Manager Agents (Cloudera Manager 6 and lower)

You can upgrade the agents using one of the two options below.

Option 1. Upgrade the Agents using Cloudera Manager (Recommended)

After the Cloudera Manager Server starts and you log in to the Cloudera Manager Admin Console, the Upgrade Cloudera Manager page displays. (It might take several minutes for the server to start.)

If the Upgrade Cloudera Manager page does not display after you upgraded the packages on the Cloudera Manager Server host, open the following URL in your web browser:

https://my_cloudera_manager_server_host:port/cmf/upgrade
https://my_cloudera_manager_server_host:port/cmf/upgrade-wizard/welcome

The status of Agent upgrades is displayed in one or more groups. Because you might have only upgraded the Cloudera Manager Agent on the Cloudera Manager Server host, the first group shows that host as having an upgraded agent. If the hosts managed by Cloudera Manager have different operating systems, a group for each operating system displays the Agent upgrade status for those hosts.

Follow the instructions on the upgrade page to upgrade all the agents.

  1. Start the Cloudera Manager Management Service.
    1. Log in to the Cloudera Manager Admin Console.
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Start.
  2. Click Yes, I would like to upgrade Cloudera Manager Agent packages now.
  3. Click Continue.

    The Upgrade Cloudera Manager Agent Packages page displays.

  4. If there are more than one group of hosts that require agent upgrades, select the group from the drop-down list labeled Upgrade Cloudera Manager Agent Packages running on:. If there is only one group that requires upgrades, this drop-down list does not appear.

    In Cloudera Manager 5.15 or higher, Cloudera Manager can upgrade agents even when they are running on different operating systems or versions (one OS group at a time).

  5. Click Upgrade Cloudera Manager Agent packages

    The Upgrade Cloudera Manager Agent Packages page displays.

  6. If you are using a local package repository instead of the public repository at https://archive.cloudera.com, select the Custom Repository option and enter the Custom Repository URL.
    RHEL / CentOS

    Use the baseurl value in the cloudera-manager.repo file as the Custom Repository. Use the gpgkey value as the Custom GPG Key URL.

    SLES

    Use the baseurl value in the cloudera-manager.repo file as the Custom Repository. Use the gpgkey value as the Custom GPG Key URL.

    Debian / Ubuntu

    Use the entire deb url contrib line from the cloudera_Manager.list file as the Custom Repository. Use the url/archive.key as the Custom GPG Key URL.

    Cloudera Manager automatically enters a custom repository URL that contains your username and password.

  7. Click Continue.

    The Accept JDK License page displays.

  8. The Select JDK screen displays the available options for the JDK used in the cluster. Choose one of the following options to install a JDK:
    • Manually Manage JDK – Select this option if you have already installed a supported JDK. For information on installing a JDK, see Upgrading the JDK.
    • Install a Cloudera-provided version of OpenJDK – Cloudera Manager installs OpenJDK 8 on all your cluster hosts, except for the Cloudera Manager server host(s).
    • Install a system-provided version of OpenJDK – Cloudera Manager installs the default version of OpenJDK provided by the host operating system.
  9. If you want to install JDK 78 on all hosts, select Install Oracle Java SE Development Kit.
  10. Click Continue.

    The Enter Login Credentials page displays.

  11. Specify the credentials and initiate Agent installation:
    1. Select root for the root account, or select Another user and enter the username for an account that has password-less sudo permission.
    2. Select an authentication method:
      • If you choose the All hosts accept same password option, enter and confirm the password.
      • If you choose the All hosts accept same private key option, provide a passphrase and path to the required key files.
    3. Modify the default SSH port if necessary.
    4. Specify the maximum Number of Simultaneous Installations to run at once. The default and recommended value is 10. Adjust this parameter based on your network capacity.
  12. Click Continue.

    The Cloudera Manager Agent packages and, if selected, the JDK are installed.

  13. When the installations complete, click Finish.

    The Upgrade Cloudera Manager page displays the status of the upgrade.

    If there are additional groups of hosts that require Agent upgrades, select the next group from the Upgrade Cloudera Manager Agent Packages running on: drop-down list, and repeat the agent installation steps.

  14. Click Run Host Inspector to run the host inspector. Inspect the output and correct any warnings. If problems occur, you can make changes and then rerun the inspector.
  15. When you are satisfied with the inspection results, click Start the Cloudera Management Service.
  16. Click the link at the bottom of the page to go back to the Home Page.
  17. Click Continue.

    The Cloudera Manager Agent packages and, if selected, the JDK are installed.

    The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If problems occur, you can make changes and then click Run Again to rerun the inspector.

  18. Click Continue.

    The Review Changes page displays and suggests configuration changes you may need to make.

  19. Make any necessary changes and click Continue.

    The Upgrade Wizard restarts the Cloudera Manager Management Service.

  20. Click Finish.
  21. Click the Home Page link to return to the Home page
  22. The Cloudera Manager Home page opens and displays the status of the cluster. It can take several minutes for all of the services to display their current status. You may need to restart some services or redeploy stale client configurations.
Option 2. Upgrade the Agents using the Command Line

Perform the following commands on all hosts managed by Cloudera Manager:

(You can also multiplex the same set of commands to all hosts by using utilities such as csshX, pdsh, or pssh.)
  1. Log in to each host using ssh. For example:
    ssh host1.example.com
  2. Remove any older files in the existing repository directory:
    RHEL / CentOS
    sudo rm /etc/yum.repos.d/cloudera*manager.repo*
    SLES
    sudo rm /etc/zypp/repos.d/cloudera*manager.repo*
    Debian / Ubuntu
    sudo rm /etc/apt/sources.list.d/cloudera*.list*
  3. Create a repository file so that the package manager can locate and download the binaries. Do one of the following, depending on whether or not you are using a local package repository:
    • Using a local package repository. (Required when cluster hosts do not have access to the internet.)

      1. Configure a local package repository hosted on your network.
      2. In the Package Repository URL, replace the entire URL with the URL for your local package repository. A username and password are not required to access local repositories.
      3. Click Apply.
    • Using the Cloudera public repository

      1. Substitute your USERNAME and PASSWORD in the Package Repository URL where indicated in the URL.
      2. Click Apply

    Package Repository URL:

  4. Stop the Cloudera Manager Agent.
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
    sudo systemctl stop cloudera-scm-agent
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
    sudo service cloudera-scm-agent stop
  5. Oracle JDK 1.8
    RHEL / CentOS
    sudo yum install oracle-j2sdk1.8.x86_64
    SLES
    sudo zypper install oracle-j2sdk1.8.x86_64
    Debian / Ubuntu
    sudo apt-get install oracle-j2sdk1.8
  6. Upgrade the agent packages.

    Note: Only add cloudera-manager-server-db-2 if you are using the embedded PostgreSQL database.
    RHEL / CentOS
    sudo yum clean all
    sudo yum repolist
    sudo yum upgrade cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    SLES
    sudo zypper clean --all
    sudo zypper up cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    Debian / Ubuntu
    sudo apt-get clean
    sudo apt-get update
    sudo apt-get dist-upgrade
    sudo apt-get install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    You might be prompted about your configuration file version:
    
    Configuration file '/etc/cloudera-scm-agent/config.ini'
    ==> Modified (by you or by a script) since installation.
    ==> Package distributor has shipped an updated version.
    What would you like to do about it ? Your options are:
    Y or I : install the package maintainer's version
    N or O : keep your currently-installed version
    D : show the differences between the versions
    Z : start a shell to examine the situation
    The default action is to keep your current version.
    You may receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both prompts.
  7. If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is renamed with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.
  8. Verify that you have the correct packages installed.

    Debian / Ubuntu
    dpkg-query -l 'cloudera-manager-*'
    
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
    ||/ Name                   Version                Description
    +++-======================-======================-============================================================
    ii  cloudera-manager-agent 5.15.0-0.cm...~sq The Cloudera Manager Agent
    ii  cloudera-manager-daemo 5.15.0-0.cm...~sq Provides daemons for monitoring Hadoop and related tools.
    RHEL / CentOS / SLES
    rpm -qa 'cloudera-manager-*'
    cloudera-manager-agent-5.15.0-..cm...
    cloudera-manager-daemons-5.15.0-..cm...
    cloudera-manager-server-db-2-5.15.0-..cm...
  9. If you are using the embedded PostgreSQL database, start the database:
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
    sudo systemctl start cloudera-scm-server-db
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
    sudo service cloudera-scm-server-db start
  10. Start the Cloudera Manager Agent.
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
    sudo systemctl start cloudera-scm-agent
    If the agent starts without errors, no response displays.
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
    sudo service cloudera-scm-agent start
    You should see the following:
    Starting cloudera-scm-agent: [ OK ]
  11. In Cloudera Manager 5.15 or higher, you can monitor the progress at https://my_cloudera_manager_server_host:port/cmf/upgrade.
  12. When all the agents are upgraded, run the Host Inspector.
  13. In Cloudera Manager 5.14 or lower, Run the Run Host Inspector: Select Hosts > All Hosts and click Inspect All Hosts.
  14. The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If problems occur, you can make changes and then rerun the inspector.
  15. The Cloudera Manager Home page opens and displays the status of the cluster. It can take several minutes for all of the services to display their current status. You may need to restart some services or redeploy stale client configurations.
  16. Restart the Cloudera Manager server:
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
    sudo systemctl restart cloudera-scm-server
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
    sudo service cloudera-scm-server restart
If you are upgrading from Cloudera Manager 5.5.0 or lower to Cloudera Manager 5.5.0 or higher, hard restart the agent on all hosts to update and restart the supervisord process.
RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
sudo systemctl stop supervisord
sudo systemctl start cloudera-scm-agent
RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
sudo service cloudera-scm-agent hard_restart

Upgrade the Cloudera Manager Agents (Cloudera Manager 7.0.3 and higher)

  1. After upgrading and starting the Cloudera Manager server, open the Cloudera Manager Admin Console (if you have not already done so) using the following URL:
    http://cloudera_Manager_server_hostname:7180/cmf/upgrade
    The Upgrade Cloudera Manager screen displays:


  2. Click Upgrade Cloudera Manager Agent packages

    The Upgrade Cloudera Manager Agent Packages page displays the Select Repository step.

  3. Select one of the following:
    • Select Public Cloudera Repository if the Cloudera Manager server host has access to the internet.
    • Select the Custom Repository If you are using a local package repository instead of the public repository at https://archive.cloudera.com, option and enter the Custom Repository URL.
  4. Click Continue.
  5. The Select JDK screen displays the available options for the JDK used in the cluster. Choose one of the following options to install a JDK:
    • Manually Manage JDK – Select this option if you have already installed a supported JDK. For information on installing a JDK, see Upgrading the JDK.
    • Install a Cloudera-provided version of OpenJDK – Cloudera Manager installs OpenJDK 8 on all your cluster hosts, except for the Cloudera Manager server host(s).
    • Install a system-provided version of OpenJDK – Cloudera Manager installs the default version of OpenJDK provided by the host operating system.
  6. Click Continue.

    The Enter Login Credentials page displays.

  7. Specify the credentials and initiate Agent installation:
    1. Select root for the root account, or select Another user and enter the username for an account that has password-less sudo permission.
    2. Select an authentication method:
      • If you choose the All hosts accept same password option, enter and confirm the password.
      • If you choose the All hosts accept same private key option, provide a passphrase and path to the required key files.
    3. Modify the default SSH port if necessary.
    4. Specify the maximum Number of Simultaneous Installations to run at once. The default and recommended value is 10. Adjust this parameter based on your network capacity.
  8. Click Continue.

    The Cloudera Manager Agent packages and, if selected, the JDK are installed.

  9. When the installations are complete, click Finish.

    The Upgrade Cloudera Manager page displays the status of the upgrade. If you see a message listing Cloudera Manager Agents not upgraded, wait a few minutes for the agents to heartbeat and the click the Refresh button.



  10. After the Agents are all upgraded, Click Run Host Inspector to run the host inspector. Inspect the output and correct any warnings. If problems occur, you can make changes and then rerun the inspector.
  11. When you are satisfied with the inspection results, click Start the Cloudera Management Service.
  12. Confirm that you want to start the Cloudera Management Service by clicking Continue.
  13. After the Cloudera Management Service has started, click Finish.

    You will see a message indicating that the Cloudera Management Service has started.

    The upgrade is now complete.

  14. Click the Home Page link to return to the Home page. Review and fix any critical configuration issues. You may need to restart any clusters if they indicate stale configurations.

    To return to the Upgrade Cloudera Manager page, go to Hosts > All Hosts > Review Upgrade Status.