From Cloudera Base on premisesPDF version

Step 4: After you upgrade the Operating System to a new Major Version

My Environment

Fill in the following form to create a customized set of instructions for your environment.

Fill out the form above before you proceed.

To share this environment with others, click the icon next to My Environment to copy a link specific for this environment to the clipboard.

7.13.1 7.11.3 7.7.3 7.7.1 7.6.7 7.6.1 7.4.4 7.3.1 7.1.9.1000 7.1.9 7.1.8 7.1.7.3000 7.1.7.2000 7.1.7.1000 7.1.7 7.1.6 7.3.1 7.1.9.1000 7.1.9 7.1.8 7.1.7.3000 7.1.7.2000 7.1.7.1000 7.1.7

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

This topic describes how to upgrade the operating system on a Cloudera Manager managed host.

Cloudera Manager needs access to a package repository that contains the updated software packages. You can choose to access the Cloudera public repositories directly, or you can download those repositories and set up a local repository to access them from within your network. If your cluster hosts do not have connectivity to the Internet, you must set up a local repository.

  1. Log in to the Cloudera Manager Server host. 
    ssh my_cloudera_manager_server_host
  2. Log in to each cluster host. 
    ssh cluster_host
  3. Remove any older files in the existing repository directory:
    RHEL / CentOS
    sudo rm /etc/yum.repos.d/cloudera*manager.repo*
    SLES
    sudo rm /etc/zypp/repos.d/cloudera*manager.repo*
    Ubuntu
    sudo rm /etc/apt/sources.list.d/cloudera*.list*
  4. Fill in the form at the top of this page.
  5. Create a repository file so that the package manager can locate and download the binaries. Do one of the following, depending on whether or not you are using a local package repository:

    • Using a local package repository. (Required when cluster hosts do not have access to the internet.)

      1. Configure a local package repository hosted on your network.
      2. In the Package Repository URL, replace the entire URL with the URL for your local package repository. A username and password are not required to access local repositories.
      3. Click Apply.

    • Using the Cloudera public repository

      1. Substitute your USERNAME and PASSWORD in the Package Repository URL where indicated in the URL.
      2. Click Apply

    Package Repository URL:

  6. RHEL / CentOS

    Create a file named /etc/yum.repos.d/cloudera-manager.repo with the following content:

    [cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/redhat[version]/yum/
gpgkey=https://archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/redhat[version]/yum/RPM-GPG-KEY-cloudera
gpgcheck=1

    Copy this file to the same location on all cluster hosts.

    SLES

    Create a file named /etc/zypp/repos.d/cloudera-manager.repo with the following content:

    [cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/sles[version]/yum/
gpgkey=https://archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/sles[version]/yum/RPM-GPG-KEY-cloudera
gpgcheck=1

    Copy this file to the same location on all cluster hosts.

    Ubuntu
    Create a file named /etc/apt/sources.list.d/cloudera_manager.list with the following content:
    # Packages for Cloudera Manager
deb [arch=amd64] https://myUsername:myPassword@archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/ubuntu[version]/apt focal-cm[**Cloudera Manager version**] contrib
    Run the following command:
    sudo apt-get update

    Copy this file to the same location on all cluster hosts.

    The repository file, as created, refers to the most recent maintenance release of the specified minor release. If you would like to use a specific maintenance version, for example 5.15.1, replace 5.15 with 5.15.1 in the generated repository file shown above.

  7. A Cloudera Manager upgrade can introduce new package dependencies. Your organization may have restrictions or require prior approval for installation of new packages. You can determine which packages may be installed or upgraded:
    RHEL / CentOS
    yum deplist cloudera-manager-agent
    SLES
    zypper info --requires cloudera-manager-agent
    Ubuntu
    apt-cache depends cloudera-manager-agent

On the Cloudera Manager Server host, re-install the removed Cloudera packages.

  1. Install the packages.

    Include the cloudera-manager-server-db-2 package in the command only if you are using the embedded PostgreSQL database.

    RHEL / CentOS
    sudo yum clean all
sudo yum install cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    SLES
    sudo zypper clean --all
sudo zypper install cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    Ubuntu
    sudo apt-get clean

    sudo apt-get update

    sudo apt-get install cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2

  2. Verify that the configuration files (that were backed up) are intact. Correct if necessary.

  3. If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is renamed with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.

On the host with Cloudera Manager Server, the appropriate services typically will start automatically on reboot. Otherwise, start the Cloudera Manager Server and the local Agent as necessary.

  1. Start the Cloudera Manager Server by running the following command:
    sudo systemctl start cloudera-scm-server
  2. Start the Cloudera Manager Agent.
    sudo systemctl start cloudera-scm-agent

    If the agent starts without errors, no response displays.

  3. Verify that the Cloudera Manager Agent downloaded a proper parcel for your new operating system. You can use the following command to check-in Cloudera Manager logs for downloaded parcels: 
    grep "Completed download" /var/log/cloudera-scm-agent/cloudera-scm-agent.log
    The download might take some time. Look for the operating system in the names of the downloaded parcels. The output of the command should look similar to the following:
    # grep "Completed download" /var/log/cloudera-scm-agent/cloudera-scm-agent.log
[02/May/2023 07:26:44 +0000] 20004 WorkerThread downloader   INFO     Completed download of http://nightly-7x-rf-1.nightly-7x-rf.root.hwx.site:7180/cmf/parcel/download/CDH-7.2.17-1.cdh7.2.17.p0.40465599-el7.parcel code=200 state=downloaded
[02/May/2023 07:31:00 +0000] 20004 WorkerThread downloader   INFO     Completed download of http://nightly-7x-rf-1.nightly-7x-rf.root.hwx.site:7180/cmf/parcel/download/FLINK-1.12-csa1.4.0.0-cdh7.1.6.0-297-13947709-el7.parcel code=200 state=downloaded

    Verify that the timestamps of the log lines are recent. Re-run the grep command again until the log lines appear. Verify the operating system name and major version in the parcel names and match the new operating system installed on the host.

    In the above output example, el7 refers to Redhat Enterprise Linux 7.

  1. Install and start the Cloudera Manager Agent package on all the remaining hosts of the cluster by running the following command:

    RHEL / CentOS
    sudo yum clean all

    sudo yum install cloudera-manager-daemons cloudera-manager-agent
    SLES
    sudo zypper clean –all

    sudo zypper install cloudera-manager-daemons cloudera-manager-agent
    Ubuntu
    sudo apt-get clean

    sudo apt-get update

    sudo apt-get install cloudera-manager-daemons cloudera-manager-agent
  2. Verify that the configuration files (that were backed up) are intact. Correct if necessary.

  3. If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is renamed with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.

  4. Start the Cloudera Manager Agent
    sudo systemctl start cloudera-scm-agent

  1. Log in to Cloudera Manager as an Administrator.
  2. Go to Hosts > All Hosts page and verify that the status of all the hosts are in green.
  3. Go to Clusters > Parcels and verify that all needed parcels are in the Distributed and Activated state.
  4. Start the Cloudera Management Service as follows:
    1. On the Home > Status tab, click the options menu to the right of Cloudera Management Service and select Start.
  5. For each cluster on the home page, click the options menu to the right of each Cluster and select Start to start all the services in each of the cluster.