Cloudera Docs

Preparing for the Operating System Upgrade

Before initiating an operating system upgrade, you must complete several prerequisite operations to ensure environmental readiness. These steps apply to both offline and rolling upgrade strategies and focus on validating software compatibility and infrastructure access. Neglecting these preparations can lead to significant service disruptions, incompatible software configurations, or extended cluster downtime during the migration process.

Verify Parcel Availability

Confirm that your current versions of Cloudera Manager and Cloudera Runtime officially support the target operating system. You must verify that the Cloudera Manager server and all cluster hosts can access the necessary packages and parcels through public or local repositories. This validation step prevents the deployment of Cloudera Manager onto an unsupported platform, which could cause agent communication failures.

Pre-OS Upgrade Requirements

  • Verify that your versions of Cloudera Manager and CDH or Cloudera Runtime support your new operating system. See Operating System Requirements for Cloudera Base on premises.

  • Upgrade Cloudera Manager or Cloudera Runtime Cluster if your current versions do not support the new operating system, see Upgrading Cloudera Manager 7 or Upgrading a Cloudera Runtime Cluster.

  • Ensure all hosts can access the Cloudera Manager server, daemon, and agent packages that support the new operating system. You can provide this access through https://archive.cloudera.com, a local package repository, or directly on the host if you use a pre-created image.

  • Ensure the Cloudera Manager server can access the parcels that support the new operating system. You can provide this access using either the remote repository at https://archive.cloudera.com or a local parcel repository. If you use a pre-created image, the hosts might already contain the distributed parcels locally.

Establish access to the Software

Cloudera Manager requires access to a package repository containing updated software for the new operating system. You can access Cloudera public repositories directly, or you can download and host them through a local repository within your network; however, you must set up a local repository if your cluster hosts lack internet connectivity.

  1. Configure the Package Repository: Create a repository file so the package manager can locate and download the binaries. Perform one of the following actions, depending on whether you use a local package repository:

    • Using a Local Package Repository (Required if your cluster hosts do not have access to the internet.)

      1. Configure a local package repository hosted on your network.

      2. In the Package Repository URL, replace the entire URL with the URL for your local package repository. You do not require a username or password to access local repositories.

      3. Click Apply.

    • Using the Cloudera Public Repository

      1. Substitute your USERNAME and PASSWORD in the Package Repository URL where indicated in the URL.
      2. Click Apply
  2. Create the Cloudera Manager repository file (cloudera_manager.repo or cloudera-manager.list for Ubuntu) to copy to all hosts. Verify that this file has a valid path to the remote Cloudera repository or your local repository before you continue. Ensure that the Cloudera Manager version matches the current version on the cluster.
  3. Configure the Repository File: Perform the steps corresponding to your operating system to allow the package manager to locate the Cloudera Manager binaries.
    RHEL / CentOS
    1. Create a file named /etc/yum.repos.d/cloudera-manager.repo and add the following content:
      [cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/redhat[version]/yum/
gpgkey=https://archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/redhat[version]/yum/RPM-GPG-KEY-cloudera
gpgcheck=1

    2. Copy this file to the same location on all cluster hosts.

    SLES
    1. Create a file named /etc/zypp/repos.d/cloudera-manager.repo and add the following content:
      [cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/sles[version]/yum/
gpgkey=https://archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/sles[version]/yum/RPM-GPG-KEY-cloudera
gpgcheck=1

    2. Copy this file to the same location on all cluster hosts.

    Ubuntu
    1. Create a file named /etc/apt/sources.list.d/cloudera_manager.list and add the following content:
      # Packages for Cloudera Manager
deb [arch=amd64] https://myUsername:myPassword@archive.cloudera.com/p/cm7/[**Cloudera Manager version**]/ubuntu[version]/apt focal-cm[**Cloudera Manager version**] contrib
    2. Run the following command to synchronize your local package index with the newly added Cloudera repository:
      sudo apt-get update

    3. Copy this file to the same location on all cluster hosts.

  4. Review Cloudera Manager Package Dependencies: A Cloudera Manager upgrade often introduces new package dependencies. Since your organization might restrict new package installations or require prior approval, you can identify the exact packages the system intends to install or upgrade. Run the following command for your operating system to review these dependencies before you begin:
    RHEL / CentOS
    yum deplist cloudera-manager-agent
    SLES
    zypper info --requires cloudera-manager-agent
    Ubuntu
    apt-cache depends cloudera-manager-agent

Backing up before upgrading the Operating System

This section describes how to back up critical files and databases on your host before you begin the operating system upgrade. Performing these backups ensures you can recover your configuration, runtime state, and monitoring data if the upgrade fails.

Backing up Cloudera Manager Agent and Server Directories

Follow these steps to preserve the Cloudera Manager Agent configuration and the Cloudera Manager Server runtime state.

  1. Create a top-level backup directory. 
    export CM_BACKUP_DIR="`date +%F`-CM"
echo $CM_BACKUP_DIR
mkdir -p $CM_BACKUP_DIR
  2. Back up the Agent directory and the runtime state. 
    
sudo -E tar -cf $CM_BACKUP_DIR/cloudera-scm-agent.tar --exclude=*.sock /etc/cloudera-scm-agent /etc/default/cloudera-scm-agent /var/run/cloudera-scm-agent /var/lib/cloudera-scm-agent
  3. Back up the Server directories: 
    
sudo -E tar -cf $CM_BACKUP_DIR/cloudera-scm-server.tar /etc/cloudera-scm-server /etc/default/cloudera-scm-server
Backing up Cloudera Manager Databases

Cloudera recommends that you schedule regular backups of the databases that Cloudera Manager uses to store configuration, monitoring, and reporting data. You must also back up databases for any managed services that require them. Choose the procedure corresponding to your database type:

Backing up PostgreSQL Databases

Use this procedure for both embedded and external PostgreSQL databases:

  1. Log in to the host where you installed the Cloudera Manager Server.
  2. Locate the Cloudera Manager database name, user, and password in /etc/cloudera-scm-server/db.properties:
    com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=NnYfWIjlbk
  3. Run the following command as root using the parameters from the previous step:
    # pg_dump -h hostname -p 7432 -U scm > /tmp/scm_server_db_backup.$(date +%Y%m%d)
  4. Enter the password you found in the com.cloudera.cmf.db.password property from Step 2 when the system requests it.
  5. Back up a database created for a specific role on the local host by running the following command as the roleuser:
    # pg_dump -h hostname -p 7432 -U roleuser > /tmp/roledb
  6. Enter the password you specified during the initial database creation when the system requests it.
Backing up MariaDB Databases
To back up the MariaDB database, run the mysqldump command on the MariaDB host as follows:
mysqldump -h <hostname> -u <username> -p<password> <database> > /tmp/database-backup.sql

Example:

  • Back up the Activity Monitor database amon (created in Creating Databases for Cloudera Software) on the local host as the root user, with the password amon_password:
    mysqldump -pamon_password amon > /tmp/amon-backup.sql
  • Back up the sample Activity Monitor database amon on remote host myhost.example.com as the root user, with the password amon_password:
    mysqldump -hmyhost.example.com -uroot -pamon_password amon > /tmp/amon-backup.sql
Backing up MySQL Databases
To back up the MySQL database, run the mysqldump command on the MariaDB host as follows:
mysqldump -h <hostname> -u <username> -p<password> <database> > /tmp/database-backup.sql
Example:
  • Back up the Activity Monitor database amon (created in Creating Databases for Cloudera Software) on the local host as the root user, with the password amon_password:
    mysqldump -pamon_password amon > /tmp/amon-backup.sql
  • Back up the sample Activity Monitor database amon on remote host myhost.example.com as the root user, with the password amon_password:
    mysqldump -hmyhost.example.com -uroot -pamon_password amon > /tmp/amon-backup.sql
  • Back up all databases on the MySQL host by running the following command:
    mysqldump --all-databases -ppassword > /tmp/all_databases_backup.sql
Backing up Oracle Databases

For Oracle environments, coordinate with your Database Administrator (DBA) to ensure you back up all databases according to your organization's policies.

Additional Prerequisites for Rolling Upgrades

Rolling upgrades require specific service configurations to maintain cluster availability while individual hosts are taken offline for maintenance. You must verify that High Availability (HA) is correctly enabled and functional for all critical services across the cluster. This preparation ensures that the workload shifts seamlessly to active nodes, preventing data loss or service outages during the sequential host upgrade process.

Enable and verify High Availability (HA) on all applicable services before starting the rolling upgrade.