Upgrading the Cloudera Manager Server

This topic provides procedures for backing up the Cloudera Manager Server.

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

Loading Filters ... 6.3.4 6.3.3 6.3.1 6.3.0 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0 6.3.4 6.3.3 6.3.2 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0

Important: Upgrade and installation procedures for Cloudera Manager or CDH 6.x have changed. Please note the following:

A valid Cloudera Enterprise license file and a username and password are required to download and install the software. You can obtain the username and password from the Cloudera CDH Download page. See Your license file must be current and uploaded to Cloudera Manager.
To upload a license:
1. Download the license file and save it locally.
2. In Cloudera Manager, go to the Home page.
3. Select Administration > License.
4. Click Upload License.
5. Browse to the license file you downloaded.
6. Click Upload.
If you are currently using a Cloudera trial license, you cannot upgrade to Cloudera Manager or CDH 6.3.3.
If you are using Cloudera Express, you cannot upgrade Cloudera Manager or CDH.
Several steps in the procedures have changed and now require the username and password.
Download URLs have changed.

After you complete the steps in Getting Started Upgrading Cloudera Manager and Backing Up Cloudera Manager, continue with the following:

Establish Access to the Software
Install Oracle JDK 8
Upgrade the Cloudera Manager Server

Establish Access to the Software

Cloudera Manager needs access to a package repository that contains the updated software packages. You can choose to access the Cloudera public repositories directly, or you can download those repositories and set up a local repository to access them from within your network. If your cluster hosts do not have connectivity to the Internet, you must set up a local repository.

Remove any older files in the existing repository directory:

RHEL / CentOS

sudo rm /etc/yum.repos.d/cloudera*manager.repo*

SLES

sudo rm /etc/zypp/repos.d/cloudera*manager.repo*

Debian / Ubuntu

sudo rm /etc/apt/sources.list.d/cloudera*.list*

Fill in the form at the top of this page.
Create a repository file so that the package manager can locate and download the binaries. Do one of the following, depending on whether or not you are using a local package repository:
- Using a local package repository. (Required when cluster hosts do not have access to the internet.)
  1. Configure a local package repository hosted on your network.
  2. In the Package Repository URL, replace the entire URL with the URL for your local package repository. A username and password are not required to access local repositories.
  3. Click Apply.
- Using the Cloudera public repository
  1. Substitute your USERNAME and PASSWORD in the Package Repository URL where indicated in the URL.
  2. Click Apply
Package Repository URL:

Tip: If you have a mixed operating system environment, adjust the Operating System filter at the top of the page for each operating system. The guide will generate the repo file for you automatically here.

RHEL / CentOS

Create a file named /etc/yum.repos.d/cloudera-manager.repo with the following content:

[cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15
gpgkey=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera
gpgcheck=1

SLES

Create a file named /etc/zypp/repos.d/cloudera-manager.repo with the following content:

[cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/5.15
gpgkey=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/RPM-GPG-KEY-cloudera
gpgcheck=1

Debian / Ubuntu

Debian is not a supported operating system for Cloudera Manager 6.x.

Create a file named /etc/apt/sources.list.d/cloudera_manager.list with the following content:

# Packages for Cloudera Manager
deb https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib
deb-src https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib

Run the following command:

sudo apt-get update

The repository file, as created, refers to the most recent maintenance release of the specified minor release. If you would like to use a specific maintenance version, for example 5.15.1, replace 5.15 with 5.15.1 in the generated repository file shown above.

Make the following changes to the repository file:
1. Add /p after https://archive.cloudera.com.
2. Prepend your username and password to the URLs.
For example:
```
https://username:password@archive.cloudera.com/p/...
```
A Cloudera Manager upgrade can introduce new package dependencies. Your organization may have restrictions or require prior approval for installation of new packages. You can determine which packages may be installed or upgraded:
RHEL / CentOS
```
yum deplist cloudera-manager-agent
```
SLES
```
zypper info --requires cloudera-manager-agent
```
Debian / Ubuntu
```
apt-cache depends cloudera-manager-agent
```

Install Oracle JDK 8

Oracle JDK 1.8 is required on all cluster hosts managed by Cloudera Manager 6.0.0 or higher. If it is supported for your version of Cloudera Manager, you can also manually install OpenJDK 1.8 or OpenJDK 11. See Manually Installing OpenJDK. If JDK 1.8 is already installed on your hosts, skip the steps in this section.

If you are upgrading to Cloudera Manager 6.0.0 or higher, you can manually install JDK 8 on the Cloudera Manager server host, and then, as part of the Cloudera Manager upgrade process, you can specify that Cloudera Manager upgrade the JDK on the remaining hosts.

Install JDK 1.8.

RHEL / CentOS

sudo yum install oracle-j2sdk1.8.x86_64

SLES

sudo zypper install oracle-j2sdk1.8.x86_64

Debian / Ubuntu

sudo apt-get install oracle-j2sdk1.8

Open the following file in a text editor:
```
/etc/default/cloudera-scm-server
```
Edit the line that begins with export JAVA_HOME (if this line does not exist, add it) and change the path to the path of the new JDK (you can find the path under /usr/java).
For example: (RHEL and SLES)
```
export JAVA_HOME="/usr/java/jdk1.8.0_141-cloudera"
```
For example: (Ubuntu)
```
export JAVA_HOME="/usr/lib/jvm/java-8-oracle-cloudera"
```
Save the file.

Upgrade the Cloudera Manager Server

Log in to the Cloudera Manager Admin Console.
If your cluster is running the embedded PostgreSQL database, stop all services that are using the embedded database. These can include:
- Hive service and all services such as Impala and Hue that use the Hive metastore
- Oozie
- Sentry
- Sqoop
Stop the Cloudera Management Service.
1. Log in to the Cloudera Manager Admin Console.
2. Select Clusters > Cloudera Management Service.
3. Select Actions > Stop.
Important: Not stopping the Cloudera Management Service at this point might cause management roles to crash or the Cloudera Manager Server might fail to restart.
Ensure that you have disabled any scheduled replication or snapshot jobs and wait for any running commands from the Cloudera Manager Admin Console to complete before proceeding with the upgrade.
Important: If there are replication jobs, snapshot jobs, or other commands running when you stop Cloudera Manager Server, Cloudera Manager Server might fail to start after the upgrade.
If you have any Hive Replication Schedules that replicate to a cloud destination, delete these replication clusters before continuing with the upgrade. You can re-create these Replication Schedules after the Cloudera Manager upgrade is complete.
Log in to the Cloudera Manager Server host.
```
ssh my_cloudera_manager_server_host
```
Stop the Cloudera Manager Server.
RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
```
sudo systemctl stop cloudera-scm-server
```
RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
```
sudo service cloudera-scm-server stop
```
If you are using the embedded PostgreSQL database, stop the Cloudera Manager Embedded PostgreSQL database:
RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
```
sudo systemctl stop cloudera-scm-server-db
```
RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
```
sudo service cloudera-scm-server-db stop
```
If you are not using the embedded PostgreSQL database and you attempt to stop it, you might see a message indicating that the service cannot be found. If you see a message that the shutdown failed, then the embedded database is still running, probably because services are connected to the Hive metastore. If the database shutdown fails due to connected services, issue the following command:
RHEL-compatible 7 and higher, Ubuntu 16.04
```
sudo service cloudera-scm-server-db next_stop_fast
sudo service cloudera-scm-server-db stop
```
All other Linux distributions
```
sudo service cloudera-scm-server-db fast_stop
```
Stop the Cloudera Manager Agent.
RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
```
sudo systemctl stop cloudera-scm-agent
```
RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
```
sudo service cloudera-scm-agent stop
```

Upgrade the packages. Include the cloudera-manager-server-db-2 package in the command only if you are using the embedded PostgreSQL database.

RHEL / CentOS

sudo yum clean all
sudo yum upgrade cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2

SLES

sudo zypper clean --all
sudo zypper up cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2

Debian / Ubuntu

sudo apt-get clean

sudo apt-get update

sudo apt-get dist-upgrade

sudo apt-get install cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2

You might be prompted about your configuration file version:

Configuration file '/etc/cloudera-scm-agent/config.ini'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.

You may receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both prompts.

You may be prompted to accept the GPG key. Answer y.

Retrieving key from https://archive.cloudera.com/.../cm/RPM-GPG-KEY-cloudera
Importing GPG key ...
 Userid     : "Yum Maintainer <webmaster@cloudera.com>"
 Fingerprint: ...
 From       : https://archive.cloudera.com/.../RPM-GPG-KEY-cloudera

If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is renamed with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.

Verify that you have the correct packages installed.

Debian / Ubuntu

dpkg-query -l 'cloudera-manager-*'

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                   Version                Description
+++-======================-======================-============================================================
ii  cloudera-manager-agent 5.15.0-0.cm...~sq The Cloudera Manager Agent
ii  cloudera-manager-daemo 5.15.0-0.cm...~sq Provides daemons for monitoring Hadoop and related tools.
ii  cloudera-manager-serve 5.15.0-0.cm...~sq The Cloudera Manager Server

RHEL / CentOS / SLES

rpm -qa 'cloudera-manager-*'

cloudera-manager-server-5.15.0-..cm...
cloudera-manager-agent-5.15.0-..cm...
cloudera-manager-daemons-5.15.0-..cm...
cloudera-manager-server-db-2-5.15.0-..cm...

If you are using the embedded PostgreSQL database, start the database:
RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
```
sudo systemctl start cloudera-scm-server-db
```
RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
```
sudo service cloudera-scm-server-db start
```
Start the Cloudera Manager Agent.
RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
```
sudo systemctl start cloudera-scm-agent
```
If the agent starts without errors, no response displays.
RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
```
sudo service cloudera-scm-agent start
```
You should see the following:
```
Starting cloudera-scm-agent: [ OK ]
```
Start the Cloudera Manager Server.
RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
```
sudo systemctl start cloudera-scm-server
```
If the Cloudera Manager Server starts without errors, no response displays.
RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
```
sudo service cloudera-scm-server start
```
You should see the following:
```
Starting cloudera-scm-server: [ OK ]
```
Use a Web browser to open the Cloudera Manager Admin Console using the following URL:
```
http://cloudera_Manager_server_hostname:7180/cmf/upgrade
```
```
http://cloudera_Manager_server_hostname:7180/cmf/upgrade-wizard/welcome
```
It can take several minutes for the Cloudera Manager Server to start, and the Cloudera Manager Admin Console is unavailable until the server startup is complete and the Upgrade Cloudera Manager page displays. Continue with the steps on the next page to upgrade the Cloudera Manager Agents.

Note: If you have problems starting the server or the agent, such as database permissions problems, you can use log files to troubleshoot the problem:

Server log:

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

Agent log:

tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log

tail -f /var/log/messages

To complete the Cloudera Manager upgrade, continue with Upgrading the Cloudera Manager Agents.

Categories: Cloudera Manager | Upgrading | All Categories

Backing Up

Upgrading the Agents