Upgrading the JDK

Cloudera Manager and CDH require a supported version of the Java Development Kit (JDK) to be installed on all hosts. For details, see Java Requirements

Supported JDKs
Cloudera Enterprise Version Supported Oracle JDK Supported OpenJDK
5.3 -5.15 1.7, 1.8 none
5.16 and higher 5.x releases 1.7, 1.8 1.8
6.0 1.8 none
6.1 1.8 1.8
6.2 1.8 1.8
6.3 1.8 1.8, 11.0.3 or higher

.

Loading Filters ... 6.3.4 6.3.3 6.3.1 6.3.0 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0 6.3.4 6.3.3 6.3.2 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0

There are several procedures you can use to upgrade the JDK:
  • Installing JDK 8 as part of an Upgrade to Cloudera Manager 6.x

    If you are upgrading to Cloudera Manager 6.0.0 or higher, you can manually install JDK 1.8 on the Cloudera Manager server host, and then, as part of the Cloudera Manager upgrade process, you can specify that Cloudera Manager upgrade the JDK on the remaining hosts.

  • Manually Installing Oracle JDK 1.8

    You can manually install JDK 1.8 on all managed hosts. If you are upgrading to any version of Cloudera Manager 5.x, you must use this procedure. Continue with the steps in the next section.

  • Manually Migrating to OpenJDK

Continue reading:

Manually Installing Oracle JDK 1.8

You can manually install Oracle JDK 1.8 on all managed hosts. If you are upgrading to any version of Cloudera Manager 5.x, you must use the following procedure:

  1. Download the .tar.gz file for one of the 64-bit versions of Oracle JDK 1.8 from Java SE 8 Downloads. (This link is correct at the time of writing, but can change.) See Supported Java versions for CDH 5 or Java Requirements.
  2. Perform the following steps on all hosts that you are upgrading:
    1. Log in to the host as root using ssh.
    2. Copy the downloaded .tar.gz file to the host.
    3. Extract the JDK to the folder /usr/java/jdk-version. For example: 
      tar xvfz /path/to/jdk-8u<update_version>-linux-x64.tar.gz -C /usr/java/
  3. If you have configured TLS for Cloudera Manager, copy the jssecacerts file from the previous JDK installation to the new JDK installation. This step is not required when using JDK 1.8.0_162 or greater. JDK 1.8.0_162 enables unlimited strength encryption by default.
    For example: 
    cp previous_java_home/jre/lib/security/jssecacerts new_java_home/jre/lib/security
    (Substitute previous_java_home and new_java_home with the paths to the JDK installations.)
  4. Configure the location of the JDK on cluster hosts.
    1. Open the Cloudera Manager Admin Console.
    2. In the main navigation bar, click the Hosts tab. If you are configuring the JDK location on a specific host only, click the link for that host.
    3. Click the Configuration tab.
    4. Select Category > Advanced.
    5. Set the Java Home Directory property to the custom location.
    6. Click Save Changes.
  5. On the Cloudera Manager Server host only (not required for other hosts):
    1. Open the file /etc/default/cloudera-scm-server in a text editor.
    2. Edit the line that begins with export JAVA_HOME (if this line does not exist, add it) and change the path to the path of the new JDK (you can find the path under /usr/java).
      For example: (RHEL and SLES) 
      export JAVA_HOME="/usr/java/jdk1.8.0_141-cloudera"
      For example: (Ubuntu) 
      export JAVA_HOME="/usr/lib/jvm/java-8-oracle-cloudera"
    3. Save the file.
    4. Restart the Cloudera Manager Server.
      RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
      sudo systemctl restart cloudera-scm-server
      RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
      sudo service cloudera-scm-server restart
  6. Restart the Cloudera Management Service.
    1. Log in to the Cloudera Manager Admin Console.
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Restart.
  7. Restart all clusters:
    1. On the Home > Status tab, click to the right of the cluster name and select Restart.
    2. Click Restart that appears in the next screen to confirm. If you have enabled high availability for HDFS, you can choose Rolling Restart instead to minimize cluster downtime. The Command Details window shows the progress of stopping services.

      When All services successfully started appears, the task is complete and you can close the Command Details window.

  8. Delete the files from your previous Java installation. If you do not delete these files, Cloudera Manager and other components may continue to use the old version of the JDK.

OpenJDK

You must install a supported version of OpenJDK. If your deployment uses a version of OpenJDK lower than 1.8.0_181, see TLS Protocol Error with OpenJDK.

Manually Installing OpenJDK

Before installing Cloudera Manager and CDH, perform the steps in this section to install OpenJDK on all hosts in your cluster(s).

OpenJDK 11 is supported as of Cloudera Manager and CDH 6.3. Note the following:
  • The package names used when installing the OpenJDK 11 and OpenJDK 8 are different and are noted in the steps below.
  • The path for the default truststore has changed from (OpenJDK 8) jre/lib/security/cacerts to (OpenJDK 11) lib/security/cacerts
  • See the following blog post for general information about migrating to Java 11: All You Need to Know For Migrating To Java 11.

See Supported JDKs for information on which JDK versions are supported for Cloudera Enterprise releases.

You must install a supported version of OpenJDK. If your deployment uses a version of OpenJDK lower than 1.8.0_181, see TLS Protocol Error with OpenJDK.

  1. Log in to each host and run the command for the version of the JDK you want to install:
    RHEL
    OpenJDK 8 
    su -c yum install java-1.8.0-openjdk-devel
    OpenJDK 11 
    su -c yum install java-11-openjdk-devel
    Ubuntu
    OpenJDK 8 
    sudo apt-get install openjdk-8-jdk
    OpenJDK 11 
    sudo apt-get install openjdk-11-jdk
    SLES
    OpenJDK 8 
    sudo zypper install java-1_8_0-openjdk-devel
    OpenJDK 11 
    sudo zypper install java-11-openjdk-devel
  2. Tune the JDK (OpenJDK 11 only.)

    OpenJDK 11 uses new defaults for garbage collection and other Java options specified when launching Java processes. Due to these changes you may need to tune the garbage collection by adjusting the Java options used to run cluster services, which are configured separately for each service using the service's configuration parameters. To locate the correct parameter, log in to the Cloudera Manager Admin Console, go to the cluster and service you want to configure and search for "Java Configuration Options".

    When using OpenJDK 11, Cloudera Manager and most CDH services use G1GC as the default method of garbage collection. Java 8 used "ConcurrentMarkSweep" (CMS) for garbage collection. When using G1GC, the pauses for garbage collection are shorter, so components will usually be more responsive, but they are more sensitive to JVMs with overcommitted memory usage. See Tuning JVM Garbage Collection.

Manually Migrating to OpenJDK

If you have Oracle JDK 1.7, Oracle JDK 1.8, or OpenJDK 8 installed on the hosts managed by Cloudera Manager, use the steps in this section to migrate your deployment to use OpenJDK. The steps below require you to restart all clusters, which will cause downtime as the hosts restart. If your clusters have enabled high availability for HDFS, you can use a rolling restart to restart the clusters without downtime. Note that until the rolling restart completes, some of the hosts in your cluster will still be using the Oracle JDK. If you do not want a temporarily mixed environment, you can stop the cluster before performing the steps in this section to migrate the JDK.

OpenJDK 11 is supported as of Cloudera Manager and CDH 6.3. Note the following:
  • You must upgrade to Cloudera Manager 6.3 or higher, before upgrading to OpenJDK 11.
  • The package names used when installing the OpenJDK 11 are different and are noted in the steps below.
  • The path for the default truststore has changed from (OpenJDK 8) jre/lib/security/cacerts to (OpenJDK 11) lib/security/cacerts
  • See the following blog post for general information about migrating to Java 11: All You Need to Know For Migrating To Java 11.
  • You must install a supported version of OpenJDK. If your deployment uses a version of OpenJDK lower than 1.8.0_181, see TLS Protocol Error with OpenJDK.
  1. Find out the package name of your currently installed JDK by running the following commands. The grep commands attempt to locate the installed JDK. If the JDK package is not returned, try looking for the string sdk.
    RHEL
    Oracle JDK 8 
    yum list installed |grep oracle
    OpenJDK 8 
    yum list installed |grep openjdk
    Ubuntu
    Oracle JDK 8 
    apt list --installed | grep oracle
    OpenJDK 8 
    apt list --installed | grep openjdk
    SLES
    Oracle JDK 8 
    zypper search --installed-only |grep oracle
    OpenJDK 8 
    zypper search --installed-only |grep openjdk
    The command will return values similar to the following example:: 
    oracle-j2sdk1.7.x86_64              1.7.0+update67-1           java-1.8.0-openjdk-devel
    The Oracle JDK package name in the above example is: oracle-j2sdk1.7.x86_64. The OpenJDK package is java-1.8.0-openjdk-devel.
  2. Log in to each host managed by Cloudera Manager (including the Cloudera Manager server host) and run the following command to install OpenJDK:
    RHEL
    OpenJDK 8 
    su -c yum install java-1.8.0-openjdk-devel
    OpenJDK 11 
    su -c yum install java-11-openjdk-devel
    Ubuntu
    OpenJDK 8 
    sudo apt-get install openjdk-8-jdk
    OpenJDK 11 
    sudo apt-get install openjdk-11-jdk
    SLES
    OpenJDK 8 
    sudo zypper install java-1_8_0-openjdk-devel
    OpenJDK 11 
    sudo zypper install java-11-openjdk-devel
  3. On the Cloudera Manager Server host only (not required for other hosts):
    1. Open the file /etc/default/cloudera-scm-server in a text editor.
    2. Edit the line that begins with export JAVA_HOME (if this line does not exist, add it) and change the path to the path of the new JDK (the JDK is usually installed in /usr/lib/jvm)(or /usr/lib64/jvm on SLES 12), but the path may differ depending on how the JDK was installed).

      For example:

      RHEL 6
      export JAVA_HOME="/usr/lib/jvm/java-1.8.0-openjdk.x86_64"
      RHEL 7
      export JAVA_HOME="/usr/lib/jvm/java-1.8.0-openjdk"
      Debian
      export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64"
      Ubuntu
      export JAVA_HOME="/usr/lib/jvm/openjdk-8-jdk"
      SLES
      export JAVA_HOME="/usr/lib64/jvm/java-1.8.0-openjdk"
    3. Save the file.
    4. Restart the Cloudera Manager Server.
      RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
      sudo systemctl restart cloudera-scm-server
      RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
      sudo service cloudera-scm-server restart
  4. Tune the JDK (OpenJDK 11 only.)

    OpenJDK 11 uses new defaults for garbage collection and other Java options specified when launching Java processes. Due to these changes you may need to tune the garbage collection by adjusting the Java options used to run cluster services, which are configured separately for each service using the service's configuration parameters. To locate the correct parameter, log in to the Cloudera Manager Admin Console, go to the cluster and service you want to configure and search for "Java Configuration Options".

    When using OpenJDK 11, Cloudera Manager and most CDH services use G1GC as the default method of garbage collection. Java 8 used "ConcurrentMarkSweep" (CMS) for garbage collection. When using G1GC, the pauses for garbage collection are shorter, so components will usually be more responsive, but they are more sensitive to JVMs with overcommitted memory usage. See Tuning JVM Garbage Collection.

  5. Restart the Cloudera Management Service.
    1. Log in to the Cloudera Manager Admin Console.
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Restart.
  6. Restart all clusters:
    1. On the Home > Status tab, click to the right of the cluster name and select either Restart or Rolling Restart. Selecting Rolling Restart minimizes cluster downtime and is available only If you have enabled high availability for HDFS.
    2. Click Restart or Rolling Restart that appears in the next screen to confirm. The Command Details window shows the progress of stopping services.

      When All services successfully started appears, the task is complete and you can close the Command Details window.

  7. Remove the Oracle JDK:
    1. Perform the following steps on all hosts managed by Cloudera Manager:
      1. Run the following command to remove the Oracle JDK, using the package names from Step 1: (If you do not delete these files, Cloudera Manager and other components may continue to use the old version of the JDK.)
        RHEL
        yum remove <JDK package name>
        Ubuntu
        apt-get remove <JDK package name>
        SLES
        zypper rm <JDK package name>
      2. Confirm that the package has been removed:
        RHEL
        yum list installed |grep -i oracle
        Ubuntu
        apt list --installed | grep -i oracle
        SLES
        zypper search --installed-only |grep -i oracle

Using AES-256 Encryption

If you are using CentOS/Red Hat Enterprise Linux 5.6 or higher, or Ubuntu, which use AES-256 encryption by default for tickets, you must install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy File on all cluster and Hadoop user machines. For JCE Policy File installation instructions, see the README.txt file included in the jce_policy-x.zip file. For more information, see Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy File

Alternately, you can configure Kerberos to not use AES-256 by removing aes256-cts:normal from the supported_enctypes field of the kdc.conf or krb5.conf file. After changing the kdc.conf file, you must restart both the KDC and the kadmin server for those changes to take affect. You may also need to re-create or change the password of the relevant principals, including, potentially the Ticket Granting Ticket principal (krbtgt/REALM@REALM). If AES-256 is still used after completing steps, the aes256-cts:normal setting existed when the Kerberos database was created. To fix this, create a new Kerberos database and then restart both the KDC and the kadmin server.

To verify the type of encryption used in your cluster:

  1. On the local KDC host, type this command to create a test principal: 
    kadmin -q "addprinc test"
  2. On a cluster host, type this command to start a Kerberos session as the test principal: 
    kinit test
  3. On a cluster host, type this command to view the encryption type in use: 
    klist -e

    If AES is being used, output like the following is displayed after you type the klist command; note that AES-256 is included in the output:

    Ticket cache: FILE:/tmp/krb5cc_0
Default principal: test@SCM
Valid starting     Expires            Service principal
05/19/11 13:25:04  05/20/11 13:25:04  krbtgt/SCM@SCM
    Etype (skey, tkt): AES-256 CTS mode with 96-bit SHA-1 HMAC, AES-256 CTS mode with 96-bit SHA-1 HMAC

Configuring a Custom Java Home Location

Although not recommended, the Java Development Kit (JDK), which Cloudera services require, may be installed at a custom location if necessary. These steps assume you have already installed the JDK as documented in Step 2: Install Java Development Kit or as part of an upgrade.

To modify the Cloudera Manager configuration to ensure the JDK can be found:
  1. Open the Cloudera Manager Admin Console.
  2. In the main navigation bar, click the Hosts tab. If you are configuring the JDK location on a specific host only, click the link for that host.
  3. Click the Configuration tab.
  4. Select Category > Advanced.
  5. Set the Java Home Directory property to the custom location.
  6. Click Save Changes.
  7. Restart all services.