Install and Upgrade Known Issues
Upgrade to CDH 5.13 or higher Requires Pre-installation of Spark 2.1 or Spark 2.2
If your cluster has Spark 2.0 or Spark 2.1 installed and you want to upgrade to CDH 5.13 or higher, you must first upgrade to Spark 2.1 release 2 or later before upgrading CDH. To install these versions of Spark, do the following before running the CDH Upgrade Wizard:
- Install the Custom Service Descriptor (CSD) file. See
- Download, distribute, and activate the Parcel for the version of Spark that you are installing:
- Spark 2.1 release 2: The parcel name includes "cloudera2" in its name.
- Spark 2.2 release 1: The parcel name includes "cloudera1" in its name.
Affected versions: CDH 5.13.0 and higher
Cloudera bug: CDH-56775
Flume Kafka client incompatible changes in CDH 5.8
Due to the change of offset storage from ZooKeeper to Kafka in the CDH 5.8 Flume Kafka client, data might not be consumed by the Flume agents, or might be duplicated (if kafka.auto.offset.reset=smallest) during an upgrade to CDH 5.8.
Workaround: See Upgrading to CDH 5.8 When Using the Flume Kafka Client
Cloudera Bug: TSB-173
Upgrades to CDH 5.4.1 from Releases Earlier than 5.4.0 May Fail
Problem: Because of a change in the implementation of the NameNode metadata upgrade mechanism, upgrading to CDH 5.4.1 from a version lower than 5.4.0 can take an inordinately long time. In a cluster with NameNode high availability (HA) configured and a large number of edit logs, the upgrade can fail, with errors indicating a timeout in the pre-upgrade step on JournalNodes.
What to do:
To avoid the problem: Do not upgrade to CDH 5.4.1; upgrade to CDH 5.4.2 instead.
If you experience the problem: If you have already started an upgrade and seen it fail, contact Cloudera Support. This problem involves no risk of data loss, and manual recovery is possible.
If you have already completed an upgrade to CDH 5.4.1, or are installing a new cluster: In this case you are not affected and can continue to run CDH 5.4.1.
Potential job failures during YARN rolling upgrades to CDH 5.3.4
Problem: A MapReduce security fix introduced a compatibility issue that results in job failures during YARN rolling upgrades from CDH 5.3.3 to CDH 5.3.4.
Cloudera Bug: CDH-28680
Release affected: CDH 5.3.4
Release containing the fix: CDH 5.3.5
- Upgrade to CDH 5.3.5.
- Restart any jobs that might have failed during the upgrade.
- Explicitly set the version of MapReduce to be used so it is picked on a per-job basis.
- Update the YARN property, MR Application Classpath (mapreduce.application.classpath), either in Cloudera Manager or in the mapred-site.xml file. Remove all existing values and add a new entry: <parcel-path>/lib/hadoop-mapreduce/*, where <parcel-path> is the absolute path to the parcel installation. For example, the default installation path for the CDH 5.3.3 parcel would be: /opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/lib/hadoop-mapreduce/*.
- Wait until jobs submitted with the above client configuration change have run to completion.
- Upgrade to CDH 5.3.4.
- Update the MR Application Classpath (mapreduce.application.classpath) property to point to the new CDH 5.3.4
parcel.
Do not delete the old parcel until after all jobs submitted prior to the upgrade have finished running.
NameNode Incorrectly Reports Missing Blocks During Rolling Upgrade
Problem: During a rolling upgrade to any of the releases listed below, the NameNode may report missing blocks after rolling back multiple DataNodes. This is caused by a race condition with block reporting between the DataNode and the NameNode. No permanent data loss occurs, but data can be unavailable for up to six hours before the problem corrects itself.
Releases affected: CDH 5.0.6, 5.1.5, 5.2.5, 5.3.3, 5.4.1, 5.4.2
What to do:
To avoid the problem: Cloudera advises skipping the affected releases and installing a release containing the fix. For example, do not upgrade to CDH 5.4.2; upgrade to CDH 5.4.3 instead.
The releases containing the fix are: CDH 5.3.4, 5.4.3
If you have already completed an upgrade to an affected release, or are installing a new cluster: You can continue to run the release, or upgrade to a release that is not affected.
Upgrading to CDH 5.4 or later requires an HDFS upgrade
Upgrading to CDH 5.4.0 or later from an earlier CDH 5 release requires an HDFS upgrade, and upgrading from a release earlier than CDH 5.2.0 requires additional steps. See Upgrading from an Earlier CDH 5 Release to the Latest Release for further information. See also What's New In CDH 5.4.x.
JDK Does Not Have Up-to-date tzdata
The Europe/Moscow time zone has changed. This time zone change is not incorporated into JDK lower than 8u31 and 7u75. Cloudera ships JDK7u67, which does not reflect these tzdata updates. If you run UDF related to time zone conversion, you receive incorrect results.
Workaround: Upgrade to JDK version 8u31 or 7u75.
Extra step needed on Ubuntu Trusty if you add the Cloudera repository
If you install or upgrade CDH on Ubuntu Trusty using the command line, and add the Cloudera repository yourself (rather than using the "1-click Install" method) you need to perform an additional step to ensure that you get the CDH version of ZooKeeper, rather than the version that is bundled with Trusty. See Steps to Install CDH 5 Manually.
Cloudera Bug: CDH-23569
Upgrading hadoop-kms from 5.2.x and 5.3.x releases fails on SLES
Upgrading hadoop-kms fails on SLES when you try to upgrade an existing version from 5.2.x releases earlier than 5.2.4, and from 5.3.x releases earlier than 5.3.2. For details and troubleshooting instructions, see Troubleshooting: Upgrading hadoop-kms from 5.2.x and 5.3.x Releases on SLES.
Cloudera Bug: CDH-25053
Must build native libraries when installing from tarballs
When installing Hadoop from Cloudera tarballs, you must build your own native libraries. The tarballs do not include libraries that are built for the different distributions and architectures.
Installing from tarballs is deprecated and will be removed from CDH in a future release.
Cloudera Bugs: CDH-7304, CDH-5623