CDH 5 and Cloudera Manager 5 Requirements and Supported Versions

In an enterprise data hub, Cloudera Manager and CDH interact with several products such as Apache Accumulo, Apache Impala, Hue, Cloudera Search, and Cloudera Navigator. This guide provides information about which major and minor release version of a product is supported with which release version of CDH and Cloudera Manager.

Compatibility across different release versions of Cloudera Manager and CDH must be taken into account, especially when carrying out install/upgrade procedures.

JDK compatibility also varies across different Cloudera Manager and CDH versions. Certain versions of CDH 5 are compatible with both JDK 7 and JDK 8. In such cases, ensure all your services are deployed on the same major version. For example, you should not run Hadoop on JDK 7 while running Sqoop on JDK 8. Additionally, since Cloudera does not support mixed environments, all nodes in your cluster must be running the same major JDK version. Cloudera supports Oracle JDKs and, starting with Cloudera Enterprise 5.16, OpenJDK. For more information about the supported JDK versions, see CDH and Cloudera Manager Supported JDK Versions.

CDH Requirements for Cloudera Manager

Cloudera uses the following versioning convention: major.minor.maintenance. If a cluster runs Cloudera Manager 5.14.0, the major version is 5; the minor version is 14; and the maintenance version is 0.

The Cloudera Manager minor version must always be equal to or greater than the CDH minor version. Older versions of Cloudera Manager might not support features in newer versions of CDH.

For example, Cloudera Manager 5.12.0 can manage CDH 5.12.2 because the minor versions are equal. Cloudera Manager 5.12.0 cannot manage CDH 5.14.0 because the Cloudera Manager minor version, 12, is less than the CDH minor version, 14.

CDH and Cloudera Manager Supported Operating Systems

Points to note:

CDH and Cloudera Manager 5.16.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible

RHEL / CentOS / Oracle Linux RHCK

7.9, 7.8, 7.7, 7.6, 7.5, 7.4, 7.3, 7.2, 7.1

6.10, 6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.5, 7.4, 7.3, 7.2, 7.1 (UEK default)

6.10, 6.9, 6.8 (UEK R2, R4)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP5, 12 SP4*, 12 SP3, 12 SP2, 12 SP1

11 SP4, 11 SP3

Ubuntu/Debian
Ubuntu

16.04 LTS (Xenial)

14.04 LTS (Trusty)

Debian

8.9, 8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

* SLES 12 SP4 certification includes a known issue when MariaDB is being used as the Cloudera Manager server database: use MariaDB 10.0 instead of the default MariaDB 10.2 provided in the operating system release.

See OPSAPS-52340.

CDH and Cloudera Manager 5.15.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible

RHEL / CentOS / Oracle Linux RHCK

7.9, 7.8, 7.6, 7.5, 7.4, 7.3, 7.2, 7.1

6.10, 6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.5, 7.4, 7.3, 7.2, 7.1 (UEK default)

6.9, 6.8 (UEK R2, R4)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP3, 12 SP2, 12 SP1

11 SP4, 11 SP3

Ubuntu/Debian
Ubuntu

16.04 LTS (Xenial)

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.9, 8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.14.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible

RHEL / CentOS / Oracle Linux RHCK

7.6, 7.5, 7.4, 7.3, 7.2, 7.1

6.10, 6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.5, 7.4, 7.3, 7.2, 7.1 (UEK default)

6.9, 6.8 (UEK R2, R4)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP2, 12 SP1

11 SP4, 11 SP3

Ubuntu/Debian
Ubuntu

16.04 LTS (Xenial)

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.9, 8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.13.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible

RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 7.2

7.6, 7.5, 7.4, 7.3, 7.2, 7.1

6.10, 6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.5, 7.4, 7.3, 7.2, 7.1 (UEK default)

6.9, 6.8 (UEK R2, R4)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP2, 12 SP1

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

16.04 LTS (Xenial)

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.9, 8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.12.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible

RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 7.2

7.5, 7.4, 7.3, 7.2, 7.1

6.10, 6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.5, 7.4, 7.3, 7.2, 7.1 (UEK default)

6.9, 6.8 (UEK R2, R4)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP2, 12 SP1

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

16.04 LTS (Xenial)

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.9, 8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.11.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible

RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 7.2

7.5, 7.4, 7.3, 7.2, 7.1

6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.4, 7.3, 7.2, 7.1 (UEK default)

6.9, 6.8 (UEK R2, R4)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP2, 12 SP1

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

16.04 LTS (Xenial)

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

Operating System Support for CDSW Gateway Hosts (CDH 5.11 and higher only)

Cloudera Data Science Workbench is supported only on a subset of the operating systems supported by CDH and Cloudera Manager. For the complete list, see CDSW - Supported Operating Systems.

A CDSW Gateway host that is dedicated to running Cloudera Data Science Workbench must use an operating system supported by CDSW even if the remaining hosts in your cluster are running any of the other supported operating systems. All hosts must run the same version of a supported JDK.

CDH and Cloudera Manager 5.10.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.7

7.3, 7.2, 7.1

6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.3, 7.2, 7.1 (UEK default)

6.9, 6.8 (UEK R2, R4)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP1

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.9.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.7

7.2, 7.1

6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.2, 7.1 (UEK default)

6.9, 6.8 (UEK R2, R4)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

12 SP1

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.4, 8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.8.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.6

7.2, 7.1

6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.2, 7.1 (UEK default)

6.9 (UEK R2, R4)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

8.2 (Jessie)

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.7.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.6

7.2, 7.1

6.9, 6.8, 6.7, 6.6, 6.5, 6.4

5.11, 5.10, 5.7

Oracle Linux (OL)

7.2, 7.1 (UEK default)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.11, 5.10, 5.7 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP4, 11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.8, 7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.6.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.6

7.1

6.7, 6.6, 6.5, 6.4

5.10, 5.7

Oracle Linux (OL)

7.1 (UEK default)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.10, 5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.5.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.6

7.1

6.7, 6.6, 6.5, 6.4

5.10, 5.7

Oracle Linux (OL)

7.1 (UEK default)

6.7, 6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.10, 5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.4.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.5

6.6, 6.5, 6.4

5.10, 5.7

Oracle Linux (OL)

6.6, 6.5 (UEK R2, R3)

6.4 (UEK R2)

5.10, 5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.3.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.4

6.5, 6.4

5.7

Oracle Linux (OL)

6.5 (UEK R2, R3)

6.4 (UEK R2)

5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.2.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.4

6.5, 6.4

5.7

Oracle Linux (OL)

6.5 (UEK R2, R3)

6.4 (UEK R2)

5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

14.04 LTS (Trusty)

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.1.x Supported Operating Systems

Operating System Version (bold=new)
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.4

6.5, 6.4

5.7

Oracle Linux (OL)

6.5 (UEK R2, R3)

6.4 (UEK R2)

5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

CDH and Cloudera Manager 5.0.x Supported Operating Systems

Operating System Version
Red Hat Enterprise Linux-compatible
RHEL / CentOS / Oracle Linux RHCK

Max SE Linux support: 6.6

6.5, 6.4

5.7

Oracle Linux (OL)

6.5 (UEK R2, R3)

6.4 (UEK R2)

5.6 (UEK R2)

SUSE Linux Enterprise Server
SLES

11 SP3, 11 SP2

Ubuntu/Debian
Ubuntu

12.04 LTS (Precise)

Debian

7.1, 7.0 (Wheezy)

Filesystem Requirements

Supported Filesystems

The Hadoop Distributed File System (HDFS) is designed to run on top of an underlying filesystem in an operating system. Cloudera recommends that you use either of the following filesystems tested on the supported operating systems:

  • ext3: This is the most tested underlying filesystem for HDFS.
  • ext4: This scalable extension of ext3 is supported in more recent Linux releases.
  • XFS: This is the default filesystem in RHEL 7.

Kudu Filesystem Requirements - Kudu is supported on ext4 and XFS. Kudu requires a kernel version and filesystem that supports hole punching. Hole punching is the use of the fallocate(2) system call with the FALLOC_FL_PUNCH_HOLE option set. For more details, see Kudu troubleshooting - Error During Hole Punch Test.

File Access Time

Linux filesystems keep metadata that record when each file was accessed. This means that even reads result in a write to the disk. To speed up file reads, Cloudera recommends that you disable this option, called atime, using the mount option in /etc/fstab:

/dev/sdb1 /data1 ext4 defaults,noatime 0

Apply the change without rebooting:

mount -o remount /data1

Filesystem Mount Options

The filesystem mount options have a sync option that allows you to write synchronously.

However, using the sync option will lead to poor performance for services that write data to disks, such as HDFS, YARN, Kafka and Kudu. In CDH most writes are already replicated. Therefore, having synchronous writes to disks is unnecessary, expensive, and not worth the added safety it provides.

NFS and NAS options must not be used as DataNode Data Directory mounts, not even when using Hierarchal Storage features.

CDH and Cloudera Manager Supported Databases

Cloudera Manager and CDH come packaged with an embedded PostgreSQL database for use in a non-production environment. The embedded PostgreSQL database is not supported in production environments. For production environments, you must configure your cluster to use external databases.

In most cases (but not all), Cloudera supports versions of MariaDB, MySQL and PostgreSQL that are native to each supported Linux distribution.

After installing a database, upgrade to the latest patch and apply appropriate updates. Available updates may be specific to the operating system on which it is installed.

Notes:
  • Use UTF8 encoding for all custom databases.
  • For MySQL 5.6 and 5.7, you must install the MySQL-shared-compat or MySQL-shared package. This is required for the Cloudera Manager Agent package installation.
  • MySQL GTID-based replication is not supported.
  • Both the Community and Enterprise versions of MySQL are supported, as well as MySQL configured by the AWS RDS service.
  • Cloudera supports Oracle Exadata and RAC instances when they serve as backend databases for CDH components without high availability. Cloudera software is architected with the assumption of a single database instance, and supports normal operations between Cloudera Enterprise and Oracle Exadata (or RAC) in such an environment. Cloudera cannot reasonably test and certify on each strategy for each RDBMS. Cloudera expects HA solutions for RDBMS to be transparent to Cloudera software, and therefore are not supported and debugged by Cloudera. It is the responsibility of the customer to provision, configure, and manage the RDBMS HA deployment, so that Cloudera software behaves exactly as it would when interfacing with a single, non-HA service. Cloudera will support and help customers troubleshoot issues when a cluster has HA enabled. While diagnosing database-related problems in Cloudera components, customers may be required to temporarily disable or bypass HA mechanisms for troubleshooting purposes. If an HA-related issue is found, it is the responsibility of the customer to engage with the database vendor so that a solution to that issue can be found.
MySQL Support across CM/CDH 5 Releases
MySQL Cloudera Manager/CDH 5.10 - 5.16 Cloudera Manager/CDH 5.1 - 5.9 Cloudera Manager/CDH 5.0
5.1
5.5
5.6  
5.7    
MariaDB Support across CM/CDH 5 Releases
MariaDB Cloudera Manager/CDH 5.9 - 5.16 Cloudera Manager/CDH 5.5 - 5.8
5.5
10.0  
10.2 Recommend 10.0 because of the known issue OPSAPS-52340.
PostgreSQL Support across CM/CDH 5 Releases
PostgreSQL Cloudera Manager/CDH 5.16 Cloudera Manager/CDH 5.15 Cloudera Manager/CDH 5.13 - 5.14 Cloudera Manager/CDH 5.11 - 5.12 Cloudera Manager/CDH 5.5 - 5.10 Cloudera Manager/CDH 5.2 - 5.4 Cloudera Manager/CDH 5.1 Cloudera Manager/CDH 5.0
8.1
8.3
8.4
9.1
9.2  
9.3    
9.4      
9.5        
9.6          
10 See Warning below.            
Oracle Support across CM/CDH 5 Releases
Oracle Cloudera Manager/CDH 5.13 - 5.16 Cloudera Manager/CDH 5.6 - 5.12 Cloudera Manager/CDH 5.0 - 5.5
11g R2
12c R1  
12c R2    
18 Cloudera Manager/CDH 5.16 only. Requires Java 8 JDBC driver    
19 Cloudera Manager/CDH 5.16 only. Requires Java 8 JDBC driver    

CDH and Cloudera Manager Supported JDK Versions

Only 64 bit JDKs are supported. Oracle JDK 7 is supported across all versions of Cloudera Manager 5 and CDH 5. Oracle JDK 8 is supported in Cloudera Enterprise 5.3.x and higher.

OpenJDK 8 is supported in Cloudera Enterprise 5.16.1 and higher.

Oracle JDK 9 is not supported in any Cloudera Manager or CDH version.

Unless specifically excluded, Cloudera supports later updates to a major JDK release from the release that support was introduced. For example, Oracle JDK 8u102 was released prior to CDH 5.9, but is supported in CDH 5.3 and higher because support for Oracle JDK 8 was added in CDH 5.3. Cloudera excludes or removes support for select Java updates when security is jeopardized.

Running CDH nodes within the same cluster on different JDK releases is not supported. JDK release across a cluster needs to match the patch level.

JDK 7

All JDK 7 updates, from the minimum required version, are supported in Cloudera Manager/CDH 5.0 and higher unless specifically excluded. Updates above the minimum that are not listed are supported but not tested.

The Cloudera Manager repository contains Oracle JDK 1.7.0_67 (for example) and can be automatically installed during a new installation or an upgrade.

Oracle JDK 7 versions that are tested and recommended
Oracle JDK Version Notes
1.7u80 Recommended / Latest version tested
1.7u75 Recommended
1.7u67 Recommended
1.7u55 Minimum required

JDK 8

All Oracle JDK 8 updates, from the minimum required version, are supported in Cloudera Manager/CDH 5.3 and higher unless specifically excluded. Updates above the minimum that are not listed are supported but not tested.

Oracle JDK 8 versions that are tested and recommended
Oracle JDK Version Notes
1.8u181 Recommended / Latest version tested
1.8u162 Recommended / Latest version tested
1.8u144 Recommended
1.8u131 Recommended
1.8u121 Recommended
1.8u111 Recommended
1.8u102 Recommended
1.8u91 Recommended
1.8u74 Recommended
1.8u31 Minimum required
OpenJDK 1.8 versions that are tested and recommended
OpenJDK Version Notes
1.8u181 Minimum required / Latest version tested

Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Requirement

If you are using CentOS/Red Hat Enterprise Linux 5.6 or higher, or Ubuntu, you must install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy File on all cluster hosts (including gateway hosts). This is to ensure that JDK uses the same default encryption type (aes256-cts) as that used by Red Hat/CentOS operating systems, Kerberos, and the rest of the CDH cluster.

On a managed deployment, Cloudera Manager can install the JCE on hosts. For unmanaged deployments, use the installation instructions in the README.txt file included in jce_policy-x.zip.

Enabling Unlimited Strength Encryption for JDK 1.8.0_151 (and later)

  • As of JDK 1.8.0_151, unlimited strength encryption can be enabled using the java.security file as documented in the JDK 1.8.0_151 release notes. You do not need to install the JCE policy files.

  • As of JDK 1.8.0_161, unlimited strength encryption has been enabled by default. No further action is required.

Cloudera Manager Supported Browsers

The Cloudera Manager Admin Console, which you use to install, configure, manage, and monitor services, supports the latest version of the following browsers:
  • Mozilla Firefox
  • Google Chrome
  • Internet Explorer
  • Safari.

Hue Supported Browsers

Hue works with the two most recent LTS (long term support) or ESR (extended support release) browsers. Cookies and JavaScript must be on. Hue can display in older, and other, browsers, but you might not have access to all of its features.

CDH and Cloudera Manager Supported Transport Layer Security Versions

The following components are supported by the indicated versions of Transport Layer Security (TLS):

Components Supported by TLS

Component

Role Name Port Version
Cloudera Manager Cloudera Manager Server   7182 TLS 1.2
Cloudera Manager Cloudera Manager Server   7183 TLS 1.2
      9099 TLS 1.2
Flume   Avro Source/Sink   TLS 1.2
Flume   Flume HTTP Source/Sink   TLS 1.2
HBase Master HBase Master Web UI Port 60010 TLS 1.2
HDFS NameNode Secure NameNode Web UI Port 50470 TLS 1.2
HDFS Secondary NameNode Secure Secondary NameNode Web UI Port 50495 TLS 1.2
HDFS HttpFS REST Port 14000 TLS 1.1, TLS 1.2
Hive HiveServer2 HiveServer2 Port 10000 TLS 1.2
Hue Hue Server Hue HTTP Port 8888 TLS 1.2
Impala Impala Daemon Impala Daemon Beeswax Port 21000 TLS 1.0, TLS 1.1, TLS 1.2

We recommend that clients use the highest supported version, TLS 1.2.

Impala Impala Daemon Impala Daemon HiveServer2 Port 21050 TLS 1.0, TLS 1.1, TLS 1.2

We recommend that clients use the highest supported version, TLS 1.2.

Impala Impala Daemon Impala Daemon Backend Port 22000 TLS 1.0, TLS 1.1, TLS 1.2

We recommend that clients use the highest supported version, TLS 1.2.

Impala Impala StateStore StateStore Service Port 24000 TLS 1.0, TLS 1.1, TLS 1.2

We recommend that clients use the highest supported version, TLS 1.2.

Impala Impala Daemon Impala Daemon HTTP Server Port 25000 TLS 1.0, TLS 1.1, TLS 1.2

We recommend that clients use the highest supported version, TLS 1.2.

Impala Impala StateStore StateStore HTTP Server Port 25010 TLS 1.0, TLS 1.1, TLS 1.2

We recommend that clients use the highest supported version, TLS 1.2.

Impala Impala Catalog Server Catalog Server HTTP Server Port 25020 TLS 1.0, TLS 1.1, TLS 1.2

We recommend that clients use the highest supported version, TLS 1.2.

Impala Impala Catalog Server Catalog Server Service Port 26000 TLS 1.0, TLS 1.1, TLS 1.2

We recommend that clients use the highest supported version, TLS 1.2.

Oozie Oozie Server Oozie HTTPS Port 11443 TLS 1.1, TLS 1.2
Solr Solr Server Solr HTTP Port 8983 TLS 1.1, TLS 1.2
Solr Solr Server Solr HTTPS Port 8985 TLS 1.1, TLS 1.2
Spark History Server   18080 TLS 1.2
YARN ResourceManager ResourceManager Web Application HTTP Port 8090 TLS 1.2
YARN JobHistory Server MRv1 JobHistory Web Application HTTP Port 19890 TLS 1.2

Cloudera Manager Resource Requirements

Cloudera Manager requires the following resources:
  • Disk Space
    • Cloudera Manager Server
      • 5 GB on the partition hosting /var.
      • 500 MB on the partition hosting /usr.
      • For parcels, the space required depends on the number of parcels you download to the Cloudera Manager Server and distribute to Agent hosts. You can download multiple parcels of the same product, of different versions and different builds. If you are managing multiple clusters, only one parcel of a product/version/build/distribution is downloaded on the Cloudera Manager Server—not one per cluster. In the local parcel repository on the Cloudera Manager Server, the approximate sizes of the various parcels are as follows:
        • CDH 5 (which includes Impala and Search) - 1.5 GB per parcel (packed), 2 GB per parcel (unpacked)
        • Impala - 200 MB per parcel
        • Cloudera Search - 400 MB per parcel
    • Cloudera Management Service -The Host Monitor and Service Monitor databases are stored on the partition hosting /var. Ensure that you have at least 20 GB available on this partition.
    • Agents - On Agent hosts, each unpacked parcel requires about three times the space of the downloaded parcel on the Cloudera Manager Server. By default, unpacked parcels are located in /opt/cloudera/parcels.
  • RAM - 4 GB is recommended for most cases and is required when using Oracle databases. 2 GB might be sufficient for non-Oracle deployments with fewer than 100 hosts. However, to run the Cloudera Manager Server on a machine with 2 GB of RAM, you must tune down its maximum heap size (by modifying -Xmx in /etc/default/cloudera-scm-server). Otherwise the kernel might kill the Server for consuming too much RAM.
  • Python - Cloudera Manager requires Python 2.4 or higher (but is not compatible with Python 3.0 or higher). Hue in CDH 5 and package installs of CDH 5 require Python 2.6 or 2.7. All supported operating systems include Python version 2.4 or higher. Cloudera Manager is compatible with Python 2.4 through the latest version of Python 2.x. Cloudera Manager does not support Python 3.0 and higher.
  • Perl - Cloudera Manager requires perl.
  • python-psycopg2 package - Cloudera Manager 5.8 and higher has a new dependency on the package python-psycopg2. This package is not available in standard SLES 11 and SLES 12 repositories. You need to add this repository or install it manually to any machine that runs the Cloudera Manager Agent before you install or upgrade Cloudera Manager.

    If the Cloudera Manager Server and Agent run on the same host, install the Cloudera Manager Server first and then add the python-psycopg2 repository or package. After adding the repository or package, install the Cloudera Manager Agent.

    Download the python-psycopg2 repository or package from the following URL by selecting the correct SLES version: https://software.opensuse.org/package/python-psycopg2.

  • iproute package - Cloudera Manager 5.12 and higher has a new dependency on the iproute package. Any machine that runs the Cloudera Manager Agent requires the package. Which version you need depends on the operating system:
    iproute package
    Operating System iproute version
    CentOS 6.x iproute-2.6
    CentOS 7.x iproute-3.10
    RHEL 6.x iproute-2.6
    RHEL 7.x iproute-3.10
    SLES 11 SP3 iproute2-2.6
    SLES 11 SP4 iproute2-3.0
    SLES 12 and 12 SP1 iproute2-3.12
    SLES 12 SP4 iproute2-4.4

CDH and Cloudera Manager Networking and Security Requirements

The hosts in a Cloudera Manager deployment must satisfy the following networking and security requirements:

  • Networking Protocols Support
    CDH requires IPv4. IPv6 is not supported and must be disabled.

    See also Configuring Network Names.

  • Multihoming Support

    Multihoming CDH or Cloudera Manager is not supported outside specifically certified Cloudera partner appliances. Cloudera finds that current Hadoop architectures combined with modern network infrastructures and security practices remove the need for multihoming. Multihoming, however, is beneficial internally in appliance form factors to take advantage of high-bandwidth InfiniBand interconnects.

    Although some subareas of the product may work with unsupported custom multihoming configurations, there are known issues with multihoming. In addition, unknown issues may arise because multihoming is not covered by our test matrix outside the Cloudera-certified partner appliances.

  • Entropy

    Data at rest encryption requires sufficient entropy to ensure randomness.

    See entropy requirements in Data at Rest Encryption Requirements.

  • Cluster hosts must have a working network name resolution system and correctly formatted /etc/hosts file. All cluster hosts must have properly configured forward and reverse host resolution through DNS. The /etc/hosts files must:
    • Contain consistent information about hostnames and IP addresses across all hosts
    • Not contain uppercase hostnames
    • Not contain duplicate IP addresses

    Cluster hosts must not use aliases, either in /etc/hosts or in configuring DNS. A properly formatted /etc/hosts file should be similar to the following example:

    127.0.0.1 localhost.localdomain localhost
    192.168.1.1 cluster-01.example.com cluster-01
    192.168.1.2 cluster-02.example.com cluster-02
    192.168.1.3 cluster-03.example.com cluster-03 
  • In most cases, the Cloudera Manager Server must have SSH access to the cluster hosts when you run the installation or upgrade wizard. You must log in using a root account or an account that has password-less sudo permission. For authentication during the installation and upgrade procedures, you must either enter the password or upload a public and private key pair for the root or sudo user account. If you want to use a public and private key pair, the public key must be installed on the cluster hosts before you use Cloudera Manager.

    Cloudera Manager uses SSH only during the initial install or upgrade. Once the cluster is set up, you can disable root SSH access or change the root password. Cloudera Manager does not save SSH credentials, and all credential information is discarded when the installation is complete.

  • If single user mode is not enabled, the Cloudera Manager Agent runs as root so that it can make sure the required directories are created and that processes and files are owned by the appropriate user (for example, the hdfs and mapred users).
  • No blocking is done by Security-Enhanced Linux (SELinux).
  • No blocking by iptables or firewalls; port 7180 must be open because it is used to access Cloudera Manager after installation. Cloudera Manager communicates using specific ports, which must be open.
  • For RHEL and CentOS, the /etc/sysconfig/network file on each host must contain the hostname you have just set (or verified) for that host.
  • Cloudera Manager and CDH use several user accounts and groups to complete their tasks. The set of user accounts and groups varies according to the components you choose to install. Do not delete these accounts or groups and do not modify their permissions and rights. Ensure that no existing systems prevent these accounts and groups from functioning. For example, if you have scripts that delete user accounts not in a whitelist, add these accounts to the list of permitted accounts. Cloudera Manager, CDH, and managed services create and use the following accounts and groups:
Users and Groups

Component (Version)

Unix User ID Groups Functionality
Cloudera Manager (all versions) cloudera-scm cloudera-scm Clusters managed by Cloudera Manager run Cloudera Manager Server, monitoring roles, and other Cloudera Server processes as cloudera-scm.

Requires keytab file named cmf.keytab because name is hard-coded in Cloudera Manager.

Apache Accumulo (Accumulo 1.4.3 and higher) accumulo accumulo Accumulo processes run as this user.
Apache Avro ~ ~ No special user:group.
Apache Flume flume flume The sink that writes to HDFS as user must have write privileges.
Apache HBase hbase hbase The Master and the RegionServer processes run as this user.
HDFS hdfs hdfs, hadoop The NameNode and DataNodes run as this user, and the HDFS root directory as well as the directories used for edit logs should be owned by it.
Apache Hive hive hive

The HiveServer2 process and the Hive Metastore processes run as this user.

A user must be defined for Hive access to its Metastore DB (for example, MySQL or Postgres) but it can be any identifier and does not correspond to a Unix uid. This is javax.jdo.option.ConnectionUserName in hive-site.xml.

Apache HCatalog hive hive

The WebHCat service (for REST access to Hive functionality) runs as the hive user.

HttpFS httpfs httpfs

The HttpFS service runs as this user. See HttpFS Security Configuration for instructions on how to generate the merged httpfs-http.keytab file.

Hue hue hue

Hue services run as this user.

Hue Load Balancer (Cloudera Manager 5.5 and higher) apache apache The Hue Load balancer has a dependency on the apache2 package that uses the apache user name. Cloudera Manager does not run processes using this user ID.
Impala impala impala, hive Impala services run as this user.
Apache Kafka (CDK 1.2.0 Powered By Apache Kafka) kafka kafka Kafka services run as this user.
Java KeyStore KMS (CDH 5.2.1 and higher) kms kms The Java KeyStore KMS service runs as this user.
Key Trustee KMS (CDH 5.3 and higher) kms kms The Key Trustee KMS service runs as this user.
Key Trustee Server (CDH 5.4 and higher) keytrustee keytrustee The Key Trustee Server service runs as this user.
Kudu kudu kudu Kudu services run as this user.
Llama llama llama Llama runs as this user.
Apache Mahout ~ ~ No special users.
MapReduce mapred mapred, hadoop Without Kerberos, the JobTracker and tasks run as this user. The LinuxTaskController binary is owned by this user for Kerberos.
Apache Oozie oozie oozie The Oozie service runs as this user.
Parquet ~ ~ No special users.
Apache Pig ~ ~ No special users.
Cloudera Search solr solr The Solr processes run as this user.
Apache Spark spark spark The Spark History Server process runs as this user.
Apache Sentry sentry sentry The Sentry service runs as this user.
Apache Sqoop sqoop sqoop This user is only for the Sqoop1 Metastore, a configuration option that is not recommended.
Apache Sqoop2 sqoop2 sqoop, sqoop2 The Sqoop2 service runs as this user.
Apache Whirr ~ ~ No special users.
YARN yarn yarn, hadoop Without Kerberos, all YARN services and applications run as this user. The LinuxContainerExecutor binary is owned by this user for Kerberos.
Apache ZooKeeper zookeeper zookeeper The ZooKeeper processes run as this user. It is not configurable.

Product Compatibility Matrix for Apache Accumulo

This matrix contains compatibility information across versions of Apache Accumulo, and CDH and Cloudera Manager. For detailed information on each release, see Apache Accumulo documentation.
Product Lowest supported Cloudera Manager Version Lowest supported CDH Version Lowest supported Impala Version Lowest supported Search Version Integrated into CDH
Accumulo 1.7.2 Cloudera Manager 5.0.0 CDH 5.5.0 Not Supported Not Supported No
Accumulo 1.6.0 Cloudera Manager 5.0.0 CDH 4.6.0 - 4.x.x, CDH 5.1.0 Not Supported Not Supported No
Accumulo 1.4.4 Cloudera Manager 5.0.0 CDH 4.5.0 (Not for use with CDH 5) Not Supported Not Supported No
Accumulo 1.4.3 Cloudera Manager 5.0.0 CDH 4.3.0 (Not for use with CDH 5) Not Supported Not Supported No

Product Compatibility Matrix for Apache Impala

This matrix contains compatibility information across versions of Impala and CDH/Cloudera Manager. For detailed information on each release, see the Impala documentation.

Product Supported Cloudera Manager Versions Supported CDH Versions Integrated into CDH
Impala 2.12.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.15.x CDH 5.15.x
Impala 2.11.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.14.x CDH 5.14.x
Impala 2.10.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.13.x CDH 5.13.x
Impala 2.9.x (plus additional patches) for CDH 5.12.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.12.x CDH 5.12.x
Impala 2.8.x (plus additional patches) for CDH 5.11.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.11.x CDH 5.11.x
Impala 2.8.x (plus a few minor patches) for CDH 5.10.x; although version string still says Impala 2.7

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.10.x CDH 5.10.x
Impala 2.7.x for CDH 5.9.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.9.x CDH 5.9.x
Impala 2.6.x for CDH 5.8.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.8.x CDH 5.8.x
Impala 2.5.x for CDH 5.7.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.7.x CDH 5.7.x
Impala 2.4.x for CDH 5.6.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.6.x CDH 5.6.x
Impala 2.3.x for CDH 5.5.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.5.x CDH 5.5.x
Impala 2.2.x for CDH 5.4.x

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.4.x CDH 5.4.x
Impala 2.2.0

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.4.0 CDH 5.4.0
Impala 2.1.8

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.10 CDH 5.3.10
Impala 2.1.7

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.9 CDH 5.3.9
Impala 2.1.6

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.8 CDH 5.3.8
Impala 2.1.5

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.6 CDH 5.3.6
Impala 2.1.4

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.4 and CDH 5.3.5 CDH 5.3.4 and CDH 5.3.5
Impala 2.1.3

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.3 CDH 5.3.3
Impala 2.1.2

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.2 CDH 5.3.2
Impala 2.1.1

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.1 CDH 5.3.1
Impala 2.1.0

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.3.0 CDH 5.3.0
Impala 2.0.5

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.6 CDH 5.2.6
Impala 2.0.4

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.5 CDH 5.2.5
Impala 2.0.3

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.4 CDH 5.2.4
Impala 2.0.2

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.3 CDH 5.2.3
Impala 2.0.1

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.1 CDH 5.2.1
Impala 2.0.0

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.2.0 CDH 5.2.0
Impala 1.4.4

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.1.5 CDH 5.1.5
Impala 1.4.3

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.1.4 CDH 5.1.4
Impala 1.4.2

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.1.3 CDH 5.1.3
Impala 1.4.1

Cloudera Manager 5.0.0 - 5.x.x

CDH 5.1.2 CDH 5.1.2
Impala 1.4.0

Cloudera Manager 5.0.0 - 5.x.x; Recommended: Cloudera Manager 5.1.0

CDH 5.1.0 CDH 5.1.0
Impala 1.3.3 Cloudera Manager 5.0.0 - 5.x.x CDH 5.0.5 CDH 5.0.5
Impala 1.3.2 Cloudera Manager 5.0.0 - 5.x.x CDH 5.0.4 CDH 5.0.4
Impala 1.3.1 Cloudera Manager 5.0.0 - 5.x.x CDH 5.0.1 - 5.0.x CDH 5.0.1
Impala 1.3.0 Cloudera Manager 5.0.0 - 5.x.x CDH 5.0.0 CDH 5.0.0

Product Compatibility Matrix for CDK Powered By Apache Kafka

CDK Powered By Apache Kafka is currently distributed as a package and in a parcel that is independent of the CDH parcel. The parcel integrates with Cloudera Manager using a Custom Service Descriptor (CSD). Using the latest features of Kafka sometimes requires a newer version of Cloudera Manager and/or CDH that supports the feature. This matrix describes the compatibility between CDK Powered by Apache Kafka and Cloudera Manager and CDH.

For the latest documentation, see Kafka Documentation.

CDK Powered By Apache Kafka Version Upstream Kafka Version Minimum Supported CM Version Minimum Supported CDH Version (No Security) Minimum Supported CDH Version (with Sentry) Feature Notes
4.1.0 2.2.1 5.13.0 CDH 5.13.0 CDH 5.13.0  
4.0.0 2.1.0 5.13.0 CDH 5.13.0 CDH 5.13.0  
3.1.1 1.0.1 5.13.0 CDH 5.13.0 CDH 5.13.0  
3.1.0 1.0.1 5.13.0 CDH 5.13.0 CDH 5.13.0 Sentry-HA Supported
3.0.0 0.11.0 5.13.0 CDH 5.13.0 CDH 5.13.0 Sentry-HA not supported
2.2.x 0.10.2 5.9.0 CDH 5.4.0 CDH 5.9.0  
2.1.x 0.10.0 5.9.0 CDH 5.4.0 CDH 5.9.0 Sentry Authorization
2.0.x 0.9.0 5.5.3 CDH 5.4.0 CDH 5.4.0 Enhanced Security

Kafka Client-Broker Compatibility across Kafka Versions

Maintaining compatibility across different Kafka clients and brokers is a common issue. This can occur as part of any of the following scenarios:

  • Upgrading your Kafka cluster without upgrading your Kafka clients
  • Using a third-party application that produces to or consumes from your Kafka cluster
  • Having a client program communicating with two or more Kafka clusters (e.g. consuming from one cluster and producing to a different cluster)
  • Using Flume or Spark as a Kafka consumer

In these cases, it is important to understand client/broker compatibility across Kafka versions. Here are general rules that apply:

  1. Newer Kafka brokers can talk to older Kafka clients. The reverse is not true.
  2. Changes in either the major part or the minor part of the upstream version major.minor determines whether the client and broker are compatible.

As a result, the general pattern for upgrading Kafka from version A to version B is:

  1. Change Kafka server.properties to refer to version A.
  2. Upgrade the brokers to version B and restart the cluster.
  3. Upgrade the clients to version B.
  4. After all the clients are upgraded, change the properties from the first step to version B and restart the cluster.

Kafka Client Versions Used by Apache Flume in CDH

Flume has an embedded Kafka client which it uses to talk to Kafka clusters. Since the generally accepted practice is to have the broker running the same or newer version as the client, a CDH Flume version requires being matched to a minimum Kafka version. This matrix lists the CDK Powered By Apache Kafka (CDK) client versions used by Apache Flume in CDH, as well as the underlying Apache Kafka base version. For more information about CDK, see the Cloudera Distribution of Kafka Documentation.

Flume CDH Version Minimum Supported CDK Client Version (Remote or Local) Embedded Kafka Client Base Version
5.16.x 2.2.0 0.10.2.0
5.15.x 2.2.0 0.10.2.0
5.14.x 2.2.0 0.10.2.0
5.13.x 2.0.2 0.9.0.0
5.12.x 2.0.2 0.9.0.0
5.11.x 2.0.2 0.9.0.0
5.10.x 2.0.2 0.9.0.0
5.9.x 2.0.2 0.9.0.0
5.8.x 2.0.0 0.9.0.0
5.7.x 2.0.0 0.9.0.0

Kafka Client Versions used by Apache Spark in CDH

Spark maintains two embedded Kafka clients and can be configured to use either one. For information on compatibility with Kafka brokers, refer to the appropriate entry in the table below.

CDS Version Embedded Kafka Client (Choose based on Kafka cluster) Minimum Apache Kafka Broker Version Minimum CDK Broker Version Upstream Integration Guide API Stability
CDS 2.4 spark-streaming-kafka-0-10 0.10.0 Kafka 2.1 Spark 2.4 + Kafka 0.10 Experimental
spark-streaming-kafka-0-8 0.8.2.1 Kafka 2.0 Spark 2.4 + Kafka 0.8 Stable
CDS 2.3 spark-streaming-kafka-0-10 0.10.0 Kafka 2.1 Spark 2.3 + Kafka 0.10 Experimental
spark-streaming-kafka-0-8 0.8.2.1 Kafka 2.0 Spark 2.3 + Kafka 0.8 Stable
CDS 2.2 spark-streaming-kafka-0-10 0.10.0 Kafka 2.1 Spark 2.2 + Kafka 0.10 Experimental
spark-streaming-kafka-0-8 0.8.2.1 Kafka 2.0 Spark 2.2 + Kafka 0.8 Stable
CDS 2.1 spark-streaming-kafka-0-10 0.10.0 Kafka 2.1 Spark 2.1 + Kafka 0.10 Experimental
spark-streaming-kafka-0-8 0.8.2.1 Kafka 2.0 Spark 2.1 + Kafka 0.8 Stable
CDS 2.0 spark-streaming-kafka-0-8 0.8.2.x Kafka 2.0 Spark 2.0 + Kafka Stable

Product Compatibility Matrix for Apache Kudu

This matrix contains compatibility information across versions of Kudu and CDH/Cloudera Manager. Starting with Apache Kudu 1.5.0 / CDH 5.13.0, Kudu is fully integrated into CDH and ships as part of the CDH parcel and packages.

Documentation for Kudu 1.5.0 / CDH 5.13.x (and higher) can be found at Apache Kudu Guide. Documentation for all lower versions can be found at Apache Kudu 1.4.0 (and lower).

Product Feature Lowest Supported Cloudera Manager Version Supported CDH Versions Integrated into CDH
Kudu 1.7.0 / CDH 5.15.x Fully integrated into the CDH 5.15 parcel and packages. Cloudera Manager 5.15.0 CDH 5.15.x Yes
Kudu 1.6.0 / CDH 5.14.x Fully integrated into the CDH 5.14 parcel and packages. Cloudera Manager 5.14.0 CDH 5.14.x Yes
Kudu 1.5.0 / CDH 5.13.x Fully integrated into the CDH 5.13 parcel and packages. Cloudera Manager 5.13.0 CDH 5.13.x Yes
Kudu 1.4.0 / CDH 5.12.x Available as a CSD within Cloudera Manager. Cloudera Manager 5.12.0 CDH 5.12.x No
Kudu 1.3.0 / CDH 5.11.x Available as a CSD within Cloudera Manager. Cloudera Manager 5.11.0 CDH 5.11.x No
Kudu 1.2.0 / CDH 5.10.x Distributed both as package and parcel.

Kudu's first generally available (GA) release that is supported on Cloudera Manager and CDH 5.10.x clusters.

Cloudera Manager 5.10.0 CDH 5.10.x No
Kudu 1.1.x (and lower) - Beta Public beta releases that are not commercially supported on CDH clusters. N/A N/A No

Supported Operating Systems (Kudu 1.2.0 / CDH 5.10.x and higher)

Starting in CDH 5.13, Kudu supports the same OSs and versions as Cloudera Manager and CDH with the following exceptions.

  • RHEL 5 is not supported.
  • Ubuntu 12 is not supported.
  • Debian 7 is not supported.
  • macOS
    • 10.10 Yosemite, 10.11 El Capitan, and 10.12 Sierra are supported.
    • Pre-built macOS packages are not provided.
  • Microsoft Windows is not supported.

For a complete list of supported versions, see CDH and Cloudera Manager Supported Operating Systems.

Filesystem Requirements

Kudu is supported on ext4 and XFS. Kudu requires a kernel version and filesystem that supports hole punching. Hole punching is the use of the fallocate(2) system call with the FALLOC_FL_PUNCH_HOLE option set. For more details, see Kudu troubleshooting - Error During Hole Punch Test.

nscd Recommendation

Although not a strict requirement, it's highly recommended that you use nscd to cache both DNS name resolution and static name resolution for Kudu. See Troubleshooting Kudu for more information.

JDK Requirements

  • JDK 7
  • JDK 8

    Note that Spark 2.2 (and higher) requires Java 8 at runtime even though the Kudu Spark 2.x integration is Java 7 compatible. Spark 2.2 is the default dependency version as of Kudu 1.5.0 / CDH 5.13.

Hardware Requirements

  • Kudu currently requires a CPU that supports the SSSE3 and SSE4.2 instruction sets. If you are to run Kudu inside a VM, enable SSE4.2 pass-through to pass through SSE4.2 support into the VM.
  • One or more hosts to run Kudu masters. You should have either one master (provides no fault tolerance), three masters (can tolerate one failure), or five masters (can tolerate two failures).
  • One or more hosts to run Kudu tablet servers. With replication, a minimum of three tablet servers is necessary.
  • Storage - If solid state storage is available, storing Kudu WALs on such high-performance media might significantly improve latency when Kudu is configured for its highest durability levels.

Product Compatibility Matrix for Cloudera Navigator

This matrix contains compatibility information across versions of Cloudera Navigator, Cloudera Manager, and CDH. For detailed information on each release, see Cloudera Navigator documentation.
Product Feature Lowest supported Cloudera Manager Version Lowest supported CDH Version Lowest supported Impala Version Lowest supported Search Version
Cloudera Navigator 2.15.x Auditing, Metadata, Analytics, and Security 5.16.0
  • Audit Component
    • HDFS, HBase - 5.0.0
    • Hue - 5.5.0 for LDAP user operations
    • HiveServer2 - 5.0.0
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 5.0.0
    • Pig - 5.0.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark - 1.6, 2.3
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.14.x Auditing, Metadata, Analytics, and Security 5.15.0
  • Audit Component
    • HDFS, HBase - 5.0.0
    • Hue - 5.5.0 for LDAP user operations
    • HiveServer2 - 5.0.0
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 5.0.0
    • Pig - 5.0.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark - 1.6, 2.3
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.13.x Auditing, Metadata, Analytics, and Security 5.14.0
  • Audit Component
    • HDFS, HBase - 5.0.0
    • Hue - 5.5.0 for LDAP user operations
    • HiveServer2 - 5.0.0
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 5.0.0
    • Pig - 5.0.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark - 1.6, 2.3
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.12.x Auditing, Metadata, Analytics, and Security 5.13.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • HiveServer2 - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark - 1.6
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.11.x Auditing, Metadata, Analytics, and Security 5.12.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • HiveServer2 - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark - 1.6
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.10.x Auditing, Metadata, Analytics, and Security 5.11.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • HiveServer2 - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark - 1.6
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.9.x Auditing, Metadata, Analytics, and Security 5.10.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • HiveServer2 - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark (available but unsupported) - 5.4.0
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.8.x Auditing, Metadata, Analytics, and Security 5.9.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • HiveServer2 - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, HiveServer2, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark (available but unsupported) - 5.4.0
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.7.x Auditing, Metadata, Analytics, and Security 5.8.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark (available but unsupported)- 5.4.0
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.6.x Auditing, Metadata, Analytics, and Security 5.7.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.4.0, 5.5.0 for LDAP user operations
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala - 5.4.0
    • Spark (available but unsupported)- 5.4.0
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.5.x Auditing, Metadata, Analytics, and Security 5.6.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Impala, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala and Spark (available but unsupported)- 5.4.0
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.4.x Auditing, Metadata, Analytics, and Security 5.5.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Impala, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala and Spark (available but unsupported)- 5.4.0
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.3.x Auditing, Metadata, and Security 5.4.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Impala, MapReduce, Oozie, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
    • Impala and Spark (available but unsupported)- 5.4.0
Impala 1.3.0 with CDH 5.0.0 CDH 5.4.0
Cloudera Navigator 2.2.x Auditing, Metadata, and Security 5.3.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
Impala 1.3.0 with CDH 5.0.0 Not Supported
Cloudera Navigator 2.1.x Auditing, Metadata, and Security 5.2.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
Impala 1.3.0 with CDH 5.0.0 Not Supported
Cloudera Navigator 2.0.1 Auditing, Metadata, and Security 5.1.2
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
Impala 1.3.0 with CDH 5.0.0 Not Supported
Cloudera Navigator 2.0.0 Auditing, Metadata, and Security 5.1.0
  • Audit Component
    • HDFS, HBase - 4.0.0
    • Hue - 4.2.0
    • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
    • Sentry - 5.1.0
  • Metadata Component
    • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
    • Pig - 4.6.0
    • YARN - 5.0.0
Impala 1.2.1 with CDH 4.4.0 Not Supported
Cloudera Navigator 1.2.x Auditing 5.0.0
  • HDFS, HBase - 4.0.0
  • Hue - 4.2.0
  • Hive - 4.2.0, 4.4.0 for operations denied due to lack of privileges.
Impala 1.3.0 with CDH 5.0.0 Not Supported
Metadata (2.0 beta 2) 5.0.0
  • HDFS, Hive, Oozie, MapReduce, Sqoop 1 - 4.4.0
  • Pig - 4.6.0
Not Supported Not Supported

Cloudera Navigator Supported Databases

Cloudera Navigator supports the same databases as CDH and Cloudera Manager. For more information, see CDH and Cloudera Manager Supported Databases.

Cloudera Navigator Supported Browsers

The Cloudera Navigator console supports the following browsers:
  • Mozilla Firefox 24 and higher
  • Google Chrome 36 and higher
  • Internet Explorer 11
  • Safari 5 and higher

Cloudera Navigator Supported CDH and Managed Service Versions

This section describes the CDH and managed service versions supported by the Cloudera Navigator auditing and metadata features.

Cloudera Navigator Auditing

This section describes the audited operations and service versions supported by Cloudera Navigator auditing.
Component Operations (For details, see Service Audit Events). Minimum Supported Service Version
HDFS
  • Operations that access or modify a file's or directory's data or metadata
  • Operations denied due to lack of privileges
CDH 5.0.0
HBase
  • In simple authentication mode, if the HBase Secure RPC Engine property is false (the default), the username in log events is UNKNOWN. To see a meaningful username:
    1. Click the HBase service.
    2. Click the Configuration tab.
    3. Select Service-wide > Security.
    4. Set the HBase Secure RPC Engine property to true.
    5. Save the change and restart the service.
CDH 5.0.0
Hive
  • Operations (except grant, revoke, and metadata access only) sent to HiveServer2
  • Operations denied due to lack of privileges
Limitations:
  • Actions taken against Hive using the Hive CLI are not audited. Therefore if you have enabled auditing you should disable the Hive CLI to prevent actions against Hive that are not audited.
  • In simple authentication mode, the username in log events is the username passed in the HiveServer2 connect command. If you do not pass a username in the connect command, the username is log events is anonymous.
CDH 5.0.0
Hue
  • Operations (except grant, revoke, and metadata access only) sent through the Beeswax Server
  • User operations such as log in, log out, add and remove user, add and remove LDAP group, add and remove user from LDAP group
CDH 5.0.0
Impala
  • Queries denied due to lack of privileges
  • Queries that pass analysis
Impala 1.3.0 with CDH 5.0.0
Navigator Metadata Server
  • Viewing and changing audit reports
  • Viewing and changing authorization configurations
  • Viewing and changing metadata
  • Viewing and changing policies
  • Viewing and changing saved searches
Cloudera Navigator 2.3
Sentry
  • Operations sent to the HiveServer2 and Hive Metastore Server roles and Impala service
  • Adding and deleting roles, assigning roles to groups and removing roles from groups, creating and deleting privileges, granting and revoking privileges
  • Operations denied due to lack of privileges
You do not directly configure the Sentry service for auditing. Instead, when you configure the Hive and Impala services for auditing, grant, revoke, and metadata operations appear in the Hive or Impala service audit logs.
CDH 5.1.0
Solr
  • Index creation and deletion
  • Schema and configuration file modification
  • Index, service, document tag access
CDH 5.4.0

Cloudera Navigator Metadata

This section describes the CDH and managed service versions supported by the Cloudera Navigator metadata feature.
Component Minimum Supported Version
HDFS. However, federated HDFS is not supported. CDH 5.0.0
Hive CDH 5.0.0
Impala CDH 5.4.0
MapReduce CDH 5.0.0
Oozie. Supported actions:
  • 2.4 - map-reduce, pig, hive, hive2, sqoop
  • 2.3 and lower - map-reduce, pig, hive, sqoop

Unsupported actions include email, shell, and ssh.

CDH 5.0.0
Pig CDH 5.0.0
Spark CDH 5.4.0
Sqoop 1. All Cloudera connectors are supported. CDH 5.0.0
YARN CDH 5.0.0

Data at Rest Encryption Requirements

Encryption comprises several components, each with its own requirements.

Data at rest encryption protection can be applied at a number of levels within Hadoop:

  • OS filesystem-level
  • Network-level
  • HDFS-level (protects both data at rest and in transit)

This section contains the various hardware and software requirements for all encryption products used for Data at Rest Encryption.

For more information on supported operating systems, see Product Compatibility Matrix for Cloudera Navigator Encryption.

For more information on the components, concepts, and architecture for encrypting data at rest, see Encrypting Data at Rest.

Entropy Requirements

See Entropy Requirements.

Cloudera Manager Requirements

Installing and managing Key Trustee Server using Cloudera Manager requires Cloudera Manager 5.4.0 and higher. Key Trustee Server does not require Cloudera Navigator Audit Server or Metadata Server.

umask Requirements

Key Trustee Server installation requires the default umask of 0022.

Network Requirements

For new Key Trustee Server installations (5.4.0 and higher) and migrated upgrades (see Migrate Apache Web Server to CherryPy for more information), Key Trustee Server requires the following TCP ports to be opened for inbound traffic:

  • 11371

    Clients connect to this port over HTTPS.

  • 11381 (PostgreSQL)

    The passive Key Trustee Server connects to this port for database replication.

For upgrades that are not migrated to the CherryPy web server, the pre-upgrade port settings are preserved:

  • 80

    Clients connect to this port over HTTP to obtain the Key Trustee Server public key.

  • 443 (HTTPS)

    Clients connect to this port over HTTPS.

  • 5432 (PostgreSQL)

    The passive Key Trustee Server connects to this port for database replication.

TLS Certificate Requirements

To ensure secure network traffic, Cloudera recommends obtaining Transport Layer Security (TLS) certificates specific to the hostname of your Key Trustee Server. To obtain the certificate, generate a Certificate Signing Request (CSR) for the fully qualified domain name (FQDN) of the Key Trustee Server host. The CSR must be signed by a trusted Certificate Authority (CA). After the certificate has been verified and signed by the CA, the Key Trustee Server TLS configuration requires:

  • The CA-signed certificate
  • The private key used to generate the original CSR
  • The intermediate certificate/chain file (provided by the CA)

Cloudera recommends not using self-signed certificates. If you use self-signed certificates, you must use the --skip-ssl-check parameter when registering Navigator Encrypt with the Key Trustee Server. This skips TLS hostname validation, which safeguards against certain network-level attacks. For more information regarding insecure mode, see Registration Options.

Product Compatibility Matrix for Cloudera Navigator Encryption

Cloudera Navigator encryption comprises several components. See Cloudera Navigator Data Encryption Overview for more information on the components, concepts, and architecture involved in encrypting data.

Although the version numbers differ between some Cloudera Navigator encryption components and Cloudera Enterprise, the encryption component releases are generally coordinated with Cloudera Enterprise releases. See the following table for information on which component versions correspond with a given Cloudera Enterprise release.

Cloudera Navigator Encryption Components Release Versions
Cloudera Enterprise Release Cloudera Navigator Key Trustee Server Version Key Trustee KMS Version Cloudera Navigator Key HSM Version Cloudera Navigator Encrypt Version
5.16.x No release; use 5.15.x 5.16.x No release; use 1.11.x 3.16.x
5.15.x 5.15.x 5.15.x No release; use 1.11.x 3.15.x
5.14.x 5.14.x 5.14.x 1.11.x No release; use 3.14.x
5.13.x 5.13.x 5.13.x No release; use 1.10.x 3.14.x
5.12.x 5.12.x 5.12.x 1.10.x 3.13.x
5.11.x 5.11.x 5.11.x 1.9.x 3.12.x
5.10.x 5.10.x 5.10.x No release; use 1.8.x 3.11.x
5.9.x 5.9.x 5.9.x 1.8.x No release; use 3.10.x
5.8.x 5.8.x 5.8.x 1.7.x 3.10.x
5.7.x 5.7.x 5.7.x 1.6.x 3.9.x
5.6.x No release; use 5.5.x No release; use 5.5.x No release; use 1.5.x No release; use 3.8.x
5.5.x 5.5.x 5.5.x 1.5.x 3.8.x
5.4.x 5.4.x 5.4.x 1.4.x 3.7.x
5.3.x 3.8.x 5.3.x 1.3.x 3.6.x

Cloudera Navigator Key Trustee Server

Because of a change in the ports used by Key Trustee Server, Navigator Encrypt versions lower than 3.7 and Key Trustee KMS versions lower than 5.4 are not supported in Key Trustee Server 5.4 and higher.

Key Trustee Server: Recommended Hardware and Supported Distributions

Recommended Hardware and Supported Distributions

Key Trustee Server must be installed on a dedicated server or virtual machine (VM) that is not used for any other purpose. The backing PostgreSQL database must be installed on the same host as the Key Trustee Server, and must not be shared with any other services. For high availability, the active and passive Key Trustee Servers must not share physical resources. See Resource Planning for Data at Rest Encryption for more information.

The recommended minimum hardware specifications are as follows:

  • Processor: 1 GHz 64-bit quad core
  • Memory: 8 GB RAM
  • Storage: 20 GB on moderate-to high-performance disk drives
Cloudera Navigator Key Trustee Server Compatibility Matrix
Cloudera Navigator Key Trustee Server Version Supported Operating Systems Supported Cloudera Manager Versions Supported Cloudera Navigator Key HSM Versions Supported Key Trustee KMS Versions Supported Cloudera Navigator Encrypt Versions
5.15.x
  • RHEL and CentOS: 6.4 - 6.10, 7.1 - 7.6
  • Oracle Linux: 6.7 - 6.9, 7.1 - 7.5
5.15.x and higher 1.4.x and higher 5.4.x and higher 3.7.x and higher
5.14.x
  • RHEL and CentOS: 6.4 - 6.10, 7.1 - 7.6
  • Oracle Linux: 6.7 - 6.9, 7.1 - 7.5
5.14.x and higher 1.4.x and higher 5.4.x and higher 3.7.x and higher
5.13.x
  • RHEL and CentOS: 6.4 - 6.10, 7.1 - 7.6
  • Oracle Linux: 6.7 - 6.9, 7.1 - 7.5
5.13.x and higher 1.4.x and higher 5.4.x and higher 3.7.x and higher
5.12.x
  • RHEL and CentOS: 6.4 - 6.10, 7.1 - 7.5
  • Oracle Linux: 6.7 - 6.9, 7.1 - 7.5
5.12.x and higher 1.4.x and higher 5.4.x and higher 3.7.x and higher
5.11.x
  • RHEL and CentOS: 6.4 - 6.8, 7.1 - 7.5
  • Oracle Linux: 6.7, 6.8, 7.1 - 7.5
5.11.x and higher 1.3.x and higher 5.4.x and higher 3.7.x and higher
5.10.x
  • RHEL and CentOS: 6.4 - 6.8, 7.1 - 7.3
  • Oracle Linux: 6.7, 6.8, 7.1 - 7.3
5.10.x and higher 1.3.x and higher 5.4.x and higher 3.7.x and higher
5.9.x
  • RHEL and CentOS: 6.4 - 6.8, 7.1, 7.2
  • Oracle Linux: 6.7, 6.8, 7.1, 7.2
5.9.x 1.3.x and higher 5.4.x and higher 3.7.x and higher
5.8.x
  • RHEL and CentOS: 6.4 - 6.7, 7.1, 7.2
  • Oracle Linux: 6.7, 7.1, 7.2
5.8.x, 5.9.x 1.3.x and higher 5.4.x and higher 3.7.x and higher
5.7.x
  • RHEL and CentOS: 6.4 - 6.7, 7.1, 7.2
  • Oracle Linux: 6.7, 7.1, 7.2
5.7.x - 5.9.x 1.3.x and higher 5.4.x and higher 3.7.x and higher
5.5.x
  • RHEL and CentOS: 6.4 - 6.7, 7.1
  • Oracle Linux: 6.7, 7.1, 7.2
5.5.x - 5.9.x 1.3.x and higher 5.4.x and higher 3.7.x and higher
5.4.x
  • RHEL and CentOS: 6.4 - 6.6
5.4.x - 5.9.x 1.3.x and higher 5.4.x and higher 3.7.x and higher
3.8.x
  • RHEL and CentOS: 6.4, 6.5
Not supported 1.3.x 5.3.x 3.6.x

Key Trustee KMS

Key Trustee KMS: Recommended Hardware and Supported Distributions

The recommended minimum hardware specifications are as follows:

  • Oracle Java Runtime Environment (JRE) 7 or higher with Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files:
  • Processor: 2 GHz 64-bit quad core
  • Memory: 16 GB RAM
  • Storage: 40 GB on moderate-to high-performance disk drives

The Key Trustee KMS workload is CPU-intensive. Cloudera recommends using machines with capabilities equivalent to your NameNode hosts, with Intel CPUs that support AES-NI for optimum performance. Also, Cloudera strongly recommends that you enable TLS for both the HDFS and the Key Trustee KMS services to prevent the passage of plain text key material between the KMS and HDFS data nodes.

Key Trustee KMS Compatibility Matrix
Key Trustee KMS Version Supported Operating Systems Supported Key Trustee Server Versions Supported Cloudera Manager Versions Supported CDH Versions
5.16.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.10, 7.1 - 7.6
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.9, 7.1 - 7.5
  • SLES: 11 SP2 - SP4, 12 SP1 - SP3
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 14.04, 16.04
5.4.x and higher 5.16.x and higher 5.3.x - 5.16.x
5.15.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.10, 7.1 - 7.6
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.9, 7.1 - 7.5
  • SLES: 11 SP2 - SP4, 12 SP1 - SP3
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04, 16.04
5.4.x and higher 5.15.x and higher 5.3.x - 5.15.x
5.14.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.10, 7.1 - 7.6
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.9, 7.1 - 7.5
  • SLES: 11 SP2 - SP4, 12 SP1, 12 SP2
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04, 16.04
5.4.x and higher 5.14.x and higher 5.3.x - 5.14.x
5.13.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.10, 7.1 - 7.6
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.9, 7.1 - 7.5
  • SLES: 11 SP2 - SP4, 12 SP1, 12 SP2
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04, 16.04
5.4.x and higher 5.13.x and higher 5.3.x - 5.13.x
5.12.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.10, 7.1 - 7.5
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.9, 7.1 - 7.5
  • SLES: 11 SP2 - SP4, 12 SP1, 12 SP2
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04, 16.04
5.4.x and higher 5.12.x and higher 5.3.x - 5.12.x
5.11.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.8, 7.1 - 7.5
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.8, 7.1 - 7.5
  • SLES: 11 SP2 - SP4, 12 SP1, 12 SP2
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04, 16.04
5.4.x and higher 5.11.x and higher 5.3.x - 5.11.x
5.10.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.8, 7.1 - 7.3
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.8, 7.1 - 7.3
  • SLES: 11 SP2 - SP4, 12 SP1
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04
5.4.x and higher 5.10.x and higher 5.3.x - 5.10.x
5.9.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.8, 7.1, 7.2
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.8, 7.1, 7.2
  • SLES: 11 SP2 - SP4, 12 SP1
  • Debian: 7.1, 7.8, 8.2, 8.4
  • Ubuntu: 12.04, 14.04
5.4.x and higher 5.9.x and higher 5.3.x - 5.9.x
5.8.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.7, 7.1, 7.2
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.7, 7.1, 7.2
  • SLES: 11 SP2 - SP4
  • Debian: 7.1, 7.8, 8.2
  • Ubuntu: 12.04, 14.04
5.4.x and higher 5.8.x and higher 5.3.x - 5.8.x
5.7.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.7, 7.1, 7.2
  • Oracle Linux: 5.7, 5.10, 5.11, 6.4 - 6.7, 7.1, 7.2
  • SLES: 11 SP2 - SP4
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04
5.4.x and higher 5.7.x and higher 5.3.x - 5.7.x
5.5.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.7, 7.1
  • Oracle Linux: 5.7, 5.10, 6.4 - 6.7, 7.1
  • SLES: 11 SP2, SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
5.4.x and higher 5.5.x and higher 5.3.x - 5.5.x
5.4.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.6
  • Oracle Linux: 5.7, 5.10, 6.4 - 6.6
  • SLES: 11 SP2, SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
5.4.x and higher 5.4.x and higher 5.3.x, 5.4.x
5.3.x
  • RHEL and CentOS: 5.7, 6.4, 6.5
  • Oracle Linux: 5.7, 6.4, 6.5
  • SLES: 11 SP2, 11 SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
3.8.x 5.3.x and higher 5.3.x

Cloudera Navigator HSM KMS

Navigator HSM KMS: Recommended Hardware and Supported Distributions

The recommended minimum hardware specifications are as follows:

  • Oracle Java Runtime Environment (JRE) 7 or higher with Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files:
  • Processor: 2 GHz 64-bit quad core
  • Memory: 16 GB RAM
  • Storage: 40 GB on moderate-to high-performance disk drives

Supported HSM devices:

  • SafeNet Luna
    • HSM software version: 6.2.2-5
    • HSM firmware version: 6.10.9
    • Client: 6.2.2
  • Thales nShield Solo, nShield Connect
    • Server version: 3.67.11cam4
    • Firmware: 2.65.2
    • Security World Version: 12.30
HSM KMS Compatibility Matrix
HSM KMS Version Supported Operating Systems Supported Cloudera Manager Version Supported CDH Versions
5.16.x
  • RHEL and CentOS: 6.8, 6.9, 7.1 - 7.5
5.16.x and higher 5.12.x - 5.16.x
5.15.x
  • RHEL and CentOS: 6.8, 6.9, 7.1 - 7.5
5.15.x and higher 5.12.x - 5.15.x
5.14.x
  • RHEL and CentOS: 6.8, 6.9, 7.1 - 7.4
5.14.x and higher 5.12.x - 5.14.x
5.13.x
  • RHEL and CentOS: 6.8, 6.9, 7.1 - 7.4
5.13.x and higher 5.12.x, 5.13.x
5.12.x
  • RHEL and CentOS: 6.8, 6.9, 7.1 - 7.3
5.12.x and higher 5.12.x

Cloudera Navigator Key HSM

Cloudera Navigator Key HSM must be installed on the same host as Key Trustee Server. Although Key HSM is compatible across all versions of Key Trustee Server, Cloudera strongly recommends also upgrading Key HSM after you upgrade Key Trustee Server. See Installing Cloudera Navigator Key HSM and Upgrading Cloudera Navigator Key HSM for more information.

Key HSM: Recommended Hardware and Supported Distributions

The following are prerequisites for installing Navigator Key HSM:

  • Oracle Java Runtime Environment (JRE) 7 or higher with Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files:
  • A supported HSM device:
    • Thales (formerly SafeNet) Luna
      • v6
        • HSM firmware version: 6.2.1
        • HSM software version: 5.2.3-1
      • v7
        • HSM firmware version: 7.0.3
        • HSM software version: 7.2.0
      • HSM firmware version: 6.2.1
      • HSM software version: 5.2.3-1
    • SafeNet KeySecure
      • HSM firmware version: 6.2.1
      • HSM software version: 8.0.1, 8.1.0, 8.7.0
    • Thales nShield Solo, nShield Connect
      • HSM firmware version: 11.4.0
      • Client software version: 2.28.9cam136
  • Key Trustee Server 3.8 or higher

Root access is required to install Navigator Key HSM.

Cloudera Navigator Key HSM Compatibility Matrix
Cloudera Navigator Key HSM Version Supported Operating Systems Supported Key Trustee Server Versions
1.11.x
  • RHEL and CentOS: 6.4 - 6.9, 7.1 - 7.6
3.8.x and higher
1.10.x
  • RHEL and CentOS: 6.4 - 6.9, 7.1 - 7.3
3.8.x and higher
1.9.x
  • RHEL and CentOS: 6.4 - 6.8, 7.1, 7.2
3.8.x and higher
1.8.x
  • RHEL and CentOS: 6.4 - 6.8, 7.1, 7.2
3.8.x and higher
1.7.x
  • RHEL and CentOS: 6.4 - 6.7, 7.1, 7.2
3.8.x and higher
1.6.x
  • RHEL and CentOS: 6.4 - 6.7, 7.1, 7.2
3.8.x and higher
1.5.x
  • RHEL and CentOS: 6.4 - 6.7, 7.1
3.8.x and higher
1.4.x
  • RHEL and CentOS: 6.4 - 6.6
3.8.x and higher
1.3.x
  • RHEL and CentOS: 6.4, 6.5
3.8.x and higher

Cloudera Navigator Encrypt

Supported command-line interpreters:

  • sh (Bourne)
  • bash (Bash)
  • dash (Debian)

Network Requirements

For new Navigator Key Trustee Server (5.4.0 and higher) installations, Navigator Encrypt initiates TCP traffic over port 11371 (HTTPS) to the Key Trustee Server.

For upgrades and Key Trustee Server versions lower than 5.4.0, Navigator Encrypt initiates TCP traffic over ports 80 (HTTP) and 443 (HTTPS) to the Navigator Key Trustee Server.

Internet Access

You must have an active connection to the Internet to download many package dependencies, unless you have internal repositories or mirrors containing the dependent packages.

Maintenance Window

Data is not accessible during the encryption process. Plan for system downtime during installation and configuration.

Administrative Access

To enforce a high level of security, all Navigator Encrypt commands require administrative (root) access (including installation and configuration). If you do not have administrative privileges on your server, contact your system administrator before proceeding.

Package Dependencies

Navigator Encrypt requires these packages, which are resolved by your distribution package manager during installation:

  • dkms
  • keyutils
  • ecryptfs-utils
  • libkeytrustee
  • navencrypt-kernel-module
  • openssl
  • lsof
  • gcc
  • cryptsetup

These packages may have other dependencies that are also resolved by your package manager. Installation works with gcc, gcc3, and gcc4.

Cloudera Navigator Encrypt Compatibility Matrix
Cloudera Navigator Encrypt Version Supported Operating Systems Supported Key Trustee Server Versions
3.16.x
  • RHEL and CentOS: 7.7, 5.7, 5.10, 6.4 - 6.10, 7.1 - 7.5
  • Oracle Linux: 7.7, 6.4 - 6.9, 7.1, 7.4, 7.5
  • SLES: 11 SP2 - SP4, 12 SP1 - SP3
  • Debian: 7.1, 7.8
  • Ubuntu: 14.04, 14.04.3, 16.04
5.4.x - 5.16.x
3.15.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.9, 7.1 - 7.5
  • Oracle Linux: 6.4 - 6.9, 7.1, 7.4, 7.5
  • SLES: 11 SP2 - SP4, 12 SP1 - SP3
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3, 16.04
5.4.x - 5.15.x
3.14.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.9, 7.1 - 7.4
  • Oracle Linux: 6.4 - 6.9, 7.1, 7.4
  • SLES: 11 SP2 - SP4; 12 SP1, SP2
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3, 16.04
5.4.x - 5.13.x
3.13.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.9, 7.1 - 7.4
  • Oracle Linux: 6.4 - 6.9, 7.1
  • SLES: 11 SP2 - SP4; 12 SP1, SP2
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3, 16.04
5.4.x - 5.12.x
3.12.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.9, 7.1 - 7.3
  • Oracle Linux: 6.4 - 6.9, 7.1
  • SLES: 11 SP2 - SP4; 12 SP1, SP2
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3, 16.04
5.4.x - 5.12.x
3.11.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.8, 7.1 - 7.3
  • Oracle Linux: 6.4 - 6.7, 7.1
  • SLES: 11 SP2 - SP4, 12 SP1
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3
5.4.x - 5.11.x
3.10.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.8, 7.1, 7.2
  • Oracle Linux: 6.4 - 6.7, 7.1
  • SLES: 11 SP2 - SP4
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3
5.4.x - 5.11.x
3.9.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.7, 7.1, 7.2
  • Oracle Linux: 6.4 - 6.7, 7.1
  • SLES: 11 SP2 - SP4
  • Debian: 7.1, 7.8
  • Ubuntu: 12.04, 14.04, 14.04.3
5.4.x - 5.11.x
3.8.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.7, 7.1
  • Oracle Linux: 6.4 - 6.7, 7.1
  • SLES: 11 SP2, SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
5.4.x - 5.11.x
3.7.x
  • RHEL and CentOS: 5.7, 5.10, 6.4 - 6.6
  • Oracle Linux: 6.4 - 6.6
  • SLES: 11 SP2, SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
5.4.x - 5.11.x
3.6.x
  • RHEL and CentOS: 5.7, 6.4, 6.5
  • Oracle Linux: 6.4, 6.5
  • SLES: 11 SP2, SP3
  • Debian: 7.1
  • Ubuntu: 12.04, 14.04
3.8.x

Product Compatibility Matrix for Apache Sentry

Sentry enables role-based, fine-grained authorization for HiveServer2 and provides classic database-style authorization for Hive, Cloudera Impala and Cloudera Search. You can use either the Sentry service (introduced in Cloudera Manager 5.1.0 and CDH 5.1.0) or the policy file approach to secure your data.

Product Feature Lowest supported Cloudera Manager Version Lowest supported CDH Version Lowest supported Impala Version Lowest supported Search Version Integrated into CDH
Apache Sentry Sentry Service Cloudera Manager 5.1.0 CDH 5.1.0 Impala 1.4.0 for CDH 5 Search for CDH 5.8.0 Yes
Policy File Cloudera Manager 5.0.0 CDH 5.0.0 Impala 1.3.0 Search 1.1.0 Yes

Support for Other Filesystems

Filesystems Lowest Release Supported
Amazon S3 (s3a) Apache Sentry for CDH 5.8
Amazon RDS Apache Sentry for CDH 5.9

Product Compatibility Matrix for Apache Spark

Spark is a fast, general engine for large-scale data processing. For installation and configuration instructions, see Setting Up Apache Spark Using the Command Line. To see new features introduced with each release, refer to the CDH 5 Release Notes.

Since Spark 1.6 has been integrated into the CDH package, its compatibility with Cloudera Manager and CDH depends on the CDH 5.x.x release it is shipped with.

Product Lowest Supported Cloudera Manager Version Lowest Supported CDH Version Lowest supported Impala Version Lowest supported Search Version Integrated into CDH
Apache Spark 1.6.x Cloudera Manager 5.7.x CDH 5.7.x Not Supported Not Supported CDH 5.7.0
Apache Spark 1.5.x Cloudera Manager 5.5.x and Cloudera Manager 5.6.x CDH 5.5.x and CDH 5.6.x Not Supported Not Supported CDH 5.5.0 and CDH 5.6.0
Apache Spark 1.3.x Cloudera Manager 5.4.x CDH 5.4.x Not Supported Not Supported CDH 5.4.0
Apache Spark 1.2.x Cloudera Manager 5.3.x CDH 5.3.x Not Supported Not Supported CDH 5.3.0
Apache Spark 1.1.x Cloudera Manager 5.2.x CDH 5.2.x Not Supported Not Supported CDH 5.2.0
Apache Spark 1.0.x Cloudera Manager 5.1.x CDH 5.1.x Not Supported Not Supported CDH 5.1.0
Apache Spark 0.9.x Cloudera Manager 5.0.x CDH 5.0.x Not Supported Not Supported CDH 5.0.0

Product Compatibility for Dell EMC Isilon

See also Hadoop Distributions and Products Supported by OneFS on the Dell EMC Community website.

For compatibility with Cloudera Manager Replication and Snapshot features, see Product Compatibility Matrix for Backup and Disaster Recovery. For information on using Isilon with CDH, see Using CDH with Isilon Storage.

CDH Version Isilon OneFS Version
5.12 - 5.16
  • 8.1.2.x
  • 8.1.1.x
  • 8.1.0.x
  • 8.0.1.x
  • 8.0.0.x releases 8.0.0.4 and higher
5.8 - 5.11
  • 8.1.2.x
  • 8.1.1.x
  • 8.1.0.x
  • 8.0.1.x
  • 8.0.0.x releases 8.0.0.4 and higher
  • 8.0.0.3 (with patch 184679)
5.4.4 - 5.7
  • 8.1.2.x
  • 8.1.1.x
  • 8.1.0.x
  • 8.0.1.x
  • 8.0.0.x releases 8.0.0.4 and higher
  • 8.0.0.3 (with patch 184679)
  • 7.2.0.3 and higher (Cloudera recommends 7.2.1.1)

Product Compatibility Matrix for Backup and Disaster Recovery

This matrix contains compatibility information across features of Cloudera Manager Backup and Disaster Recovery and CDH and Cloudera Manager.
Product Feature Lowest supported Cloudera Manager Version Lowest supported CDH Version Supported Services
Backup & Disaster Recovery Replication Cloudera Manager 4.5.0 CDH 4.0.1 HDFS, Hive, Impala
Replication to and from Amazon S3* Cloudera Manager 5.9.0 CDH 5.9.0 HDFS, Hive, Impala
Replication to and from Microsoft ADLS Cloudera Manager 5.15.0 CDH 5.15.0 HDFS, Hive, Impala
Snapshots Cloudera Manager 5.0.0 CDH 5.0.0 HDFS, Hive, Impala
Snapshots from Isilon storage Not Supported Not Supported HDFS, Hive, Impala
*BDR does not support S3 as a source or destination when S3 is configured to use SSE-KMS.

Supported Replication Scenarios

In Cloudera Manager 5, replication is supported between CDH 5 or CDH 4 clusters. The following tables describe support for HDFS and Hive/Impala replication. The versions listed in the table are the lowest version of Cloudera Manager and CDH required to perform the replication. For example, replication with TLS/SSL enabled on Hadoop services on the source and destination clusters requires Cloudera Manager and CDH version 5.0 or higher on the source and destination.

The tables below list the supported replication scenarios:

Service Source Destination
  Cloudera Manager Version CDH Version Comment Cloudera Manager Version CDH Version Comment
HDFS, Hive 4 4   5 4  
HDFS, Hive 4 4.4   5 5  
HDFS, Hive 5 5 TLS/SSL enabled on Hadoop services 5 5 TLS/SSL enabled on Hadoop services
HDFS, Hive 5 5 TLS/SSL enabled on Hadoop services 5 5 TLS/SSL not enabled on Hadoop services
HDFS, Hive 5 5.1 TLS/SSL enabled on Hadoop services and YARN 5 4 or 5  
HDFS, Hive 5 4   4.7.3 or higher 4  
HDFS, Hive 5 4   5 4  
HDFS, Hive 5 5   5 5  
HDFS, Hive 5 5   5 4.4 or higher  
HDFS, Hive 5.7 5.7, with Isilon storage See Supported Replication Scenarios for Clusters using Isilon Storage. 5.7 5.7, with Isilon storage See Supported Replication Scenarios for Clusters using Isilon Storage.
HDFS, Hive 5.7 5.7 5.7 5.7, with Isilon storage
HDFS, Hive 5.7 5.7, with Isilon storage 5.7 5.7
HDFS, Hive 5.8 5.8, with or without Isilon Storage 5.8 5.8, with or without Isilon Storage

Unsupported Replication Scenarios

The tables below list the unsupported replication scenarios:

Service Source Destination
  Cloudera Manager Version CDH Version Comment Cloudera Manager Version CDH Version Comment
Any 4 or 5 4 or 5 Kerberos enabled. 4 or 5 4 or 5 Kerberos not enabled
Any 4 or 5 4 or 5 Kerberos not enabled. 4 or 5 4 or 5 Kerberos enabled
HDFS, Hive 4 or 5 4 Where the replicated data includes a directory that contains a large number of files or subdirectories (several hundred thousand entries), causing out-of-memory errors.

To work around this issue, follow this procedure.

4 or 5 5  
Hive 4 or 5 4 Replicate HDFS Files is disabled. 4 or 5 4 or 5 Over-the-wire encryption is enabled.
Hive 4 or 5 4 Replication can fail if the NameNode fails over during replication. 4 or 5 5, with high availability enabled Replication can fail if the NameNode fails over during replication.
Hive 4 or 5 4 The clusters use different Kerberos realms. 4 or 5 5 An older JDK is deployed. (Upgrade the CDH 4 cluster to use JDK 7 or JDK6u34 to work around this issue.)
Any 4 or 5 4 SSL enabled on Hadoop services. 4 or 5 4 or 5  
Hive 4 or 5 4.2 or higher If the Hive schema contain views. 4 or 5 4  
HDFS 4 or 5 4, with high availability enabled Replications fail if NameNode failover occurs during replication. 4 or 5 5, without high availability Replications fail if NameNode failover occurs during replication.
HDFS 4 or 5 4 or 5 Over the wire encryption is enabled. 4 or 5 4  
HDFS 4 or 5 5 Clusters where there are URL-encoding characters such as % in file and directory names. 4 or 5 4  
Hive 4 or 5 4 or 5 Over the wire encryption is enabled and Replicate HDFS Files is enabled. 4 or 5 4  
Hive 4 or 5 4 or 5 From one cluster to the same cluster. 4 or 5 4 or 5 From one cluster to the same cluster.
HDFS, Hive 4 or 5 5 Where the replicated data includes a directory that contains a large number of files or subdirectories (several hundred thousand entries), causing out-of-memory errors.

To work around this issue, follow this procedure.

4 or 5 4  
HDFS 4 or 5 5 The clusters use different Kerberos realms. 4 or 5 4 An older JDK is deployed. (Upgrade the CDH 4 cluster to use JDK 7 or JDK6u34 to work around this issue.)
Hive 4 or 5 5 Replicate HDFS Files is enabled and the clusters use different Kerberos realms. 4 or 5 4 An older JDK is deployed. (Upgrade the CDH 4 cluster to use JDK 7 or JDK6u34 to work around this issue.)
Any 4 or 5 5 SSL enabled on Hadoop services and YARN. 4 or 5 4 or 5  
Any 4 or 5 5 SSL enabled on Hadoop services. 4 or 5 4  
HDFS 4 or 5 5, with high availability enabled Replications fail if NameNode failover occurs during replication. 4 or 5 4, without high availability Replications fail if NameNode failover occurs during replication.
HDFS, Hive 5 5   4 4  
Hive 5.2 5.2 or lower Replication of Impala UDFs is skipped. 4 or 5 4 or 5  

Supported Replication Scenarios for Clusters using Isilon Storage

Note the following when scheduling replication jobs for clusters that use Isilon storage:
  • As of CDH 5.8 and higher, Replication is supported for clusters using Kerberos and Isilon storage on the source or destination cluster, or both. See Configuring Replication with Kerberos and Isilon. Replication between clusters using Isilon storage and Kerberos is not supported in CDH 5.7.
  • Make sure that the hdfs user is a superuser in the Isilon system. If you specify alternate users with the Run As option when creating replication schedules, those users must also be superusers.
  • Cloudera recommends that you use the Isilon root user for replication jobs. (Specify root in the Run As field when creating replication schedules.)
  • Select the Skip checksum checks property when creating replication schedules.
  • Clusters that use Isilon storage do not support snapshots. Snapshots are used to ensure data consistency during replications in scenarios where the source files are being modified. Therefore, when replicating from an Isilon cluster, Cloudera recommends that you do not replicate Hive tables or HDFS files that could be modified before the replication completes.

See Using CDH with Isilon Storage.

Supported Configurations with Virtualization and Cloud Platforms

This section lists supported cloud and virtualization platforms for deploying Cloudera software. The linked reference architectures are not replacements for statements of support above, but are guides to assist with deployment and sizing options.

Amazon Web Services

For information on deploying Cloudera software on a Amazon Web Services (AWS) cloud infrastructure, see the Cloudera Enterprise Reference Architecture for AWS Deployments.

Google Cloud Platform

For information on deploying Cloudera software on a Google Cloud Platform infrastructure, see the Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments.

Microsoft Azure

For information on deploying Cloudera software on a Microsoft Azure cloud infrastructure, see the Cloudera Enterprise Reference Architecture for Azure Deployments.

Support Limitations for CDH and Cloudera Manager in Microsoft Azure

  • Virtual machines must use Cloudera-published CentOS 6.x/7.x OS Images or Red Hat published 6.x/7.x Images. Red Hat images must include bootstrapped configuration as specified in the Azure Bootstrap Scripts folder in the Cloudera Director GitHub repository.
  • Master Node Services must use Azure Premium Storage Disks for persistent storage.
  • The VM local temporary disk must not be used for any persistent data for HDFS, Kudu, or other services.
  • Microsoft Azure Storage - Block Blob (previously known as Windows Azure Storage Blob, accessed via wasb:// URI’s) is only supported for backups using Hadoop’s DistCP. Other services are not supported running directly against Azure Storage block blobs.

VMware

For information on deploying Cloudera software on a VMware-based infrastructure, see the Reference architecture for deploying on VMware.

Recommendation when deploying on VMware in the current release:

  • Use the part of Hadoop Virtual Extensions that has been implemented in HADOOP-8468. This will prevent data loss when a physical node that hosts two or more DataNodes goes down .

Red Hat OpenStack Platform (OSP) 11

For information on deploying Cloudera software on RedHat OpenStack Platform (OSP) 11, see Reference Architecture for Deploying CDH 5.x on Red Hat OSP 11.

For information on deploying Cloudera software on RedHat OpenStack Platform (OSP) 11 with Ceph storage, see Reference Architecture for Deploying Cloudera Enterprise 5.x on Red Hat OpenStack Platform 11 with Red Hat Ceph Storage 2.x.