Operating System Requirements
The following topics describe the operating system requirements for Cloudera software:
Software Dependencies
- Python - Python dependencies for the different CDP components is mentioned below:
- Cloudera Manager
- Cloudera Manager supports the system Python on supported OSes, and does not support Python 3.
- Hue
- Hue requires Python 2.7, and does not support Python 3.
- Spark
- Spark 2.4 supports Python 2.7 and 3.4-3.7.
- Perl - Cloudera Manager requires perl.
- python-psycopg2 - Cloudera Manager 6 has a dependency on the package python-psycopg2. Hue in CDH 6 requires a higher version of psycopg2 than is required by the Cloudera Manager dependency. For more information, see Installing the psycopg2 Python Package.
- iproute package - Cloudera Enterprise 6 has a dependency on the iproute package. Any
host that runs the Cloudera Manager Agent requires the package. The required version varies depending on the operating system:
iproute package Operating System iproute version RHEL 7 Compatible iproute-3.10 RHEL 6 Compatible iproute-2.6 SLES 12 SP2, SP3 iproute2-3.12 SLES 12 SP4 iproute2-4.4 SLES 12 SP5 iproute2-4.12
CDH and Cloudera Manager Supported Operating Systems
CDH provides 64-bit packages for select versions of RHEL-compatible, SLES, and Ubuntu operating systems.
- Red Hat only supports specific upgrades from RHEL 6 to 7. Contact your OS vendor and review the Red Hat article What are the supported use cases for upgrading to RHEL 7?.
- Cloudera does not support CDH cluster deployments in Docker containers.
- Cloudera Enterprise, with the exception of Cloudera Navigator Encrypt, is supported on platforms with Security-Enhanced Linux (SELinux) enabled and in enforcing mode. Cloudera is not responsible for policy support or policy enforcement. If you experience issues with SELinux, contact your OS provider.
Cloudera Enterprise 6.3.x Supported Operating Systems
| Operating System | Version (bold=new) |
|---|---|
| RHEL-compatible | |
|
RHEL/CentOS/OL with RHCK kernel |
7.9, 7.8, 7.7, 7.6, 7.5, 7.4, 7.3, 7.2 6.10, 6.9 |
| Oracle Linux (OL) |
7.6, 7.4, 7.3, 7.2 (UEK default) 6.10 (UEK default) |
| SUSE Linux Enterprise Server | |
| SLES |
12 SP5, 12 SP4*, 12 SP3 |
| Ubuntu | |
| Ubuntu |
18.04 LTS (Bionic) 16.04 LTS (Xenial) |
* SLES 12 SP4 certification includes a known issue when MariaDB is being used as the Cloudera Manager server database: use MariaDB 10.0 instead of the default MariaDB 10.2 provided in the operating system release.
See OPSAPS-52340.
Cloudera Enterprise 6.2.x Supported Operating Systems
| Operating System | Version (bold=new) |
|---|---|
| RHEL-compatible | |
|
RHEL/CentOS/OL with RHCK kernel |
7.9, 7.8, 7.7, 7.6, 7.5, 7.4, 7.3, 7.2 6.10, 6.9, 6.8 |
| Oracle Linux (OL) |
7.6, 7.4, 7.3, 7.2 (UEK default) 6.10 (UEK default) |
| SUSE Linux Enterprise Server | |
| SLES |
12 SP5, 12 SP4*, 12 SP3, 12 SP2 |
| Ubuntu | |
| Ubuntu |
18.04 LTS (Bionic) 16.04 LTS (Xenial) |
* SLES 12 SP4 certification includes a known issue when MariaDB is being used as the Cloudera Manager server database: use MariaDB 10.0 instead of the default MariaDB 10.2 provided in the operating system release.
See OPSAPS-52340.
Cloudera Enterprise 6.1.x Supported Operating Systems
| Operating System | Version (bold=new) |
|---|---|
| RHEL-compatible | |
|
RHEL/CentOS/OL with RHCK kernel |
7.6, 7.5, 7.4, 7.3, 7.2 6.10, 6.9 , 6.8 |
| Oracle Linux (OL) |
7.4, 7.3, 7.2 (UEK default) 6.10 (UEK default) |
| SUSE Linux Enterprise Server | |
| SLES |
12 SP4*, 12 SP3, 12 SP2 |
| Ubuntu | |
| Ubuntu |
16.04 LTS (Xenial) |
* SLES 12 SP4 certification includes a known issue when MariaDB is being used as the Cloudera Manager server database: use MariaDB 10.0 instead of the default MariaDB 10.2 provided in the operating system release.
See OPSAPS-52340.
Cloudera Enterprise 6.0.x Supported Operating Systems
| Operating System | Version (bold=new) |
|---|---|
| RHEL-compatible | |
|
RHEL/CentOS/OL with RHCK kernel |
7.6, 7.5, 7.4, 7.3, 7.2 6.10, 6.9 , 6.8 |
| Oracle Linux (OL) |
7.4, 7.3, 7.2 (UEK default) |
| SUSE Linux Enterprise Server | |
| SLES |
12 SP4*, 12 SP3, 12 SP2 |
| Ubuntu | |
| Ubuntu |
16.04 LTS (Xenial) |
* SLES 12 SP4 certification includes a known issue when MariaDB is being used as the Cloudera Manager server database: use MariaDB 10.0 instead of the default MariaDB 10.2 provided in the operating system release.
See OPSAPS-52340.
Filesystem Requirements
Supported Filesystems
The Hadoop Distributed File System (HDFS) is designed to run on top of an underlying filesystem in an operating system. Cloudera recommends that you use either of the following filesystems tested on the supported operating systems:
- ext3: This is the most tested underlying filesystem for HDFS.
- ext4: This scalable extension of ext3 is supported in more recent Linux releases.
- XFS: This is the default filesystem in RHEL 7.
- S3: Amazon Simple Storage Service
Kudu Filesystem Requirements - Kudu is supported on ext4 and XFS. Kudu requires a kernel version and filesystem that supports hole punching. Hole punching is the use of the fallocate(2) system call with the FALLOC_FL_PUNCH_HOLE option set. For more details, see Error during hole punch test.
File Access Time
Linux filesystems keep metadata that record when each file was accessed. This means that even reads result in a write to the disk. To speed up file reads, Cloudera recommends that you disable this option, called atime, using the noatime mount option in /etc/fstab:
/dev/sdb1 /data1 ext4 defaults,noatime 0
Apply the change without rebooting:
mount -o remount /data1
Filesystem Mount Options
The filesystem mount options have a sync option that allows you to write synchronously.
Using the sync filesystem mount option reduces performance for services that write data to disks, such as HDFS, YARN, Kafka and Kudu. In CDH, most writes are already replicated. Therefore, synchronous writes to disk are unnecessary, expensive, and do not measurably improve stability.
NFS and NAS options are not supported for use as DataNode Data Directory mounts, even when using Hierarchical Storage features.
Mounting /tmp as a filesystem with the noexec option is sometimes done as an enhanced security measure to prevent the execution of files stored there. However, this causes multiple problems with various parts of Cloudera Manager and CDH. Therefore, Cloudera does not support mounting /tmp with the noexec option.
nproc Configuration
Cloudera Manager automatically sets nproc configuration in /etc/security/limits.conf, but this configuration can be overridden by individual files in /etc/security/limits.d/. This can cause problems with Apache Impala and other components.
Make sure that the nproc limits are set sufficiently high, such as 65536 or 262144.
nscd for Kudu
Although not a strict requirement, it's highly recommended that you usenscd to cache both DNS name resolution and static name resolution for Kudu. See Troubleshooting Kudu for more information.