Operating System Requirements

CDP Private Cloud Base Supported Operating Systems

Please see the Cloudera Support Matrix for detailed information about supported operating systems.

Operating System support for the CDP Private Cloud Base Trial Installer

SLES 15 SP4 is supported when using the Trial Installer (cloudera-manager-installer.bin) to install Cloudera Manager.

important

Extra step required when using Cloudera Manager Trial installer on SLES 15 SP4.

When using cloudera-manager-installer.bin to install a trial version of Cloudera Manager, the installation will fail.

Before running cloudera-manager-installer.bin, run the following command:

SUSEConnect --list-extensions
SUSEConnect -p sle-module-legacy/15.4/x86_64
zypper install libncurses5

Important information about Runtime and Cloudera Manager Supported Operating Systems

Runtime provides parcels for select versions of RHEL-compatible operating systems.

important

In order to be covered by Cloudera Support:

All Runtime hosts in a logical cluster must run on the same major OS release.
Cloudera supports a temporarily mixed OS configuration during an OS upgrade project.
Cloudera Manager must run on the same OS release as one of the clusters it manages.

Cloudera recommends running the same minor release on all cluster nodes. However, the risk caused by running different minor OS releases is considered lower than the risk of running different major OS releases.

Points to note:

Cloudera does not support Runtime cluster deployments in Docker containers.
Cloudera Enterprise is supported on platforms with Security-Enhanced Linux (SELinux) enabled and in enforcing mode. Cloudera is not responsible for policy support or policy enforcement. If you experience issues with SELinux, contact your OS provider.
An operating system that has Control Group v2 enabled by default (RHEL 9 version currently) must be manually set to use Control Group v1. For more information, see TSB 2024-757: Deployments using cgroups v2 on RHEL9 will fail to initialize.

CDP Private Cloud Base supported operating systems

Operating System	Version
IBM PowerPC on RHEL	The following components are not supported: Impala Kudu Ozone Navigator Encrypt note Ranger KMS is the recommended Key Management Server for PowerPC deployments.

Operating System and IBM PowerPC support matrix

This matrix explains the operating system supported on IBM PowerPC. There are two core configurations with CDP Private Cloud Base and different PowerPC version deployments:

IBM PowerPC only and CDP Private Cloud Base

IBM PowerPC CPU, IBM Spectrum Scale Storage, and CDP Private Cloud Base. This is a subset of what is supported generally on IBM PowerPC.

IBM PowerPC Support	Documentation
PowerPC 8 and 9 generally without Spectrum Scale Storage	https://www.ibm.com/docs/en/linux-on-systems?topic=lpo-supported-linux-distributions-virtualization-options-power8-power9-linux-power-systems
PowerPC 10 generally without Spectrum Scale Storage	https://www.ibm.com/docs/en/linux-on-systems?topic=lpo-supported-linux-distributions-virtualization-options-power10-linux-power-servers
IBM Spectrum Scale Storage with CDP Private Cloud Base on x86 and PowerPC combinations	https://www.ibm.com/docs/en/spectrum-scale-bda?topic=requirements-support-matrix

Software Dependencies

Python - Python dependencies for the different CDP components is mentioned below:

Cloudera Manager

(For 7.1.9 SP1) You must install Python 3.8 (or 3.9 for RHEL 9.1) on all hosts before upgrading to Cloudera Manager 7.11.3.

(For 7.1.9 SP1) You must install Python 3.10 for SLES 15 SP4 and SLES 15 SP5 on all hosts before upgrading to Cloudera Manager 7.11.3 CHF7. For more information, see Installing Python 3.
important

Cloudera Manager now requires Python 3.10 on all versions of SLES 15, including SLES 15 SP4. It is not possible to support two different versions of Python for the same major version of operating system. If the cluster was previously running Python 3.8, then you must upgrade to Python 3.10.

Due to a change in support from Python 3.8 to Python 3.10 for SLES 15 SP4 and SLES 15 SP5, only a regular upgrade of Cloudera Manager to 7.11.3 CHF7 and CDP Runtime cluster to 7.1.9 SP1 is possible and must occur sequentially without starting the cluster between the Cloudera Manager and CDP Runtime cluster upgrades.

Ubuntu 18 Operating System is not supported from Cloudera Manager 7.11.3 to Cloudera Manager 7.11.3 CHF7 versions. You must upgrade the Operating System from Ubuntu 18 to Ubuntu 20 before you upgrade to Cloudera Manager 7.11.3 CHF7. For performing major OS upgrade, see Upgrading the Operating System to a new Major Version.

Hue

(For 7.1.9) The minimum required version of Python 3.8 is 3.8.12. The minimum version of Python 3.9 (RHEL 9) is 3.9.14.

(For 7.1.9 SP1) The recommended minimum version of Python 3.10 on SLES 15 SP4, SLES 15 SP5, and Ubuntu 22 is 3.10.14.

Spark

important
Spark 2 will be deprecated in Cloudera Runtime 7.1.9. Therefore, 7.1.9 is the last runtime release where Spark 2 is supported. For more information, see Deprecation Notices in Cloudera Runtime.
Spark 2.4 supports Python 2.7 and 3.4-3.7.

Spark 3.0 supports Python 2.7 and 3.4 and higher, although support for Python 2 and 3.4 to 3.5 is deprecated.

Spark 3.1 supports Python 3.6 and higher.

If the right level of Python is not picked up by default, set the PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON environment variables to point to the correct Python executable before running the pyspark command.
Perl - Cloudera Manager requires perl.
python-psycopg2 - Cloudera Manager 7 has a dependency on the package python-psycopg2. PostgreSQL-backed Hue in Runtime 7 requires a higher version of psycopg2 than is required by the Cloudera Manager dependency. For more information, see Installing the psycopg2 Python Package.

iproute package - CDP Private Cloud Base has a dependency on the iproute package. Any host that runs the Cloudera Manager Agent requires the package. The required version varies depending on the operating system:

Table 1. iproute package
Operating System	iproute version
RHEL	iproute
Ubuntu	iproute2
SLES	iproute2

rpcbind package - CDP Private Cloud Base has a dependency on the rpcinfo command which is usually found in the rpcbind package. Any host that runs the Cloudera Manager Agent requires this package. The required version varies depending on the operating system.

Filesystem Requirements

Supported Filesystems

The Hadoop Distributed File System (HDFS) is designed to run on top of an underlying filesystem in an operating system. Cloudera recommends that you use either of the following filesystems tested on the supported operating systems:

ext3: This is the most tested underlying filesystem for HDFS.
ext4: This scalable extension of ext3 is supported in more recent Linux releases.
important
Cloudera does not support in-place upgrades from ext3 to ext4. Cloudera recommends that you format disks as ext4 before using them as data directories.
XFS: This is the default filesystem in RHEL 7.
S3: Amazon Simple Storage Service

Kudu Filesystem Requirements - Kudu is supported on ext4 and XFS. Kudu requires a kernel version and filesystem that supports hole punching. Hole punching is the use of the fallocate(2) system call with the FALLOC_FL_PUNCH_HOLE option set.

File Access Time

Linux filesystems keep metadata that record when each file was accessed. This means that even reads result in a write to the disk. To speed up file reads, Cloudera recommends that you disable this option, called atime, using the noatime mount option in /etc/fstab:

/dev/sdb1 /data1 ext4 defaults,noatime 0

Apply the change without rebooting:

mount -o remount /data1

Filesystem Mount Options

The filesystem mount options have a sync option that allows you to write synchronously.

Using the sync filesystem mount option reduces performance for services that write data to disks, such as HDFS, YARN, Kafka and Kudu. In CDP, most writes are already replicated. Therefore, synchronous writes to disk are unnecessary, expensive, and do not measurably improve stability.

NFS and NAS options are not supported for use as DataNode Data Directory mounts, even when using Hierarchical Storage features.

Cloudera supports mounting /tmp with the noexec option. Mounting /tmp as a filesystem with the noexec option is sometimes done as an enhanced security measure to prevent the execution of files stored there.

Filesystem Requirements

You can control resource allocation for Cloudera Manager and CDP Runtime services (nproc, nofile, etc) from /etc/security/limits.conf, and through init scripts on traditional SysV Init systems. However, on systems using systemd the limits either needs to be set in the service’s unit file, or in /etc/systemd/system.conf, or in files present under /etc/systemd/system.conf.d/*. This is due to a known limitation with systemd as it does not use PAM login sessions (pam_limits.so) for daemon services, thereby ignoring the limits defined in /etc/security/limits.conf. Both Cloudera Manager Agent and Supervisord (responsible for starting CDP Runtime services) are daemonised during system initialisation.

You can perform either of the following steps to modify the resource limit:

For system-wide change, uncomment the process properties from /etc/systemd/system.conf, or create an override.conf under /etc/systemd/system.conf.d/. This requires a *nix system reboot for the changes to take effect. For more information, see Limits.conf.
To apply custom limits on CDP Runtime services, add the required process properties to the [Service] section in /usr/lib/systemd/system/cloudera-scm-supervisord.service.
For instance, to customise the number of child processes a process can fork. You can set the property as follows:
```
LimitNPROC=<value>
```
Then reload the configuration by running the following command for the limits to be applied in the subsequent service restarts:
```
# systemctl daemon-reload
```
Here are the list of available process properties.

nscd for Kudu

Although not a strict requirement, it's highly recommended that you use nscd to cache both DNS name resolution and static name resolution for Kudu.

Configuring system level operating system

Cloudera recommends you to set up the following configurations:

Disabling Transparent Hugepages (THP)
vm.swappiness Linux Kernel Parameter

For setting these configurations, see Disabling Transparent Hugepages (THP) and Setting the vm.swappiness Linux Kernel Parameter.