Disabling Transparent Hugepages (THP)

Most Linux platforms supported by Cloudera Runtime include a feature called transparent hugepages, which interacts poorly with Hadoop workloads and can seriously degrade performance.

Symptom: top and other system monitoring tools show a large percentage of the CPU usage classified as "system CPU". If system CPU usage is 30% or more of the total CPU usage, your system may be experiencing this issue.

To see whether transparent hugepages are enabled, run the following commands and check the output:

$ cat defrag_file_pathname
$ cat enabled_file_pathname

[always] never means that transparent hugepages is enabled.
always [never] means that transparent hugepages is disabled.

To disable Transparent Hugepages, perform the following steps on all cluster hosts:

(Required for hosts running RHEL/CentOS 7.x.) To disable transparent hugepages on reboot, add the following commands to the /etc/rc.d/rc.local file on all cluster hosts:

RHEL/CentOS 8.x:

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

RHEL/CentOS 7.x:

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

RHEL/CentOS 6.x

echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

Ubuntu/Debian, OL, SLES:

echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

Modify the permissions of the rc.local file:

chmod +x /etc/rc.d/rc.local

If your cluster hosts are running RHEL/CentOS 7.x or 8.x, modify the GRUB configuration to disable THP:
1. Add the following line to the GRUB_CMDLINE_LINUX options in the /etc/default/grub file:
```
transparent_hugepage=never
```
2. Run the following command:
```
grub2-mkconfig -o /boot/grub2/grub.cfg
```
Disable the tuned service, as described above.
You can also disable transparent hugepages interactively (but remember this will not survive a reboot).

To disable transparent hugepages temporarily as root:
```
# echo 'never' > defrag_file_pathname
# echo 'never' > enabled_file_pathname 
```
To disable transparent hugepages temporarily using sudo:
```
$ sudo sh -c "echo 'never' > defrag_file_pathname"
$ sudo sh -c "echo 'never' > enabled_file_pathname" 
```