Disabling Transparent Hugepages (THP)
Most Linux platforms supported by Cloudera Runtime include a feature called transparent hugepages, which interacts poorly with Hadoop workloads and can seriously degrade performance.
Symptom:
top
and other system monitoring tools show a large percentage of the CPU
usage classified as "system CPU". If system CPU usage is 30% or more of the total CPU usage,
your system may be experiencing this issue.
To see whether transparent hugepages are enabled, run the following commands and check the output:
$ cat defrag_file_pathname
$ cat enabled_file_pathname
-
[always] never
means that transparent hugepages is enabled. -
always [never]
means that transparent hugepages is disabled.
To disable Transparent Hugepages, perform the following steps on all cluster hosts: