Enable Transparent Huge Pages
The Fastcapa probe performs its own memory management by leveraging Transparent Huge Pages (THP). In Linux, you can use the Transparent Huge Pages to enable either dynamically or automatically upon startup. Hortonworks recommends that these be allocated on boot to increase the chance that a larger, physically contiguous chunk of memory allocated.
- Ensure that your CPU supports 1 GB THPs. A CPU flag
pdpe1gbindicates whether or not the CPU supports 1 GB THPs.
grep --color=always pdpe1gb /proc/cpuinfo | uniq
/etc/default/grubto add the following book parameters at the line starting with
GRUB_CMDLINE_LINUX=... default_hugepagesz=1G hugepagesz=1G hugepages=16
- Rebuild the Grub configuration then reboot. The location of the Grub configuration file will
differ across Linux distributions.
cp /etc/grub2-efi.cfg /etc/grub2-efi.cfg.orig /sbin/grub2-mkconfig -o /etc/grub2-efi.cfg
- After the host reboots, ensure that the THPs were successfully allocated:
$ grep HugePage /proc/meminfo AnonHugePages: 933888 kB HugePages_Total: 16 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0The total number of huge pages should be distributed fairly evenly across each non-uniform memory access (NUMA) node. In the following example, a total of 16 requested THPs are distributed as 8 to each of 2 NUMA nodes:
$ cat /sys/devices/system/node/node*/hugepages/hugepages-1048576kB/nr_hugepages 8 8
- After the THPs are reserved, mount them to make them available to the probe:
cp /etc/fstab /etc/fstab.orig mkdir -p /mnt/huge_1GB echo "nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0" >> /etc/fstab mount -fav