Enable Cgroups
You can enable CPU Scheduling to enable cgroups. You must configure certain properties
in yarn-site.xml on the ResourceManager and NodeManager hosts to enable
cgroups.
- Navigate to the Hosts option in the left navigation pane.
- Select Hosts Configuration. The Hosts Configuration page is displayed.
- Search for Enable Cgroups.
- Check the Enable Cgroup-based Resource Management box.
cgroups is a Linux kernel feature. cgroups is supported on the following Linux operating systems:
-
CentOS
-
RHEL
-
SUSE
-
Ubuntu
The setup procedure differs depending on the version of cgroups your operating system uses:
- cgroups v1 — Default on RHEL 7, RHEL 8, and older Ubuntu releases. You must manually create the cgroup directory hierarchy and set ownership before starting the NodeManager.
- cgroups v2 — Default on RHEL 9 and Ubuntu 22.04 and later. The operating
system manages the cgroup hierarchy. Do not create cgroup directories manually or run
chownon cgroup paths.
Create the cgroup directory hierarchy (cgroups v1 only)
Run the following commands on every reboot of the NodeManager hosts to set up
the cgroup hierarchy. Operating systems use different mount points for the cgroup
interface. Replace /sys/fs/cgroup with your operating system
equivalent.
mkdir -p /sys/fs/cgroup/cpu/yarn chown -R yarn /sys/fs/cgroup/cpu/yarn mkdir -p /sys/fs/cgroup/memory/yarn chown -R yarn /sys/fs/cgroup/memory/yarn mkdir -p /sys/fs/cgroup/blkio/yarn chown -R yarn /sys/fs/cgroup/blkio/yarn mkdir -p /sys/fs/cgroup/net_cls/yarn chown -R yarn /sys/fs/cgroup/net_cls/yarn mkdir -p /sys/fs/cgroup/devices/yarn chown -R yarn /sys/fs/cgroup/devices/yarn
