Linux Control Groups (cgroups)
Administrators can use Cloudera Manager to configure the Linux Control Groups (cgroup v1 or cgroup v2) kernel feature to manage host resources, such as CPU and I/O, to prevent services from negatively impacting each other. This situation, often called the noisy neighbor problem, can overcome by administrators by imposing per-resource restrictions on services and roles to isolate compute frameworks from one another.
Minimum Required Role: Full Administrator. This feature is not available when using to manage Data Hub clusters.
- Static Service Pools: This is the recommended, wizard-driven method that simplifies cgroup configuration by allowing you to allocate resources as percentages.
- Manual Configuration: For advanced control, you can set individual cgroup properties directly on cluster services and roles.
While cgroups can manage memory, Cloudera Manager primarily uses them for CPU and I/O contention. For memory, Cloudera Manager sets application-level limits (For example, the maximum Java heap size). This approach is designed to prevent the Linux kernel's Out-Of-Memory (OOM) killer from abruptly terminating a service process, which can occur when a cgroup memory hard limit is breached. While memory soft limits are less aggressive than hard limits, they are avoided for the same fundamental risk that the Linux kernel will unexpectedly terminate a service process.
