Managing cluster resources using Quota Management

Using Resource pools in a Cloudera Data Services on premises

Quotas are managed and set up in a hierarchical structure. A single root level is created for the whole cluster and cannot be manipulated. The second level in the structure is the environment node. Multiple environments can be created in a single cluster. For more details, see Registering a Cloudera on premises environment.

Quotas are primarily set up and managed through the data services. Each data service provides its own integration. When adding a Data Service to an environment, a fixed base pool for the Data Service is added. A data service is always part of an environment and counts towards the quota set for the environment.

The fixed base pool is the standard node for adding services.

Environment creation

While creating an environment a pool is created for that environment. The quota for the environment can be set as part of the flow of creating an environment and adjusted if needed using the Resource Management UI that is part of the console.

The recommendation is to set a quota even if only one environment will be created. All services created within an environment cannot use more resources than the quota set for the environment. Minimum requirements for the services are documented as part of the data service.

The monitoring service that is automatically installed as part of the environment creation requires XX GB memory and XX cpu resources. Setting a quota on the environment that is smaller than the monitoring requirements will cause failures in the environment creation.

Upgrade from earlier releases to 1.5.5

As part of the upgrade process the pool structure will be migrated to the new standard structure. Pools that were created by earlier versions will be moved around to align with the new structure.

As part of these changes the following updates will be made:

• The "default" node is no longer used and will be removed.
• A new level for environments will be inserted replacing the existing "default" node.
• A new level for services will be inserted using fixed names for the services: "cde", "cml" and "cdw"
• The option to choose a base pool in the hierarchy as part of the service install will be removed.

Even if no quota management was used in the cluster the environment level will be created for all existing environments. The environment nodes will not have a quota set as part of the migration. The services within each environment are allowed to use the whole cluster. This is equivalent to the pre 1.5.5 behaviour.

Existing service related pools with their quota setting will be retained if the pool before and after upgrade exists. Newly created service related pools will be created without a quota, like the environment level pools.

Node setup and grouping

Quota management enables you to control how resources are allocated within your Cloudera Data Services on premises clusters. In order to prevent a single workload from consuming all available cluster resources, you can limit the number of CPUs, GPUs, and memory allocated by application, user, business units, or Data Service by defining resource pools that define resource limits. Pools are organized in a hierarchical manner by defining nodes in the hierarchy with resource limits, which can then be subdivided as needed to allocate resources for an organization and to allocate resources to cluster or environment wide services such as the monitoring service.

Editing the hierarchy or nodes in the hierarchy is restricted at certain levels in the UI. These levels are directly managed by the services:

Accessing the Quotas Page

1. Open the Cloudera console.

2. Go to the Cloudera Management Console service.

3. Click Resource Utilization in the left navigation panel.

4. Select the Quotas tab.

Creating child nodes (resource pools),

1. Click the actions menu in a pool and select Add Child.

   

   The Add Quota dialog box displays.

   

2. Enter the following information:
   • Name – Enter a name for this resource pool.
   • Memory – Use the sliders to select the memory allocation for the pool. Use the drop-down list to select the units.
   • CPU – Use the sliders to select the CPU allocation for the pool. You can choose how CPUs are counted by selecting Cores or Millicores from the drop-down list.
   • GPU – Use the sliders to select the GPU allocation for the pool.
   • Validity – Enter the length of time that the pool can remain active. To keep the pool active indefinitely, enter -1.
   • Distribution policy – The distribution type of the parent quota to the child pools (elastic or inelastic).
   • Tags – Tags provide a way to add user-defined name/value pairs as metadata for the pools. Tags are not currently used in this release.

Editing a resource pool

You can edit a resource pool after creating it by clicking the actions menu for the pool and selecting Edit.

Viewing details of a resource pool

You can view the resource allocations and other information by clicking the “i” icon. The namespace for the pool appears at the top:

Sorting the resource pool display

Click the Sort by drop down list and select a sorting option to sort the resource pools within each level.

Hierarchical quotas and guaranteed resources

The quota and guaranteed resources are defined in the hierarchy of pools. Rules apply to what can be set as a quota or guaranteed resource for a pool.

The top level pool, the root, does not allow setting a quota or guaranteed resources. The quota and the guaranteed resources will be reflect the cluster size during the quota checks.

For any pool in the hierarchy the following rules apply:

1. Quota must be the same size or larger than the guaranteed resources set on a pool

   Example: a guaranteed resource of memory = 1 TB is not allowed if quota is set to memory = 100 GB

2. Quota set on a pool cannot be larger than the quota set on its parent pool

   Example: a child pool quota of memory = 1 TB is not allowed if parent pool quota is set to memory = 100 GB

3. Guaranteed resource set on a pool cannot be larger than the guaranteed resource of its parent pool

   Example: a guaranteed resource of CPU = 10 on the child is not allowed if guaranteed of the parent is set to CPU = 5

4. The sum of all guaranteed resources of pools belonging to the same parent pool cannot be larger than the guaranteed resource of that parent pool.

   Example: two child pools with a guaranteed resource of CPU = 10 each is not allowed if guaranteed of the parent is set to CPU = 15