Cloudera Docs

Quota for CML workloads

Quota management is implemented for both user and group level. A Machine Learning (ML) Workspace is allocated a set amount of resources based on configured parameters at provisioning time. Within a workspace, resources available for workloads can be further subdivided into quotas at user and/or team level.

User and team naming limitations

Follow the kubernetes label naming conventions when naming users and teams:
  • The name must be 63 characters or less.
  • The name must begin and end with an alphanumeric character ([a-z0-9A-Z]).
  • The name can contain dashes (-), underscores (_), dots (.), and alphanumerics, except for the beginning and ending characters.

User quota

By default, 8 GB memory and 2 vCPU cores are configured for each user. Such resources are sufficient for running simple sessions but might not be sufficient for the spark workloads if the executors cannot find enough resources. The Cloudera Machine Learning (CML) administrators can configure custom quota for the user on the Site Administration Page.

Figure 1. User quota settings


If the quota for a user is used up, the workload remains in the pending state until the required resources are available.

If the quota for the users is modified, it will be reflected when the next workload is submitted.

GPU resources can be edited if the workspace is provisioned with GPU resources.

Group quota

There is no quota configured by default for a team or group.

  1. As a CML administrator configure a custom quota for a group under Site Administration > Quotas tab.
  2. Configure a quota for a group, at the Quotas Teams tab.
  3. Click the Add Team Quota and a popup enables you to add a custom team quota.
Figure 2.
Figure 3. Custom quota

If the quota for the group is used up, the workload remains in PENDING state until the required resources are available.

If the quota for the group is modified, it will be reflected when the next workload is submitted.

Quota enforcement

The resources are allocated based on the project context from which the workloads are created.

  • If a workload is created in a project, which is developed in users' context, it will always take the user quota.
  • If a workload is created in a project which is developed in a team's context, it will take resources from the team's quota, provided the team has a quota configured.
  • If a workload is created in a project which is developed in a team's context and the team does not have a quota configured, it will take the resources from the users' quota.