Cloudera Docs

Creating Resource Profiles

Resource profiles define how many vCPUs and how much memory the product reserves for a particular workload, for example, Session, Job, Model.

As a site administrator, you can create multiple resource profile configurations with different combinations of vCPUs, GPUs, and memory. These profiles are made available to users when they launch workloads. When starting a new session or job, users can select the resource profile that best matches their project requirements.

  1. Go to the Site Administration > Runtimes page.
  2. In the Resource Profiles section, add a new resource profile.
  3. Configure the required vCPU, memory, and, if available, GPU settings for the profile.
  4. Enable or disable CPU bursting by selecting or deselecting the Enable CPU bursting option.
  5. Define the Workload Accelerators.

    If the cluster has GPUs, an additional Workload Accelerators section is displayed in Resource Profiles. This section lists the available GPU types, including Multi-instance GPU (MIG) profiles, along with a Maximum GPUs per Workload value.

    1. Define the default GPU Type and the Maximum GPUs per Workload.

      The Maximum GPUs per Workload setting can be configured only by an administrator. For example, if the maximum value for NVIDIA H100 GPUs is set to 5, users can create workloads that request up to 5 NVIDIA H100 GPUs, provided that the user or team has sufficient NVIDIA H100 quota assigned.

      In Cloudera AI 1.5.5 SP4 and higher releases, standardized MIG slice profiles, such as 1g/5GB, 2g/10GB, 3g/20GB, and 7g/40GB, are predefined and available for administrators to select when configuring resource profiles.

      MIG support in Cloudera AI Workbench is dynamic, based upon the hardware you can select different slice profiles.

  6. Save the profile to make it available for workloads by clicking Update.

    Cloudera recommends allocating at least 2 GB of RAM to each profile to help prevent out-of-memory errors during common user operations. Resource Profiles are created exclusively for CPU and memory.

    Additionally, CPU and memory resources must be allocated contiguously on the same node because Kubernetes schedules each pod to run on a single worker node. For instance, if a deployment has two worker nodes with a combined total of 10 vCPUs, a user cannot start a session requesting 8 vCPUs unless a single worker node has 8 available vCPUs. This is expected behavior in Kubernetes environments.

Figure 1. Resource profiles available when launching a session in Cloudera AI 1.5.5 SP4 and higher releases


Figure 2. Resource profiles available when launching a session in Cloudera AI 1.5.5 SP3 and lower releases