Cloudera Docs

Creating Resource Profiles

Resource profiles define how many vCPUs and how much memory the product will reserve for a particular workload (for example, session, job, model).

Every Resource Group contains its respective Resource Profiles. By default, every Resource Group is configured with a minimum set of Resource Profiles:
  • CPU Resource Group: Contains a minimum of two default CPU Resource Profiles.
  • GPU Resource Group: Contains a number of default GPU Resource Profiles determined by the number of GPUs in that group. For instance, if the group is configured with 4 GPUs, it will include two default profiles for GPU=1, two for GPU=2, and two for GPU=4.
As a site administrator you can create several different vCPU, GPU, and memory configurations which will be available when launching a session/job. When launching a new session, application, job, or a model, users will be able to select one of the available resource profiles depending on their project's requirements.

Administrators can add, edit, and delete CPU or GPU Resource Profiles from the Resource Profiles page.

  1. To create resource profiles, go to the Site Administration > Runtime page.
  2. Click Add CPU Resource Profile.
  3. Select the Resource Group name, and specify the CPU and Memory.
  4. Click Save.
  5. Click Add GPU Resource Profile.
  6. Select the Resource Group name, and specify the GPU, CPU, and Memory.
  7. Click Save.

    You will see the option to add GPUs to the resource profiles only if your Cloudera AI hosts are equipped with GPUs, and you have enabled them for use by setting the relevant properties in cdsw.conf.

    By default, the Enable CPU Bursting option is selected for the Resource Profiles to use burstable CPU settings to help better resource utilization. To use the resource profile as a hard limit on vCPU consumption, disable this CPU bursting option.

If there are two worker nodes and 10 vCPU available overall, if one user tries to establish a session with 8 vCPU, Cloudera AI will not allow it. The memory and CPU must be contiguous (adjacent to each other). When a user spins a session, the pod triggers on a single node and resources on the same node are utilized. This is expected behavior for Kubernetes.

Figure 1. Resource profiles available when launching a session