Managing Resource Profiles
Resource profiles define how many vCPUs and how much memory Cloudera Data Science Workbench will reserve for a particular workload (for example, session, job, model).
As a site administrator you can create several different vCPU, GPU, and memory configurations which will be available when launching a session/job. When launching a new session, users will be able to select one of the available resource profiles depending on their project's requirements.
To create resource profiles, go to the page, under Resource Profiles. Cloudera recommends that all profiles include at least 2 GB of RAM to avoid out of memory errors for common user operations.
You will
see the option to add GPUs to the resource profiles only if your Cloudera Data Science
Workbench hosts are equipped with GPUs, and you have enabled them for use by setting the
relevant properties either in Cloudera Manager (for CSD) or in cdsw.conf (for RPM).
Burstable CPUs
CDSW configures no upper bound on the CPU resources that Workloads can use so that they can use all of the CPU resources available on the node where they are running. By configuring no CPU limits, CDSW enables efficient use of the CPU resources available on your cluster nodes:
- If the CPUs are idle then the workloads can burst and take advantage of the free CPU cycles. For example, if you've launched a session with 1vCPU but the code inside it requires more than 1vCPU, the workload container can consume all the available CPU cycles on the node where it's launched.
- When the cluster is highly utilized and CPU resources are sparse, Workloads will be limited to use the number of CPU resources configured in their resource profile.
- If multiple containers are attempting to use excess CPU, CPU time is distributed in proportion to the amount of CPU initially requested by each container.
