Configuring Quotas - Technical Preview

Required Role: Site Administrator

This topic describes how to configure CPU, GPU, and memory quotas for users on a CDSW deployment. You can set default quotas for each user on the deployment as well as overriding custom quotas for specific users who might require more resources. Quotas can be enabled from the Site Administration panel.

Before you begin

By default, the quotas feature is hidden. To access this feature, first go to the CDSW service in Cloudera Manager and use the Feature flag overrides property to enable access.
  1. Log in to Cloudera Manager.
  2. Go to the CDSW service.
  3. Click Configuration.
  4. Search for the Feature flag overrides property. Add the following JSON code to enable quotas. 
    {"quotas": true}
  5. Click Save Changes.
  6. Restart the CDSW service.

    You should now be able to see the Quotas tab in the Site Admin panel in Cloudera Data Science Workbench.

Enabling Default Quotas for all CDSW Users

To enable CPU, GPU, and memory quotas for users on your CDSW deployment:

  1. Log into Cloudera Data Science Workbench with site administrator privileges.
  2. Click Admin > Quotas.
  3. Switch the toggle to ON. This applies a default quota of 2 vCPU, 8 GB memory to each user on the deployment.

    If your deployment was provisioned with GPUs, a default quota of 0 GPU per user will apply. If you want users to have access to GPUs, you must modify the default quotas as described in the next step.

  4. If you want to change the default quotas, click on Default (per user). CDSW displays the Edit default quota dialog box.
  5. Enter the CPU, Memory, and GPU quota values that should apply to all users of the deployment.
  6. Click Update.

Results

Enabling quotas will only affect new workloads. If users have already scheduled workloads that exceed the new quota limits, those will continue to run uninterrupted. If a user is over their limit, they will not be able to schedule any more workloads.

Enabling Custom Quotas for Specific Users

This section shows you how to enable custom quotas for specific users on your CDSW deployment. You can use this feature to help certain users who regularly require more resources than the defaults set in the previous section. Any custom values set here will override the default quotas.

To enable custom quotas:

  1. Log into Cloudera Data Science Workbench with site administrator privileges.
  2. Click Admin > Quotas.
  3. Under the Custom Quota section, click Add Custom Quota.
  4. Enter the Username, CPU, Memory, and GPU values that apply to the custom quota. Then click Add.

    You can use a decimal floating point value (x.01) only for the CPU and Memory settings, not for GPUs.

  5. Repeat Steps 3 and 4 to add custom quotas as needed.

    You can also duplicate an existing custom quota setting for a new user. Click the vertical ellipses at the end of the custom quota row and choose Duplicate, specify a new username, and click Add.

Results

Enabling custom quotas will only affect new workloads. If users have already scheduled workloads that exceed the new quota limits, those will continue to run uninterrupted. If a user is over their limit, they will not be able to schedule any more workloads.

Modifying Default and Custom Quotas

Editing Quotas

Modifying existing quotas will only affect new workloads. To modify an existing quota setting, click the vertical ellipses at the end of the row and choose Edit. Modify the parameters as needed and click Update to save your changes. Note that these modifications will only affect new workloads.

Deleting Quotas

To delete a custom quota, click the vertical ellipses at the end of the custom quota row and choose Remove.