Quota management in Cloudera Data Warehouse on premises

How quota management works in Cloudera Data Warehouse🔗

A namespace is created when you create a Database Catalog or a Virtual Warehouse. At this stage, a resource pool is created with the necessary resources under the selected parent resource pool. You can view the resource pools on the Resource Utilization page by going to the Cloudera Management Console > Resource Utilization > Quotas tab. Virtual Warehouse namespaces always reserve enough resources to run at least one executor group.

If you have configured auto-scaling of the executor groups for a Hive or an Impala Virtual Warehouse and the Virtual Warehouse needs to schedule more executors for the current load, Cloudera Data Warehouse tries to increase its resource pool’s quota. Auto-scaling fails if the requested resource pool quota exceeds the parent resource pool’s quota.

When a Virtual Warehouse scales down, Cloudera Data Warehouse lowers the quota on the Virtual Warehouse’s resource pool after downscaling. The Virtual Warehouses, even in the suspended state, will have enough resources to contain one executor group.

The default Database Catalog inherits the resource pool from the environment.

What happens when the quota is insufficient🔗

Suppose you are creating a new Cloudera Data Warehouse entity, such as a Virtual Warehouse with a specific resource pool. The Virtual Warehouse creation process fails if there are insufficient resources when a namespace is being created. However, if auto-scaling is enabled, then Cloudera Data Warehouse can provision extra worker pods and the process completes successfully.

Suppose you are moving an existing non-quota-managed Cloudera Data Warehouse entity under quota management. In that case, the operation succeeds only if there is sufficient CPU and memory for the common pods, a minimum of executors for a workload, and the Cloudera Data Warehouse entity fits into the resource pool.

Cloudera CLI support🔗

By using the --resource-pool Cloudera CLI option of the type string, you can specify the resource pool for the Cloudera Data Warehouse entities, namely the Virtual Warehouse, cluster/environment, and the Data Visualization instance, when using Cloudera CLI commands.