Cloudera Docs

Configuring Impala VW auto-scaling

Configuring the Impala Virtual Warehouse to use concurrency auto-scaling is critical for controlling cloud expenses.

  • You are familiar with the auto-scaling process.
  • You are creating a Virtual Warehouse for running BI-type queries.
  • In Cloudera Data Warehouse, you added an Impala Virtual Warehouse, configured the size of the Impala Virtual Warehouse, and configured auto-suspend as described in previous topics.
  • You obtained the DWAdmin role.
In this task, you configure the following properties:
  • Scale Up Delay: Sets the length of time in seconds to wait before adding more executors if queries wait in the queue.
  • Scale Down Delay: Sets the length of time in seconds to wait before removing executors if executor groups are idle.

The time to auto-scale up or down is affected by the underlying Kubernetes configuration.

By default Impala Virtual Warehouses can run 3 large queries per executor group. Executors can handle more queries that are simpler and that do not utilize concurrency on the executor.

.
  1. In Scale Up Delay, set the seconds to wait before adding more executors if queries wait in the queue.
  2. In Scale Down Delay, set the seconds to wait before removing executors when executor groups are idle.
  3. Click Create Virtual Warehouse.