You configure Workload Aware Autoscaling, available as a technical preview, when you
create a Virtual Warehouse following a step-by-step procedure using the CDW UI.
You are creating a Virtual Warehouse for running BI-type queries.
You obtained the DWAdmin role.
You perform the initial configuration of Workload Aware Autoscaling when you create a
Virtual Warehouse, and later, you can edit your configuration. Tune your
configuration based on meeting an SLA or reducing cost.
Create an Impala Virtual Warehouse completing steps in Creating a Virtual Warehouse up to,
and including, naming and selecting a Database Catalog for the Virtual
Warehouse.
In Size, select use workload aware
autoscaling.
In Small, which is an executor group set, configure
Executors Per Group.
The number of executors per group is analogous to the number of nodes you
would need on a bare metal system to handle your workload. A small executor
group set helps improve cost and prevents larger groups from starting up.
In Large, which is your second executor group set,
configure Executors Per Group.
Choose a value that is a multiple of the value you set for the small
Executors Per Group. The greater the spread
between values of Executors Per Group the better the query routing.
In Min No. of Groups, set the initial number of groups
to use with no scaling down unless the Virtual Warehouse is suspended.
In Max No. of Groups, set the upper limit for scaling,
which is often based on budget or intolerance for query latency.
An executor group set can contain up to 200 executors total. A group size
(number of executors per group) must be within 1-100. The max no. of groups
is limited by the group size . For example, a set with group size 2 can have
up to 100 groups. A set with group size 100 can have up to 2 groups.
Optionally, click Advanced Customizations for an
executor group to configure Disable AutoSuspend, AutoSuspend Timeout, Scale Up
Delay, Scale Down Delay.
Optionally, click Add Custom Set to configure more
scaling flexibility.
First, try using Workload Aware Autoscaling without a custom group. Later,
if warranted, add up to three custom sets for a total of five executor group
sets. Only use more than two executor groups under the following conditions:
Your workload is well understood.
You have a large mixed workload with clear categories of queries.
A row for configuring the Custom Set appears.
In Custom1, configure executors per group, minimum number of groups, and
maximum number of groups.
Executors per group for a custom set cannot be less than executors per group
for the small set or greater than executors per group for the large set.
Custom sized executor group sets are named group-set-custom-1,
group-set-custom-2, and so on.
Run varying workloads, establish benchmarks for your queries, and add or delete custom sets.
Tune your configuration using the REQUEST_POOL query option or
COMPUTE_PROCESSING_COST query option.
