Managing Cloudera Data WarehousePDF version

List of predefined resource templates

Cloudera Data Warehouse provides predefined resource templates for Hive, Impala, Database Catalog, and Data Visualization pods. The Default Resources template is used whenever you create a new Virtual Warehouse and Data Visualization instance, or edit an existing Database Catalog. Medium and Large Resources are double and triple the default preset respectively. For initial exploration and proof of concept use cases, you can use the Reduced Resources template. The predefined resource templates are read-only.

Table 1. Default resources for Data Visualization subcomponents
Resource type Resource limit for webapp
CPUs 2
Memory 8192 MB
Table 2. Default resources for Database Catalog subcomponents
Resource type Resource limit
Hive Query Processor HMS
CPUs 2 4
Memory 8192 MB 24576 MB
Xmx - 6432 M
Table 3. Default resources for Hive subcomponents
Resource type Resource limit
HS2 Hue backend Query coordinator Query executor Standalone compute operator Standalone query executor
CPUs 4 1 1 12 0.2 14
Memory 16384 MB 8192 MB 4096 MB 116736 MB 2048 MB 120832 MB
Scratch - - - 280 GiB - 280 GiB
Cache - - - 280 G - 280 GiB
Overhead size - - - 40 GiB - 40 GiB
Xms 8 G - 2 G 24 G - 49 G
Xmx 11468 M - 2457 M 48 G - 98 G
Xss - - - 512 k - 256 k
Max Direct Memory Size - - - 64 G - 20 G
Wait Queue size - - - 10 - 10
Table 4. Default resources for Impala subcomponents
Resource type Resource limit
Huebackend autoscaler catalogd Usage monitor Impala coordinator Impala executor Impala proxy statestored
CPUs 1 1 1 0.1 14 14 1 1
Memory 8192 MB 1024 MB 8192 MB 256 MB 112640 MB 116736 MB 1024 MB 1024 MB
Scratch - - - - 300 GiB 300 GiB - -
Cache - - - - 200 GiB 200 GiB - -
Overhead size - - - - 58 GiB 58 GiB - -
Xms - - 2 G - 2 G 2 G - -
Xmx - - 4 G - 25 G 4 G - -
Xss - - - - - - - -
Max Direct Memory Size - - - - - - - -
Wait Queue size - - - - - - - -
Table 5. Default resources for Hive and Impala subcomponents required to run Unified Analytics
Resource type Resource limit
HS2 Query coordinator Query executor Standalone query executor
CPUs 4 1 12 14
Memory 16384 MB 4096 MB 116736 MB 120832 MB
Scratch - - 280 GiB 280 GiB
Cache - - 280 GiB 280 GiB
Overhead size - - 40 GiB 40 GiB
Xms 8 G 2 G 24 G 49 G
Xmx 11468 M 2457 M 48 G 98 G
Xss - - 256 k 256 k
Max Direct Memory Size - - 64 G 20 G
Wait Queue size - - 10 10
Memory
Physical Random Access Memory (RAM) available on a node (also called a worker machine) in a cluster. This memory resource is crucial for running pods in Kubernetes.
CPU
CPU (Central Processing Unit) refers to the processing power available on a node in the cluster.
Xmx
Maximum memory allocation pool for a Java Virtual Machine (VM).
Xms
Initial memory allocation pool for a Java VM.
Xss
Java VM configuration for -Xss (thread stack size).
Cache size
Size of the data cache.
Scratch size
Limit of Impala scratch space.
Overhead size
Size for resources used by tools run by the containers.
Max Direct Memory Size
Java VM configuration for -XX:MaxDirectMemorySize (limit for Direct Byte Buffers).
Wait queue size
Overhead buffer for hive.query.isolation.slots.per.node. The total number of concurrent tasks the Hive query executor can process is the sum of the available CPU cores and the value you specify in this field.