Additional resource requirements for Cloudera Data Warehouse

Standard resource mode requirements for Cloudera Data Warehouse.

The following table lists the minimum and recommended compute (processor), memory, storage, and network bandwidth required for each OpenShift or ECS worker node using the Standard Resource Mode for production use case. Note that the actual node still needs some extra resources to run the operating system, Kubernetes engine, and Cloudera Manager agent on ECS.

Component Minimum Recommended
Node Count 10 20
CPU per worker 16 cores [or 8 cores or 16 threads that have Simultaneous Multithreading (SMT) enabled] 32+ cores (can also be achieved by enabling SMT)
Memory per worker 128 GB per node 384 GB* per node

FAST (Fully Automated Storage Tiering) Cache - Locally attached SCSI device(s) on every worker.

Preferred: NVMe and SSD.

OCP uses Local Storage Operator. ECS uses Local Path Provisioner.

1.2 TB* SATA, SSD per host 1.2 TB* NVMe/SSD per host

Persistent Volume (PV) Block Storage.

On OCP, block from a Container Storage Interface (CSI)-compliant block provider, such as OpenShift Container Storage.

ECS uses an embedded distributed block provider that aggregates local disks of workers.

Other Data Services additionally have added capacity requirements on this service.

Approximately 100 GB per Virtual Warehouse Approximately 100 GB per Virtual Warehouse
Network Bandwidth 1 GB/s guaranteed bandwidth to every CDP Private Cloud Base node 10 GB/s guaranteed bandwidth to every CDP Private Cloud Base node
* Depending on the number of executors you want to run on each physical node, the per-node requirements change proportionally. For example, if you are running 3 executor pods per physical node, you require 384 GB of memory and approximately 1.8TB (600GB per executor) of locally attached SSD/NVMe storage for FAST Cache.