Cloudera Docs

Standard resource mode requirements

Review the memory, storage, and hardware requirements for getting started with the Cloudera Data Warehouse (CDW) service in standard resource mode on Red Hat OpenShift and Embedded Container Service.

To get started with the CDW service on standard resource mode, make sure you have fulfilled the following requirements:

  • CDP Cloudera Manager must be installed and running.
  • CDP Private Cloud must be installed and running. See Installing on OpenShift and Installing on ECS for more details.
  • An environment must have been registered with Management Console on the private cloud. See CDP Private Cloud Environments for more details.
  • In addition to the general requirements, CDW also has the following minimum memory, storage, and hardware requirements for each worker node using the standard resource mode:

Depending on the number of executors you want to run on each physical node, the per-node requirements change proportionally. For example, if you are running 3 executor pods per physical node, you require 384 GB of memory and approximately 1.8 TB of locally attached SSD/NVMe storage.

The following table lists the minimum and recommended compute (processor), memory, storage, and network bandwidth required for each OpenShift or ECS worker node using the Standard Resource Mode for production use case. Note that the actual node still needs some extra resources to run the operating system, Kubernetes engine, and Cloudera Manager agent on ECS.

Component Minimum Recommended
Node Count 4 10
CPU per worker 16 cores [or 8 cores or 16 threads that have Simultaneous Multithreading (SMT) enabled] 32+ cores (can also be achieved by enabling SMT)
Memory per worker 128 GB per node 384 GB* per node

FAST (Fully Automated Storage Tiering) Cache - Locally attached SCSI device(s) on every worker.

Preferred: NVMe and SSD.

OCP uses Local Storage Operator. ECS uses Local Path Provisioner.

 1.2 TB* SATA, SSD per host 1.2 TB* NVMe/SSD per host
Network Bandwidth 1 GB/s guaranteed bandwidth to every CDP Private Cloud Base node 10 GB/s guaranteed bandwidth to every CDP Private Cloud Base node
* Depending on the number of executors you want to run on each physical node, the per-node requirements change proportionally. For example, if you are running 3 executor pods per physical node, you require 384 GB of memory and approximately 1.8TB (600GB per executor) of locally attached SSD/NVMe storage for FAST Cache.