Storage requirements
Storage requirements for Data Services.
Storage Requirements
| Data Services | Storage type | Storage required | Purpose |
|---|---|---|---|
| CDE | Block | 500GB per Virtual Cluster in Embedded NFS | Stores all information related to virtual clusters |
| CDW | Local | 100 GB per executor in LITE mode and 600 GB per executor in FULL mode | Used for caching |
| Control Plane | Block | 118 GB total if using an External Database, 318 GB total if using the Embedded Database | Storage for CDP infrastructure including Fluentd logging, Prometheus monitoring, and Vault. Backing storage for an embedded DB for control plane configuration purpose, if applicable |
| CML | Block | 600 GB per node (minimum), 4.5 TB (recommended) | Stores all CML workspace information |
| External NFS or Block | 1 TB per Node | Stores all user project files. VFS storage can either use Longhorn NFS-provisioner on Longhorn OR directly connect to your NFS. | |
| MonitoringApp | Block | 30 GB + (Env cnt x 100 GB) | Stores metrics collected by Prometheus. |
| Data Catalog | Requires Control Plane database and not a dedicated storage space | 100 GB extra in Control Plane database | Stores profiling metadata. |
