Persistent Volume usage for model storage in Cloudera AI Inference service

Persistent volume (PV) storage in Cloudera AI Inference service provides dedicated storage for model artifacts, eliminating dependency on the underlying node storage capacity.

You can use persistent volume storage to host large models when repeated downloads are impractical, such as when the model size exceeds the available root volume capacity. Cloudera AI Inference service downloads the model artifacts once and reuses them across workload restarts, avoiding the need to download the artifacts each time you recreate a workload.

You can specify the persistent volume (PV) size when deploying a Model Endpoint. After deployment, the PV size can be modified during Model Endpoint updates.

When you delete the endpoint, Cloudera AI Inference service automatically deletes the associated persistent volume claim (PVC).

Sizing guidance

Cloudera recommends allocating a persistent volume size that is at least double the size of the model artifacts. This ensures sufficient capacity for the overhead of temporary files created during the download process. For example, if your model artifacts are 350 GB, Cloudera recommends specifying a PV size of 700 GB or more.

This feature is available only for Cloudera AI on premises in 1.5.5 SP4 and higher releases.