YARN components for CSI support
To support Container Storage Interface on YARN, you must deploy the CSI driver associated with the external storage volume on each NodeManager. YARN includes the following additional components for CSI support: volume manager and CSI driver adapter.
The following diagram outlines the interactions between the YARN components and the external
storage device with the help of the CSI driver:
CSI Driver
Provided by the external storage system to interact with YARN, the CSI driver implements all the APIs required to manage the lifecycle of the storage volume. The CSI driver comprises of three gRPC services: identity, controller, and node. The CSI driver is deployed on NodeManagers such that every NodeManager runs instances of identity and node services while one of the cluster nodes runs an instance of the controller service.
Volume Manager
Reads the volume resource specification and handles the required volume operation with the help of the controller service on the CSI driver. This component manages the lifecycle of the storage volume in YARN.
CSI Driver Adapter
Runs on all NodeManagers along with the CSI driver. The adapter proxies all invocations between YARN and the CSI driver. In addition, the adapter registers with the volume manager so that either of them can work with the controller service for handling the volume operations.