Using ML Runtimes Addons
ML Runtime Addons allow you to add Spark and Hadoop CLI to sessions run on projects using ML Runtime images.
While Legacy Engines include support for both Spark and Hadoop CLI, ML Runtimes do not contain Spark and Hadoop CLI binaries to keep them small and lean. Instead Spark and Hadoop binaries are stored in persistent storage on an NFS server and can be added to your ML Runtime sessions.