Customizing CDSW for migrating host mounts

For security reasons, CML does not allow you to mount a directory directly from hosts. You need to customize CDSW runtime to make contents of a directory available to your CML workloads.

Before migration, you must perform a few pre-migration steps if you mounted additional dependencies from the host in CDSW. For example, you might have mounted directories containing libraries needed for running your workloads. You need to make these libraries available in CML. In the pre-migration steps below, you set up CDSW for the migration to mount your libraries in CML.

If you loaded libraries from a host mount in the past, Cloudera recommends you create a custom runtime in CDSW, change the project to use the new custom runtime, and then do the migration. However, for anything other than the libraries, load the data to all the sessions in CML using the custom runtime addons procedure after migration to mount data in all the workloads in CML. Custom runtime addons do not allow writes to the file system as the host mount in CDSW does.

  1. Create a customized ML runtime.
  2. If libraries were loaded from the host mount, configure your CDSW project to use the custom runtime by adding the custom runtime to CDSW before migration.
    Libraries you add to the custom runtimes will be available to the CML projects using that custom runtime.