Cloudera Docs

Managing Engine Images

By default, Cloudera Data Science Workbench ships a base engine image that includes kernels for Python, R, and Scala, along with some additional libraries that can be used to run common data analytics operations. Occasionally, new engine versions are released and shipped with Cloudera Data Science Workbench releases.

Engine images are available in the Site Administrator panel at Admin > Engines, under the Engine Images section.

There are two types of default engines: ML Runtime and Legacy Engines. Legacy engines contain the machinery necessary to run sessions using all four interpreter options that CML currently supports (Python 2, Python 3, R and Scala) and other support utilities (C and Fortran compilers, LaTeX, etc.). ML Runtimes are thinner and more lightweight than legacy engines. Rather than supporting multiple programming languages in a single engine, each Runtime variant supports a single interpreter version and a subset of utilities and libraries to run the user’s code in Sessions, Jobs, Experiments, Models, or Applications.

As a site administrator, you must select a Legacy Engine version that will be used by default for projects that are set to use Legacy Engines. Furthermore, project administrators can explicitly select which Legacy Engine image should be used as the default image for a project.

  1. Click Admin > Runtime/Engine.
  2. Choose the Legacy Engine you would like to use as the default for projects that are set to use Legacy Engines. One Legacy Engine image must be selected as a default.
  3. Modify the remaining information on the page:
    • Resource Profiles listed in the table are selectable resource options for both legacy Engines and ML Runtimes (for example, when starting a Session or Job)
    • The remaining information on the page applies to site-level settings specific for legacy Engines.

If a user publishes a new custom Docker image, site administrators are responsible for white-listing such images for use across the deployment. For more information on creating and managing custom Docker images, see https://docs.cloudera.com/cdsw/1.10.5/extensible-engines/topics/cdsw-extensible-engines.html.