Cloudera Docs

Adding ML Runtimes using Runtime Repo files

Runtime Repo files are JSON files that contain all the details of ML Runtimes that are needed by CML to add these ML Runtimes to the Runtime Catalog. When these files are hosted on URLs accessible to CML, these URLs can be registered in CML. If Enable Runtime Updates is selected in Site Administration > Runtime, ML Runtimes that appear in the Runtime Repo files will be automatically added to the Runtime Catalog within 24 hours. Additionally, you can click Update Runtimes now to immediately update the Runtime Catalog. Using Runtime Repo files, both Custom and Cloudera provided Runtimes can be added to CML workspaces.

To add, edit, or remove Runtime Repo files

  1. Log in as administrator.
  2. Navigate to Site Administration > Runtime.
  3. Add, edit or delete Runtime Repo files in the Runtime Updates section.

Cloudera provided Runtime Repo files

CML uses Runtime Repo files to automatically add new Runtimes to CML workspaces when Runtime updates are enabled on the workspace. Cloudera hosts its own Runtime Repo files that always contain the details of the latest released ML Runtimes and Data Visualization Runtimes. By default, you can find these Cloudera hosted Runtime Repos registered in your CML workspace:
Name: Cloudera ML Runtimes
URL: https://archive.cloudera.com/ml-runtimes/latest/artifacts/repo-assembly.json
Name: Cloudera DataViz Runtime
URL: https://archive.cloudera.com/cdv/latest/artifacts/repo-assembly.json

Self created Runtime Repo files

You can create your own Runtime Repo files and register them in CML. CML checks these Runtime Repo files for changes every 24 hours and adds any new ML Runtimes found in these files automatically to the Runtime Catalog.

To create a Repo assembly file:

  1. Create a JSON file with the same structure as the Cloudera provided one: 
    {
    "assembly_metadata_version": 1,
    "runtimes": [
         { 
           "image_identifier": string,
            "runtime_metadata_version": int,
            "editor": string,
            "edition": string,
            "description": string,
            "kernel": string,
            "full_version": string,
            "short_version": string,
            "maintenance_version": int,
            "git_hash": string,
            "gbn": int
         } , 
    ]
}
  2. Fill in the details of one or more ML Runtimes. If you are adding Cloudera-created Runtimes, use the values from the Cloudera-provided Runtime Repo files. For Custom Runtimes, git_hash should be an empty string, and gbn should be set to zero. All other fields should be filled according to the information in Metadata for Custom ML Runtimes.
  3. Host the JSON file on an URL that CML is able to access.
  4. Add the Runtime Repo file to CML on the Site Administration > Runtime page.