Adding ML Runtimes using Runtime Repo files

Runtime Repo files are JSON files that contain all the details of ML Runtimes that are needed by Cloudera AI to add these ML Runtimes to the Runtime Catalog. When these files are hosted on URLs accessible to Cloudera AI, these URLs can be registered in Cloudera AI. If Enable Runtime Updates is selected in Site Administration > Runtime, ML Runtimes that appear in the Runtime Repo files will be automatically added to the Runtime Catalog within 24 hours. Additionally, you can click Update Runtimes now to immediately update the Runtime Catalog. Using Runtime Repo files, both Custom and Cloudera provided Runtimes can be added to Cloudera AI Workbench.

To add, edit, or remove Runtime Repo files

Log in as administrator.
Navigate to Site Administration > Runtime.
Add, edit or delete Runtime Repo files in the Runtime Updates section.

Cloudera provided Runtime Repo files

Cloudera AI hosts two public Runtime Repo files that contain the details of the latest released ML Runtimes and Data Visualization Runtimes.

Name: Cloudera ML Runtimes: URL: https://archive.cloudera.com/ml-runtimes/latest/artifacts/repo-assembly.json
Name: Cloudera DataViz Runtimes: URL: https://archive.cloudera.com/cdv/latest/artifacts/repo-assembly.json

These Runtime Repo files are not added to Cloudera AI Private Cloud clusters by default. However, if your cluster is not air gapped, you can add these Runtime Repo files to your Cloudera AI Workbench to keep your Runtime Catalog automatically updated with the latest released Cloudera Runtimes. Note, that these Runtimes will be added to your Runtime Catalog from Cloudera hosted registries, even if your Private Cloud is configured to use a different docker registry.

If your cluster is air-gapped, you can still add new Cloudera-built ML Runtimes to your Cloudera AI Workbench by pushing these Runtimes into a docker registry that is hosted within your air-gapped environment, and then adding a Runtime Repo file to Cloudera AI. The image_identifier fields in the file must point to the self-hosted docker registry instead of any Cloudera-hosted registries.

Self created Runtime Repo files

You can create your own Runtime Repo files and register them in Cloudera AI. Cloudera AI checks these Runtime Repo files for changes every 24 hours and adds any new ML Runtimes found in these files automatically to the Runtime Catalog.

To create a Repo assembly file:

Create a JSON file with the same structure as the Cloudera provided one:

{
    "assembly_metadata_version": 1,
    "runtimes": [
         { 
           "image_identifier": string,
            "runtime_metadata_version": int,
            "editor": string,
            "edition": string,
            "description": string,
            "kernel": string,
            "full_version": string,
            "short_version": string,
            "maintenance_version": int,
            "git_hash": string,
            "gbn": int
         } , 
    ]
}

Fill in the details of one or more ML Runtimes. If you are adding Cloudera-created Runtimes, use the values from the Cloudera-provided Runtime Repo files. For Custom Runtimes, git_hash should be an empty string, and gbn should be set to zero. All other fields should be filled according to the information in Metadata for Custom ML Runtimes.
Host the JSON file on an URL that Cloudera AI is able to access.
Add the Runtime Repo file to Cloudera AI on the Site Administration > Runtime page.