Updating ML Runtime images on Cloudera AI installations

In Private Cloud ML Runtimes, the collection of docker images are bundled as part of the package. However, you can upgrade Runtime images to the latest version any time and even if you have an air-gapped installation, that is, there is no or only limited Internet access.

Airgapped installations (with no or limited Internet access)
Non-airgapped installations (with Internet access)

Download ML Runtimes from the locations specified in the repo-assembly.json file:
https://archive.cloudera.com/ml-runtimes/[***RUNTIMES VERSION***]/artifacts/repo-assembly.json
```
export ml_runtime_version=[***RUNTIME VERSION***]
wget -O repo-assembly.json
https://archive.cloudera.com/ml-runtimes/${ml_runtime_version}/artifacts/repo-assembly.json
```

Get the list of images for a particular release, that is ${ml_runtime_version} using the following command:

curl
https://archive.cloudera.com/ml-runtimes/${ml_runtime_version}/artifacts/repo-assembly.json | jq -r '.runtimes[] | .image_identifier'

Ensure that docker is running on your machine or environment.

Use the docker pull and docker save commands for each of the images in the .json manifest file to create a tar.gz file, that you can copy to your air-gapped environment.

# ensure you have enough space in image_dir
image_dir=<path to image dir in non-airgapped environment>
mkdir -p ${image_dir}
for i in $(jq -r '.runtimes[].image_identifier' repo-assembly.json)
do
 image=$(echo $i | awk -F'/' ' { printf("%s",$NF)} ')
 image_file="${image}.tar.gz"
 echo "docker pull ${i}"
 docker pull ${i}
 echo "docker save ${i} -o \"${image_dir}/${image_file}\""
 docker save ${i} -o "${image_dir}/${image_file}"
done

Transfer the tar.gz files to the air-gapped environment.
Upload the images to the existing docker registry of the air-gapped environment.

Use the docker load, docker tag and docker push commands to save the images to the air-gapped environment.

# you will need a copy of the repo-assembly.json used above

remote_image_dir=<path to image dir in airgapped environment>
DOCKER_REGISTRY_DEST=<custom docker repository:port>

for i in $(jq -r '.runtimes[].image_identifier' repo-assembly.json)
do
  image=$(echo $i | awk -F'/' ' { printf("%s",$NF)} ')
  image_file="${image}.tar.gz"
  dest_path=$(echo $i | sed "s#docker.repository.cloudera.com#${DOCKER_REGISTRY_DEST}#")

  docker load -i ${image_file}
  docker tag ${i} "${dest_path}"
  docker push "${dest_path}"
done

Make the newly added Runtime images available by following the instructions in Adding ML Runtimes using Runtime Repo files.

In non-airgapped installations (with available Internet access) upgrade ML Runtimes by setting and enabling the following options:

Download the most recent manifest and populate the Runtime Catalog by selecting Site Administration > Runtime > Update Runtimes now.
Enable automated daily upgrades to download ML Runtime images by selecting the checkbox at Site AdministrationRuntimeEnable Runtime Updates.