Cloudera Docs

Custom Runtime addons with Cloudera AI

Custom ML Runtimes enable you to create runtimes with your own choice of libraries and applications, but it is also possible to further customize existing ML Runtimes with additional configuration files or binaries such as connection drivers, without the effort of creating a new custom ML Runtime.

How this feature works

  1. Package your dependencies into a tarball file, following the specified format.
  2. The site administrator shall register the custom runtime addon by uploading the tarball file via an API call on the command line.
  3. After a custom runtime addon has been registered, the site administrator can enable or disable addons in the UI.
  4. When an addon is enabled, it is mounted to all newly started runtime-based workloads, in read-only mode.

Creating the Custom Runtime Addon

  1. You must create a structure of files and folders that follows the pattern shown here, and package it as a tar.gz archive. For example, to mount a folder called custom_folder, with all of its content under /usr/bin/custom_folder, and custom_file at /usr/custom_file, the internal structure of the tarball shall look like this:
  2. Create a json file that describes the custom runtime addon to register in Cloudera AI. The json file shall follow this structure: 
    {
    "name" : "name-of-the-addon",
    "spec" : {"paths" : ["/usr/custom_file", "/usr/bin/custom_folder"]}
}
  3. Finally, the json file and tar.gz file are uploaded by a site administrator. Use the following curl command, with these substitutions:
    • metadata.json: the name of the json file
    • addon.tar.gz: the name of the tar.gz file
    curl -v -L '<cml_cluster>/api/v2/runtimeaddons/custom'  -F metadata=@metadata.json 
-F tarball=@addon.tar.gz  -H 'Authorization: Bearer <APIv2 key>'

    After the runtime add-on has been uploaded, it will be enabled and mounted to all newly started Runtime-based workloads.

Requirements

  1. Name of the addon must contain only lowercase letters, numbers, underscores and hyphens. Also, it must be no longer than 35 characters. When added to Cloudera AI, the addon will be prefixed with the term "custom-addon-".
  2. There must be at least one path mentioned in the uploaded json file.
  3. Paths in the metadata file shall not end with a forward slash ( / ) character.
  4. Requested paths will be symlinked in the workloads’s filesystem by the cdsw user, therefore these locations must not exist in the workload’s file system and cdsw user must have write access to these locations.

Limitations

  • Injecting Python or R libraries into workloads via Custom Runtime Addons is not supported. To accomplish this, create a Custom Runtime instead. For more information, see Creating Customized ML Runtimes.
  • Misconfigured Custom Runtime Addons can break user workloads (for example, if metadata refers to paths that do not exist in the uploaded tarball archive).
  • It is not possible to add Custom Runtime Addons through the UI.
  • Custom Runtime Addons can not be updated or removed, only disabled.
  • Custom Runtime Addons are mounted to workloads in read only mode.