CML API v2 (Beta)

Cloudera Machine Learning exposes a REST API that you can use to perform operations related to projects, jobs, and runs. You can use API commands to integrate CML with third-party workflow tools or to control CML from the command line.

API v2 supercedes the existing Jobs API. For more information on the Jobs API, see Jobs API in the Related information section, below.

How to view the API Specification

You can obtain the specification of the available API commands directly from CML. In a browser, enter the following addresses:
  • REST API: https://<domain name of CML instance>/api/v2/swagger.html
  • Python API: https://<domain name of CML instance>/api/v2/python.html
You can also get json formatted output, by specifying swagger.json.

Quickstart

API key authentication

To get started, generate an API key. The API key is a randomly generated token that is unique to each user. It must be treated as highly sensitive information because it can be used to start jobs via the API. You need this API key to use in API calls.
  1. Sign in to Cloudera Machine Learning.
  2. In User Settings > API Keys, click Create Model API Key.
  3. Copy this API key to the clipboard.

Using curl from the command line

To use the curl command, it is convenient to store the domain and API key in environmental variables, as shown here:
  1. Copy the Model API key.
  2. Open a terminal, and store it to a variable. On unix-based systems:

    export API_KEY=<paste the API key value here>

  3. Copy the domain name, which is in the address bar of the browser. On unix-based systems: export DOMAIN=<domain> (a value like: ml-xxxx123456.com).

Example commands

If you have some projects, jobs, and runs already set up in your ML workspace, here are some commands to try:
  • List available projects:
    curl -X GET -H "authorization: Bearer $API_KEY" https://$DOMAIN/api/v2/projects | jq

    You can format the output for readability by piping through jq, a formatting utility.

  • You can filter the output like so:
    curl -X GET -H “authorization: Bearer $API_KEY” https://$DOMAIN/api/v2/projects?searchFilter=demo |  jq

    The output is limited to those projects that have the word “demo” in them.

    You can also paginate the output, for example by limiting each page to two projects. To do this, replace the string starting from the ‘?’ character with this: ?pageSize=2

    The output ends with a next_page_token and a string value. To get the next page use this: ?pageSize=2&pageToken=<token>

Using the Python client library

To use the Python API in your own code, first install the Python API client and point it to your cluster.

pip3 install https://$DOMAIN/api/v2/python.tar.gz

Include the following code, and specify the values for <DOMAIN> and <API_KEY> with variables or values for your installation.

import cmlapi
config = cmlapi.Configuration()
config.host = "<DOMAIN>"
client = cmlapi.ApiClient(config)
client.set_default_header("authorization", "Bearer "+"API_KEY")
api_instance = cmlapi.ApiServiceApi(client)

Then you can use commands in your script, such as a call to list projects:

projects = api_instance.list_projects()

Check the Python documentation for a full list of the available API commands.

Using the Python client library in the REPL

Here is an example of a stub Python script that contains the environmental variables for your installation. Save it to your computer and run it locally to get a Python prompt for API commands.

demo.py

import clap
import argparse
    
parser = argparse.ArgumentParser(description=‘Test the generated python package.’)
parser.add_argument(“—host”, type=str, help=‘The host name of your workspace”)
parser.add_argument(“—token”, type=str, help=‘Your API key”)
args = parser.parse_args()
    
config = clap.Configuration()
config.host = ars.host
client = cmlapi.ApiClient(config)
client.set_default_header(“authorization”, “Bearer “ + args.token)
api = cmlapi.Apiapi(client)
Run the script from your command line:
python3 -i demo.py —host https://$DOMAIN —token $API_KEY
This returns a Python prompt with api available. You can run api calls from the prompt, as follows:
>>> api
<cmlapi.api.api_api.ApiApi object at 0xlasjoid>
>>> api.api_list_projects()
    
You can specify a search filter, such as the following:
api.api_list_projects(searchFilter=‘demo’)
     
api.api_list_projects(page_size=2)
     
api.api_list_projects(page_size=2, page_token=‘<token value>’)