Cloudera Manager API

The Cloudera Manager API provides configuration and service lifecycle management, service health information and metrics, and allows you to configure Cloudera Manager itself. The API is served on the same host and port as the Cloudera Manager Admin Console, and does not require an extra process or extra configuration. The API supports both HTTP Basic Authentication and Kerberos (SPNEGO), accepting the same users and credentials as the Cloudera Manager Admin Console. Kerberos authentication is only supported from clients (such as curl) that can use SPNEGO. The Cloudera Manager Swagger and Python clients do not support SPNEGO.

If you have already enabled Kerberos authentication, you can enable it for the API by doing the following:

  1. If you have not already done so, enable Kerberos for cluster services.
  2. Navigate to Administration > Settings.
  3. Enter SPNEGO in the Search field.
  4. Check the box labeled Enable SPNEGO/Kerberos Authentication for the Admin Console and API. Leave the other SPNEGO settings blank to allow Cloudera Manager to automatically generate the principal and keytab.
  5. Click Save Changes.
  6. Restart Cloudera Manager Server.

You can access the Cloudera Manager Swagger API user interface from the Cloudera Manager Admin Console. Go to Support > API Explorer to open Swagger.

Continue reading:

API Documentation Resources

Obtaining Configuration Files

  1. Obtain the list of a service's roles: 
    http://cm_server_host:7180/api/v32/clusters/clusterName/services/serviceName/roles
  2. Obtain the list of configuration files a process is using: 
    http://cm_server_host:7180/api/v32/clusters/clusterName/services/serviceName/roles/roleName/process
  3. Obtain the content of any particular file: 
    http://cm_server_host:7180/api/v32/clusters/clusterName/services/serviceName/roles/roleName/process/
configFiles/configFileName
    For example: 
    http://cm_server_host:7180/api/v32/clusters/Cluster%201/services/OOZIE-1/roles/
OOZIE-1-OOZIE_SERVER-e121641328fcb107999f2b5fd856880d/process/configFiles/oozie-site.xml

Retrieving Service and Host Properties

To update a service property using the Cloudera Manager APIs, you'll need to know the name of the property, not just the display name. If you know the property's display name but not the property name itself, retrieve the documentation by requesting any configuration object with the query string view=FULL appended to the URL. For example: 
http://cm_server_host:7180/api/v32/clusters/Cluster%201/services/service_name/config?view=FULL
Search the results for the display name of the desired property. For example, a search for the display name HDFS Service Environment Advanced Configuration Snippet (Safety Valve) shows that the corresponding property name is hdfs_service_env_safety_valve: 
{
   "name" : "hdfs_service_env_safety_valve",
   "require" : false,
   "displayName" : "HDFS Service Environment Advanced Configuration Snippet (Safety Valve)",
   "description" : "For advanced use onlyu, key/value pairs (one on each line) to be inserted into a roles
   environment. Applies to configurations of all roles in this service except client configuration.",
   "relatedName" : "",
   "validationState" : "OK"
}
Similar to finding service properties, you can also find host properties. First, get the host IDs for a cluster with the URL: 
http://cm_server_host:7180/api/v32/hosts
This should return host objects of the form: 
{
   "hostId" : "2c2e951c-aaf2-4780-a69f-0382181f1821",
   "ipAddress" : "10.30.195.116",
   "hostname" : "cm_server_host",
   "rackId" : "/default",
   "hostUrl" : "http://cm_server_host:7180/cmf/hostRedirect/2c2e951c-adf2-4780-a69f-0382181f1821",
   "maintenanceMode" : false,
   "maintenanceOwners" : [ ],
   "commissionState" : "COMMISSIONED",
   "numCores" : 4,
   "totalPhysMemBytes" : 10371174400
}
Then obtain the host properties by including one of the returned host IDs in the URL: 
http://cm_server_host:7180/api/v32/hosts/2c2e951c-adf2-4780-a69f-0382181f1821?view=FULL

Backing Up and Restoring the Cloudera Manager Configuration

You can use the Cloudera Manager REST API to export and import all of its configuration data. The API exports a JSON document that contains configuration data for the Cloudera Manager instance. You can use this JSON document to back up and restore a Cloudera Manager deployment.

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

Exporting the Cloudera Manager Configuration

  1. Log in to the Cloudera Manager server host as the root user.
  2. Run the following command: 
    # curl -u admin_uname:admin_pass "http://cm_server_host:7180/api/v32/cm/deployment" >
path_to_file/cm-deployment.json
    Where:
    • admin_uname is a username with either the Full Administrator or Cluster Administrator role.
    • admin_pass is the password for the admin_uname username.
    • cm_server_host is the hostname of the Cloudera Manager server.
    • path_to_file is the path to the file where you want to save the configuration.

Redacting Sensitive Information from the Exported Configuration

The exported configuration may contain passwords and other sensitive information. You can configure redaction of the sensitive items by specifying a JVM parameter for Cloudera Manager. When you set this parameter, API calls to Cloudera Manager for configuration data do not include the sensitive information.
To configure redaction for the API:
  1. Log in the Cloudera Manager server host.
  2. Edit the /etc/default/cloudera-scm-server file by adding the following property (separate each property with a space) to the line that begins with export CMF_JAVA_OPTS: 
    -Dcom.cloudera.api.redaction=true
    For example: 
    export CMF_JAVA_OPTS="-Xmx2G -Dcom.cloudera.api.redaction=true"
  3. Restart Cloudera Manager: 
    sudo service cloudera-scm-server restart

Restoring the Cloudera Manager Configuration

Using a previously saved JSON document that contains the Cloudera Manager configuration data, you can restore that configuration to a running cluster.

  1. Using the Cloudera Manager Administration Console, stop all running services in your cluster:
    1. On the Home > Status tab, click to the right of the cluster name and select Stop.
    2. Click Stop in the confirmation screen. The Command Details window shows the progress of stopping services.

      When All services successfully stopped appears, the task is complete and you can close the Command Details window.

  2. Log in to the Cloudera Manager server host as the root user.
  3. Run the following command: 
    curl -H "Content-Type: application/json" --upload-file path_to_file/cm-deployment.json -u admin:admin http://cm_server_host:7180/api/v32/cm/deployment?deleteCurrentDeployment=true
    Where:
    • admin_uname is a username with either the Full Administrator or Cluster Administrator role.
    • admin_pass is the password for the admin_uname username.
    • cm_server_host is the hostname of the Cloudera Manager server.
    • path_to_file is the path to the file containing the JSON configuration file.
  4. Restart the Cloudera Manager Server.
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
    sudo systemctl restart cloudera-scm-server
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
    sudo service cloudera-scm-server restart