Setting Cloudera Manager Configurations

You can use Cloudera Director to set configurations for the various Cloudera Manager entities that it deploys:
  • Cloudera Manager
  • Cloudera Management Service
  • The various CDH components, such as HDFS, Hive, and HBase
  • Role types, such as NameNode, ResourceManager, and Impala Daemon
This functionality is available for both Cloudera Director client and Cloudera Director server:
  • Client - Using the configuration file.
  • Server - Using the Cloudera Director web UI or APIs (Java, REST, or Python).
    • To use the REST API, you can submit JSON documents to the REST service endpoint, or access the API console at http://director-server-hostname:7189/api-console.
    • You can find information about the Cloudera Director Java and Python APIs on the director-sdk GitHub page.
    • In the web UI, you can specify custom values for Cloudera Manager configurations when adding an environment or creating a Cloudera Manager cluster.

Cloudera Director enables you to customize deployment and cluster setup, and configurations are applied on top of Cloudera Manager default and automatic host-based configuration of services and roles. Set configurations either in the deployment template or in the cluster template.

Cluster Configuration Using Cloudera Manager

Some configuration changes can safely be made to Cloudera Director-managed clusters using Cloudera Manager directly. For these use cases, Cloudera Director will sync up automatically with changes made in Cloudera Manager. Other configuration changes cannot be safely made using Cloudera Manager directly because Cloudera Director will not become aware of the change, resulting in failures when a user later tries to expand or otherwise modify the cluster.

For information on configuration changes and other changes to clusters that can and cannot be safely made directly through Cloudera Manager, see Modifying or Updating Clusters Using Cloudera Manager.

Setting up a Cloudera Manager License

There are three ways to set up a Cloudera Manager license using Cloudera Director, each corresponding to a field within the Licensing configuration section of the aws.conf configuration file. The three are mutually exclusive.
  • license field - You can embed license text in the license field of the configuration file. (Cloudera recommends using triple quotes (""") for including multi-line text strings, as shown in the commented-out lines of the configuration file.) To embed a license in the license field, find the Licensing configuration section of the configuration file and enter the appropriate values.
  • licensePath field - The licensePath field can be used to specify the path to a file containing the license.
  • enableEnterpriseTrial field - The enableEnterpriseTrial flag indicates whether the 60-Day Cloudera Enterprise Trial should be activated when no license is present. This must not be set to true if a license is included using either license or licensePath.
The License configuration section of the configuration file is shown below: 
# 
# Embed a license for Cloudera Manager 
# 

# license: """ 
# -----BEGIN PGP SIGNED MESSAGE----- 
# Hash: SHA1 
# 
# { 
# "version" : 1, 
# "name" : "License Owner", 
# "uuid" : "license id", 
# "expirationDate" : 0, 
# "features" : [ "FEATURE1", "FEATURE2" ] 
# } 
# -----BEGIN PGP SIGNATURE----- 
# Version: GnuPG v1.4.11 (GNU/Linux) 
# 
# PGP SIGNATURE 
# -----END PGP SIGNATURE----- 
# """ 

# 
# Include a license for Cloudera Manager from an external file 
# 
# licensePath: "/path/to/license.txt.asc" 

# 
# Activate 60-Day Cloudera Enterprise Trial 
# 
enableEnterpriseTrial: true

For more information about Cloudera Manager licenses, see Managing Licenses in the Cloudera Manager documentation.

Deployment Template Configuration

This section shows the structure of the Cloudera Manager deployment configuration settings in both the configuration file and the API.

Configuration File

Using the configuration file, the configs section in the deployment template has the following structure: 
cloudera-manager {
   ...
   configs {
      # CLOUDERA_MANAGER corresponds to the Cloudera Manager Server configuration options 
      CLOUDERA_MANAGER { 
         enable_api_debug: false 
      } 
   
      # CLOUDERA_MANAGEMENT_SERVICE corresponds to the Service-Wide configuration options 
      CLOUDERA_MANAGEMENT_SERVICE { 
         enable_alerts : false 
         enable_config_alerts : false 
      } 

      ACTIVITYMONITOR { ... } 

      REPORTSMANAGER { ... } 

      NAVIGATOR { ... } 

      # Added in Cloudera Manager 5.2+ 
      NAVIGATORMETASERVER { ... } 

      # Configuration properties for all hosts 
      HOSTS { ... } 
   }
   ...
}

API

Using the API, the configs section for deployment templates has the following structure: 
{
   "configs":  {
      "CLOUDERA_MANAGER": {
         "enable_api_debug": "true"
      },
      "CLOUDERA_MANAGEMENT_SERVICE": {
         "enable_alerts": "false"
      }
   }
}

Cluster Template Service-wide Configuration

This section shows the structure of the Cloudera Manager service-wide configuration settings in both the configuration file and the API.

Configuration File

Using the configuration file, the configs section for service-wide configurations in the cluster template has the following structure: 
cluster {
   ...
   configs {
      HDFS {
         dfs_block_size: 1342177280
      } 
      MAPREDUCE {
         mapred_system_dir: /user/home
         mr_user_to_impersonate: mapred1
      }
   }
   ...
}

API

Using the API, the service-wide configurations block in the ClusterTemplate is labelled servicesConfigs, and has the following structure: 
{
   "servicesConfigs": {
      "HDFS": {
         "dfs_block_size": 1342177280
      },
      "MAPREDUCE": {
         "mapred_system_dir": "/user/home",
         "mr_user_to_impersonate": "mapred1"
      }
   }
}

Cluster Template Roletype Configurations

This section shows the structure of the Cloudera Manager roletype configuration settings in both the configuration file and the API.

Configuration File

Using the configuration file, roletype configurations in the cluster template are specified per instance group: 
cluster {
   ...
   masters {
      ...
      # Optional custom role configurations
      configs {
         HDFS {
            NAMENODE {
               dfs_name_dir_list: /data/nn
               namenode_port: 1234
            }
         }
      }
      ...
   }
   ...
}

API

Using the API, roletype configurations in the cluster template are specified per instance group: 
{
   "virtualInstanceGroups" : {
      "configs": {
         "HDFS": {
            "NAMENODE": {
                "dfs_name_dir_list": "/data/nn",
                "namenode_port": "1234"
            }
         }
      }
   }
}