Custom Configuration

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

Cloudera Manager exposes properties that allow you to insert custom configuration text into XML configuration, property, and text files, or into an environment. The naming convention for these properties is: XXX Advanced Configuration Snippet (Safety Valve) for YYY or XXX YYY Advanced Configuration Snippet (Safety Valve), where XXX is a service or role and YYY is the target.

The values you enter into a configuration snippet must conform to the syntax of the target. For an XML configuration file, the configuration snippet must contain valid XML property definitions. For a properties file, the configuration snippet must contain valid property definitions. Some files simply require a list of host addresses.

The configuration snippet mechanism is intended for use in cases where there is configuration setting that is not exposed as a configuration property in Cloudera Manager. Configuration snippets generally override normal configuration. Contact Cloudera Support if you are required to use a configuration snippet that is not explicitly documented.

Service-wide configuration snippets apply to all roles in the service; a configuration snippet for a role group applies to all instances of the role associated with that role group.

Server and client configurations have separate configuration snippets. In general after changing a server configuration snippet you must restart the server, and after changing a client configuration snippet you must redeploy the client configuration. Sometimes you can refresh instead of restart. In some cases however, you must restart a dependent server after changing a client configuration. For example, changing a MapReduce client configuration marks the dependent Hive server as stale, which must be restarted. The Admin Console displays an indicator when a server must be restarted. In addition, the All Configuration Issues tab on the Home page indicates the actions you must perform to resolve stale configurations.

Continue reading:

Configuration Snippet Types and Syntax

Configuration
Set configuration properties in various configuration files; the property name indicates into which configuration file the configuration will be placed. Configuration files have the extension .xml or .conf.

For example, there are several configuration snippets for the Hive service. One Hive configuration snippet property is called the HiveServer2 Advanced Configuration Snippet for hive-site.xml; configurations you enter here are inserted verbatim into the hive-site.xml file associated with the HiveServer2 role group.

To see a list of configuration snippets that apply to a specific configuration file, enter the configuration file name in the Search field in the top navigation bar. For example, searching for mapred-site.xml shows the configuration snippets that have mapred-site.xml in their name.

Some configuration snippet descriptions include the phrase for this role only. These configurations are stored in memory, and only inserted to the configuration when running an application from Cloudera Manager. Otherwise, the configuration changes are added to the configuration file on disk, and are used when running the application both from Cloudera Manager and from the command line.

Syntax:

<property>
  <name>property_name</name> 
  <value>property_value</value>
</property>
For example, to specify a MySQL connector library, put this property definition in that configuration snippet: 
<property>
  <name>hive.aux.jars.path</name> 
  <value>file:///usr/share/java/mysql-connector-java.jar</value>
</property>
Environment
Specify key-value pairs for a service, role, or client that are inserted into the respective environment.

One example of using an environment configuration snippet is to add a JAR to a classpath. Place JARs in a custom location such as /opt/myjars and extend the classpath using the appropriate service environment configuration snippet. The value of a JAR property must conform to the syntax supported by its environment. See Setting the class path.

Do not place JARs inside locations such as /opt/cloudera or /usr/lib/{hadoop*,hbase*,hive*} that are managed by Cloudera because they are overwritten at upgrades.

Syntax:

key=value
For example, to add JDBC connectors to a Hive gateway classpath, add 
AUX_CLASSPATH=/usr/share/java/mysql-connector-java.jar:\
/usr/share/java/oracle-connector-java.jar
or 
AUX_CLASSPATH=/usr/share/java/*
to Gateway Client Advanced Configuration Snippet for hive-env.sh.
Logging
Set log4j properties in a log4j.properties file.

Syntax:

key1=value1
key2=value2
For example: 
log4j.rootCategory=INFO, console max.log.file.size=200MB
max.log.file.backup.index=10
Metrics
Set properties to configure Hadoop metrics in a hadoop-metrics.properties or hadoop-metrics2.properties file.

Syntax:

key1=value1
key2=value2
For example: 
*.sink.foo.class=org.apache.hadoop.metrics2.sink.FileSink
namenode.sink.foo.filename=/tmp/namenode-metrics.out
secondarynamenode.sink.foo.filename=/tmp/secondarynamenode-metrics.out
Whitelists and blacklists
Specify a list of host addresses that are allowed or disallowed from accessing a service.

Syntax:

host1.domain1 host2.domain2

Setting an Advanced Configuration Snippet

  1. Click a service.
  2. Click the Configuration tab.
  3. In the Search box, type Advanced Configuration Snippet.
  4. Choose a property that contains the string Advanced Configuration Snippet (Safety Valve).
  5. Specify the snippet properties. If the snippet is an XML file, you have the option to use a snippet editor (the default) or an XML text field:
    • Snippet editor

      1. Click to add a property. Enter the property name, value, and optional description. To indicate that the property value cannot be overridden by another , select the Final checkbox.
    • XML text field - Enter the property name, value, and optional description in as XML elements. 
      <property>
  <name>name</name>
  <value>property_value</value>
  <final>final_value</final>
</property>
      To indicate that the property value cannot be overridden, specify <final>true</final>.
    To switch between the editor and text field, click the View Editor and View XML links at the top right of the snippet row.
  6. Enter a Reason for change, and then click Save Changes to commit the changes.
  7. Restart the service or role or redeploy client configurations as indicated.

Setting Advanced Configuration Snippets for a Cluster or Clusters

  1. Do one of the following
    • specific cluster
      1. On the Home > Status tab, click a cluster name.
      2. Select Configuration > Advanced Configuration Snippets.
    • all clusters
      1. Select Configuration > Advanced Configuration Snippets.
  2. Specify the snippet properties. If the snippet is an XML file, you have the option to use a snippet editor (the default) or an XML text field:
    • Snippet editor

      1. Click to add a property. Enter the property name, value, and optional description. To indicate that the property value cannot be overridden by another , select the Final checkbox.
    • XML text field - Enter the property name, value, and optional description in as XML elements. 
      <property>
  <name>name</name>
  <value>property_value</value>
  <final>final_value</final>
</property>
      To indicate that the property value cannot be overridden, specify <final>true</final>.
    To switch between the editor and text field, click the View Editor and View XML links at the top right of the snippet row.
  3. Enter a Reason for change, and then click Save Changes to commit the changes.
  4. Restart the service or role or redeploy client configurations as indicated.