Custom Configuration
Minimum Required Role: Configurator (also provided by Cluster Administrator, Limited Cluster Administrator , and Full Administrator)
Cloudera Manager exposes properties that allow you to insert custom configuration text into XML configuration, property, and text files, or into an environment. The naming convention for these properties is: XXX Advanced Configuration Snippet (Safety Valve) for YYY or XXX YYY Advanced Configuration Snippet (Safety Valve), where XXX is a service or role and YYY is the target.
The values you enter into a configuration snippet must conform to the syntax of the target. For an XML configuration file, the configuration snippet must contain valid XML property definitions. For a properties file, the configuration snippet must contain valid property definitions. Some files simply require a list of host addresses.
The configuration snippet mechanism is intended for use in cases where there is configuration setting that is not exposed as a configuration property in Cloudera Manager. Configuration snippets generally override normal configuration. Contact Cloudera Support if you are required to use a configuration snippet that is not explicitly documented.
Service-wide configuration snippets apply to all roles in the service; a configuration snippet for a role group applies to all instances of the role associated with that role group.
Server and client configurations have separate configuration snippets. In general after changing a server configuration snippet you must restart the server, and after changing a client configuration snippet you must redeploy the client configuration. Sometimes you can refresh instead of restart. In some cases however, you must restart a dependent server after changing a client configuration. For example, changing a MapReduce client configuration marks the dependent Hive server as stale, which must be restarted. The Admin Console displays an indicator when a server must be restarted. In addition, the All Configuration Issues tab on the Home page indicates the actions you must perform to resolve stale configurations.
Configuration Snippet Types and Syntax
- Configuration
- Set configuration properties in various configuration files; the property name
indicates into which configuration file the configuration will be placed. Configuration
files have the extension
.xml
or.conf
.For example, there are several configuration snippets for the Hive service. One Hive configuration snippet property is called the HiveServer2 Advanced Configuration Snippet for hive-site.xml; configurations you enter here are inserted verbatim into the
hive-site.xml
file associated with the HiveServer2 role group.To see a list of configuration snippets that apply to a specific configuration file, enter the configuration file name in the Search field in the top navigation bar. For example, searching for
mapred-site.xml
shows the configuration snippets that havemapred-site.xml
in their name.Some configuration snippet descriptions include the phrase for this role only. These configurations are stored in memory, and only inserted to the configuration when running an application from Cloudera Manager. Otherwise, the configuration changes are added to the configuration file on disk, and are used when running the application both from Cloudera Manager and from the command line.
Syntax:
<property> <name>property_name</name> <value>property_value</value> </property>
For example, to specify a MySQL connector library, put this property definition in that configuration snippet:<property> <name>hive.aux.jars.path</name> <value>file:///usr/share/java/mysql-connector-java.jar</value> </property>
If you do not set these properties in the correct syntax, then Cloudera Manager cannot parse the input XML and you may see an "Invalid XML" error as shown in the following image: - Environment
- Specify key-value pairs for a service, role, or client that are inserted into the
respective environment.
One example of using an environment configuration snippet is to add a JAR to a classpath. Place JARs in a custom location such as
/opt/myjars
and extend the classpath using the appropriate service environment configuration snippet. The value of a JAR property must conform to the syntax supported by its environment.Do not place JARs inside locations such as
/opt/cloudera
or/usr/lib/
{hadoop*,hbase*,hive*}
that are managed by Cloudera because they are overwritten at upgrades.Syntax:
key=value
For example, to add JDBC connectors to a Hive gateway classpath, addAUX_CLASSPATH=/usr/share/java/mysql-connector-java.jar:\ /usr/share/java/oracle-connector-java.jar
orAUX_CLASSPATH=/usr/share/java/*
to Gateway Client Advanced Configuration Snippet for hive-env.sh. - Logging
- Set log4j properties in a
log4j.properties
file.Syntax:
key1=value1 key2=value2
For example:log4j.rootCategory=INFO, console max.log.file.size=200MB max.log.file.backup.index=10
- Metrics
- Set properties to configure Hadoop metrics in a
hadoop-metrics.properties
orhadoop-metrics2.properties
file.Syntax:
key1=value1 key2=value2
For example:*.sink.foo.class=org.apache.hadoop.metrics2.sink.FileSink namenode.sink.foo.filename=/tmp/namenode-metrics.out secondarynamenode.sink.foo.filename=/tmp/secondarynamenode-metrics.out
- Whitelists and blacklists
- Specify a list of host addresses that are allowed or disallowed from accessing a
service.
Syntax:
host1.domain1 host2.domain2