Cloudera Docs

Configuring properties not exposed in Cloudera Manager

There are number of configuration properties that Streams Replication Manager accepts, but are not exposed directly in Cloudera Manager. You can configure these properties with the Streams Replication Manager's Replication Configs property. Additionally these properties can be configured on a top, cluster, or replication level.

In addition to the configuration properties exposed directly for configuration through Cloudera Manager, Streams Replications Manager accepts a number of additional Streams Replication Manager specific properties as well as Kafka properties available in the version of Kafka that you are using. Properties not exposed directly in Cloudera Manager can be set through the Streams Replication Manager's Replication Configs property. For a comprehensive list of SRM properties not available in Cloudera Manager, see Configuration Properties Reference for Properties not Available in Cloudera Manager. For a comprehensive list of Kafka client properties, see the upstream Apache Kafka documentation.

The configuration properties that you add to Streams Replication Manager's Replication Configs can be prefixed. These prefixes allow you to exercise control over when and where a configuration should be used by SRM. The prefixes and the levels of configuration they correspond to are the following:
  • Top level (no prefix): Top level or global configuration is achieved by adding the property on its own, without a prefix. A configuration like this will be used For example:
    replication.factor=3
  • Cluster level (prefix with cluster alias): Cluster level configuration can be achieved by prefixing the configuration property with a cluster alias specified in Streams Replication Manager Cluster alias. For example:
    primary.replication.factor=3
  • Replication level (prefix with replication):Replication level configuration can be achieved by prefixing the configuration property with the name of the replication. For example:
    primary->secondary.replication.factor=3

In addition to these prefixes, there also exist two other prefixes that can be used to set configuration properties for the dedicated Kafka Connect REST servers set up by the SRM driver for each replication. These are ***ALIAS***->***ALIAS***.worker. and listeners.https.. For more information on the usage of these prefixes and the configuration of the dedicated REST servers, see Configuring replication specific Kafka Connect REST servers.

Make sure that cluster aliases and replications are configured. Otherwise cluster or replication level configuration is not possible.

  1. In Cloudera Manager, select Streams Replication Manager.
  2. Go to Configuration.
  3. Configure properties not exposed in Cloudera Manager:
    1. Find the Streams Replication Manager's Replication Configs property.
    2. Click the add button and add new lines for each additional property you want to configure.
    3. Add configuration properties. For example:
      replication.factor=3
  4. Enter a Reason for change, and then click Save Changes to commit the changes.
  5. Restart Streams Replication Manager.
Configuration properties not directly exposed in Cloudera Manager are configured.