Configuring Clusters and Replications

Learn how to add and specify clusters and replications .

In order for SRM to replicate data to or from clusters, you need to:
  1. Specify cluster aliases
  2. Specify cluster connection information
  3. Add and enable replications

Specifying your clusters and enabling replications does not start replication of data itself. When a clusters and replications are added with the following method to the configuration, SRM will connect and set up communication with them, but will not automatically replicate any data. To start replicating data you need to specify which topics to replicate with the srm-control command line tool.

Use the following steps as reference when you want to add new clusters or replications to your deployment.

  1. In Cloudera Manager select Streams Replication Manager.
  2. Go to Configuration.
  3. Specify cluster aliases:
    1. Find the Streams Replication Manager Cluster alias. property.
    2. Add a comma delimited list of cluster aliases. For example:
      primary, secondary
      Cluster aliases are arbitrary names defined by the user. Aliases specified here are used in other configuration properties and with the srm-control tool to refer to the clusters added for replication.
  4. Specify cluster connection information:
    1. Find the streams.replication.manager's replication configs property.
    2. Click the add button and add new lines for each cluster alias you have specified in the Streams Replication Manager Cluster alias. property
    3. Add connection information for your clusters. For example:
      primary.bootstrap.servers=primary_host1:9092,primary_host2:9092,primary_host3:9092
      secondary.bootstrap.servers=secondary_host1:9092,secondary_host2:9092,secondary_host3:9092

      Each cluster has to be added to a new line. If a cluster has multiple hosts, add them to the same line but delimit them with commas.

  5. Add and enable replications:
    1. Find the streams.replication.manager's replication configs property.
    2. Click the add button and add new lines for each unique replication you want to add and enable.
    3. Add and enable your replications. For example:
      primary->secondary.enabled=true
      secondary->primary.enabled=true
      
  6. Enter a Reason for change, and then click Save Changes to commit the changes.
  7. Restart Streams Replication Manager.
Replicating data to or from the specified clusters is now possible.
Use the srm-control tool to kick off replication by adding topics or groups to the whitelist.