Replicating Data

A step by step guide on how to start replicating data between Kafka clusters with SRM.

Installing and starting SRM on your cluster does not automatically start data replication. In order to kick off replication, you need to update the whitelists with the srm-control tool.

In Cloudera Manager, verify that the SRM driver role is started and is in good health.

Verify that SRM is configured correctly. Make sure that connection information for each Kafka cluster is added as well as at least one source->target replication is specified and enabled.

  1. Update the topics whitelist to start data replication.
    srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add [TOPIC1],[TOPIC2]
    
  2. Verify that the topics have been added to the whitelist.
    srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --list

The topics you specify with the --add option are added to the topic whitelist and are replicated to the specified target cluster.