Replicating Data

A step by step guide on how to start replicating data between Kafka clusters with SRM.

Installing and starting SRM on your cluster does not automatically start data replication. In order to start the replication process, you need to update the allowlists with the srm-control tool.

In Cloudera Manager, verify that the SRM driver role is started and is in good health.

Verify that SRM is configured correctly. Make sure that connection information for each Kafka cluster is added as well as at least one source->target replication is specified and enabled.

  1. Update the topics allowlist to start data replication.
    srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add [TOPIC1],[TOPIC2]
    
  2. Verify that the topics have been added to the allowlist.
    srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --list

The topics you specify with the --add option are added to the topic allowlist and are replicated to the specified target cluster.