Replicating Data

A step by step guide on how to start replicating data between Kafka clusters with SRM.

In SRM a replication can be set up with the help of the srm-driver and srm-control tools.

Verify that SRM is configured correctly. Make sure that connection information for each Kafka cluster is added as well as at least one source->target replication is specified and enabled.

  1. Launch SRM with the srm-driver tool.
    srm-driver
    Alternatively, if you want to use a properties file other than the default, you can do so with the --config option.
    srm-driver --config [path/to/srm.properties]

    The driver connects to the clusters defined in your configuration. However, data replication is not yet started.

  2. Update the topics whitelist to start data replication.
    srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add [TOPIC1],[TOPIC2]
    

    Alternatively, if you want to use a properties file other than the default, you can do so with the --config option.

    srm-control --config [path/to/srm.properties] topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --add [TOPIC1],[TOPIC2]
    

The topics you specify with the --add option are added to the topic whitelist and are replicated to the specified target cluster.