Enabling reverse checkpointing

Learn how to enable the reverse checkpointing feature. This feature minimizes message duplication in failback scenarios.

The reverse checkpointing feature expects that the DefaultReplicationPolicy is used, and so replica topic names are prefixed with the source cluster alias. For this reason, reverse checkpointing is incompatible with prefixless replication through the IdentityReplicationPolicy. Make sure that the Enable Prefixless Replication property is deselected in Cloudera Manager.

In Cloudera Manager, select Streams Replication Manager.
Go to Configuration.
Find and select the Reverse Checkpointing Enabled property in Cloudera Manager.
Enable bidirectional replication.
1. Find the Streams Replication Manager's Replication Configs property.
2. Click the add button, and add the following lines:
```
[***SOURCE CLUSTER ALIAS***]->[***TARGET CLUSTER ALIAS***]=true
[***TARGET CLUSTER ALIAS***]->[***SOURCE CLUSTER ALIAS***]=true
```
Enter a Reason for change, and then click Save Changes to commit the changes.
Restart Streams Replication Manager.
Open a terminal on a host where the Streams Replication Manager client is installed.
Using the srm-control tool, perform the following steps:
1. Make sure that the required topics are added to the replication allowlist.
```
srm-control topics --source [***SOURCE CLUSTER ALIAS***] --target [***TARGET CLUSTER ALIAS***] --add [***TOPIC***]
```
2. Add the topics you want to be reverse checkpointed.
```
srm-control reverse-checkpointed-topics --source [***TARGET CLUSTER ALIAS***] --target [***SOURCE CLUSTER ALIAS***] --add [***SOURCE CLUSTER ALIAS***]__[***TOPIC***]
```
  Replace [***TOPIC***] with the name of the original topic. The full value to add is the replica topic name as it appears on the target (backup) cluster, which is the original topic name prefixed with the source cluster alias. For example, if your primary cluster alias is [***P***] and your original topic is Payments, add P.Payments.
3. Enable consumer group replication in both directions.
```
srm-control groups --source [***SOURCE CLUSTER ALIAS***] --target [***TARGET CLUSTER ALIAS***] --add [***CONSUMER GROUP***]
```
```
srm-control groups --source [***TARGET CLUSTER ALIAS***] --target [***SOURCE CLUSTER ALIAS***] --add [***CONSUMER GROUP***]
```

Reverse checkpointing is enabled, and offsets are tracked for the specified topics and groups in both directions.

By default, consumer offset sync from checkpoints to the __consumer_offsets topic is not automatic. To enable automatic consumer offset syncing, configure the following properties in the Streams Replication Manager's Replication Configs property in Cloudera Manager:

sync.group.offsets.enabled — Enables automatic consumer offset syncing from checkpoints.
sync.group.offsets.interval.seconds — Controls the interval at which offsets are synced.

The following properties are also essential for controlling message duplication after a failover or failback:

offset.lag.max — Controls the maximum offset lag. Affects the amount of records reprocessed after a failover or failback.
offset.flush.interval.ms — Controls how often an offset sync is produced, even if offset.lag.max is not reached.