Known Issues in Streams Replication Manager
Learn about the known issues in Streams Replication Manager, the impact or changes to the functionality, and the workaround.
- CDPD-22089: SRM does not sync re-created source topics until the offsets have caught up with target topic
- Messages written to topics that were deleted and re-created are not replicated until the source topic reaches the same offset as the target topic. For example, if at the time of deletion and re-creation there are a 100 messages on the source and target clusters, new messages will only get replicated once the re-created source topic has 100 messages. This leads to messages being lost.
- CDPD-14019: SRM may automatically re-create deleted topics
auto.create.topics.enableis enabled, deleted topics are automatically recreated on source clusters.
- Prior to deletion, remove the topic from the topic allowlist
srm-controltool. This prevents topics from being re-created.
srm-control topics --source [SOURCE_CLUSTER] --target [TARGET_CLUSTER] --remove [TOPIC1][TOPIC2]
- CDPD-13864 and CDPD-15327: Replication stops after the network configuration of a source or target cluster is changed
- If the network configuration of a cluster which is taking part in a replication flow is changed, for example, port numbers are changed as a result of enabling or disabling TLS, SRM will not update its internal configuration even if SRM is reconfigured and restarted. From SRM’s perspective, it is the cluster identity that has changed. SRM cannot determine whether the new identity corresponds to the same cluster or not, only the owner or administrator of that cluster can know. In this case, SRM tries to use the last known configuration of that cluster which might not be valid, resulting in the halt of replication.
- The internal topic storing the configuration of SRM can be
deleted. After a restart SRM will re-create and re-populate it with the configuration data
loaded from its property file. The topic is hosted on the target cluster of the
replication flow. The topic name is:
mm2-configs.[SOURCE_ALIAS].internal. However, changing a replicated cluster's identity is generally not recommended.
- SRM Cannot Replicate ACLs to or from Kafka Clusters that are Configured with Sentry
- When Sentry contains Kafka authorization
policies for any
ConsumerGroupresource, SRM cannot replicate authorization rules from one Kafka cluster to another in environments where Sentry is enabled. This is due to a Kafka resource conversion error in Sentry. For more information regarding the underlying Sentry issue, see SENTRY-2535 and the Kafka known issues in CDH Known Issues.
- Disable authorization policy synchronization in SRM. This
can be achieved by setting the
sync.topic.acls.enabledproperty to false.
- CDPD-11709: Blacklisted topics appear in the list of replicated topics
- If a topic was originally replicated but was later disallowed
(blacklisted), it will still appear as a replicated topic under the
/remote-topicsREST API endpoint. As a result, if a call is made to this endpoint, the disallowed topic will be included in the response. Additionally, the disallowed topic will also be visible in the SMM UI. However, it's Partitions and Consumer Groups will be 0, its Throughput, Replication Latency and Checkpoint Latency will show N/A.